Florida A & M Office of University Assessment 2013-2014 ... 2013-2014 Annual...the OUA realized several key accomplishments over the course of the 2013-2014 academic year. Of significance,

Florida A & M

University

Office of University

Assessment

2013-2014 Annual

Institutional

Assessment Report

A Title III Co-Sponsored Activity

Pending Approval

December 2014 page 2

Table of Contents Executive Summary .............................................................................................................. 3

Message from the Director .................................................................................................... .6

Introduction ...................................................................................................................... .7

Overview of Annual Report ................................................................................................... 7

OUA Mission Statement ....................................................................................................... 7

FAMU Assessment Philosophy ................................................................................................ 7

OUA Goals and Objectives ................................................................................................... 8

Instructional Programs Assessment Activities ............................................................................... 8

Trends in Instructional Programs Assessment Plans/Reports Submissions .......................................... 9

Academic Learning Compacts .............................................................................................. 11

Administrative and Educational Support Units Activities ................................................................ 13

Trend in Administrative and Educational Support Units Plans/Reports Submission ............................ 13

Institutional Level Assessment Activities ................................................................................... 14

Exit Survey .................................................................................................................... 14

ETS Proficiency Profile ..................................................................................................... 17

National Survey of Student Engagement ................................................................................ 22

Office of University Assessment Activities/Updates ..................................................................... 26

STARS Assessment Model ................................................................................................... 26

OUA Focused Objectives for 2014-2015 Assessment Cycle ........................................................... 27

Workshops/Training Sessions .............................................................................................. 28

Assessment Committees Activities/Updates ............................................................................... 29

General Education Assessment Committee Activities/Updates ..................................................... 29

Institutional Level Assessment Committee Activities/Updates ..................................................... 31

Quality Enhancement Plan Committee Activities/Updates .......................................................... 32

Licensure/Certification Examination Pass Rates .......................................................................... 34

Summary of Key University Assessment Findings and Recommendations ........................................... 39

Office of University Assessment Team ..................................................................................... 42


Executive Summary

The Office of University Assessment (OUA) at Florida Agricultural and Mechanical University is committed to

promoting a culture of evidence at FAMU through the implementation of a manageable and sustainable

assessment process that leads to quality program and service improvement. The OUA achieves this mission by

providing support to all academic programs and administrative and educational support units by building capacity

for the use of best practices in assessment and continuous quality improvement. Through our concerted efforts,

the OUA realized several key accomplishments over the course of the 2013-2014 academic year. Of significance,

is the improvement in the timeliness’ and quality of the assessment reports and plans submitted by Instructional

Programs (IPs) and Administrative and Educational Support Units (ADESUs) for the 2012-2013 and 2013-2014

academic year. As of February 2014, the OUA collected and documented 100% (n=104) of the assessment reports

and plans from (IPs) and 92% (n=45) and 94% (n=47) of the assessment reports and plans from (ADESUs)

respectively, and 100% (n=13) of the assessment reports and plans for both research and community service. The

success relative to the submissions of reports and plans is largely attributed to targeted efforts over the course

of the year (i.e. site visitations, workshops etc.) in creating more awareness of the important role that

assessment plays in creating a culture of evidence and continuous improvement for the university. During the

2013-2014 academic year, the OUA transitioned from the FAMOUS Assessment Approach to the FAMU STARS

Assessment Approach, a five step approach to quality assessment designed to provide tighter alignment of

outcomes to the University’s Strategic Initiatives and Presidents goals. The FAMU STARS Assessment Approach

was also designed to allow for the aggregation and disaggregation of outcomes and outcome types at the program,

unit and division level.

Collaborative Efforts

In an effort to build capacity for assessment that matters, the OUA facilitated Assessment training sessions with

all Assessment Coordinators and others responsible for assessment within their units across the university. These

training sessions afforded participants the opportunity to engage in meaningful ways with the OUA as we work

to broaden our impact on assessment for the university. Over the course of the 2013-2014 academic year a total

of (35) training sessions were conducted. There was a general consensus among the participants that the sessions

were helpful in increasing their understanding of best practices in assessment. Based on the success of this

approach in engaging the university community around quality assessment, the OUA will continue this format

during the course of the next assessment cycle. Following is a summary of key institutional level assessment

activities and recommendations for improvement.

Institutional Level Assessment

The assessment activities conducted at the institutional level included the administration of the ETS Proficiency

Profile (EPP) for entering freshmen and graduating seniors, the Exit Survey to graduating seniors, the National

Survey of Student Engagement (NSSE) and focus group exercises conducted with current students relative to

general education outcomes. Assessment activities also included the review and submission of Academic Learning

Compacts to the Board of Governors. The OUA conducted a comprehensive review of these results and advanced

recommendations relative to each of the aforementioned areas. A sample of the results from the assessment

tools and activities follow.


Exit Survey

Over the course of the 2013-2014 academic year, (N=2,197) students graduated from the university. Of this

population, (n=2,042) students responded to the survey representing an overall response rate of 92.9%. Consistent

with the findings of previous years, graduating students have consistently provided low ratings relative to their

level of satisfaction with the Availability of Parking (26.35%) and Financial Aid Processes (32.31%). Areas with

the highest level of satisfaction were the value of your FAMU degree (89.26%), University Libraries (87.97%), and

Size of Classes (83.73%). Considering these findings, the respective departments and units within the university

need to identify and implement programmatic strategies to combat these areas of opportunity. While the

majority of graduating students were overall satisfied with their educational outcomes and their major field of

study, in keeping with a culture of continuous improvement, we need to continually assess these areas in an

effort to stay current and competitive in an environment that is constantly changing.

ETS Proficiency Profile

In an effort to evaluate General Education learning outcomes, FAMU students are required to take the ETS

Proficiency Profile at entry (freshmen) and exit (senior) levels. In the Fall of 2013, (n=334) incoming freshmen

attempted to take the EPP test. Results were calculated for (n=308) of those students who completed 75% or

more of the test items. Of those students that completed the EPP, (n=1) student had a perfect score. The mean

score for incoming freshmen was 431.26 (SD=17.75) based on a 400-500 range. In the Spring 2014, (n=600)

graduating seniors took the EPP test. Results were calculated for (n=537) of those students. Sixty-three students

completed less than 75% of the test and as such, their scores were excluded from the analysis. Results show that

the total mean score for the graduating seniors was 430.56 (SD=18.31) based on a 400-500 range. As can be

discerned from these results, graduating seniors performed marginally below their freshman counterparts.

Further analysis of the results also confirms that our students are entering the university with deficiency in

critical thinking skills, especially in the area of ‘Evaluate Hypothesis’. Additionally, reading has become an area

of concern. It is important that more emphasis is placed on these skills in an effort to transform students through

the FAMU experience.

National Survey of Student Engagement

The 2014 administration of the National Survey of Student Engagement (NSSE) represents the third time that this

instrument has been administered at FAMU. The instrument was administered to first-year and senior students'

participating in programs and activities that the institutions provide for their learning and personal development.

The 2014 NSSE was launched on February 13, 2014 and closed on April 15, 2014 with an overall response rate of

25% or (n=835) respondents. When compared to the average response rate for the State University System (SUS)

in Florida, FAMU’s response rate was four percentage points higher. Based on analysis of the findings, FAMU

first-year students and seniors perceived higher levels of engagement with faculty when compared to other

students from institutions in the SUS, Carnegie Classification and NSSE 2013 & 2014 taken as a group. FAMU

seniors also perceived higher levels of learning with peers as described by the engagement indicator

(Collaborative Learning). Conversely, FAMU first-year students perceived lower levels of engagement in the

areas of Discussions with Diverse Other, Effective Teaching Practices and Quality of Interactions. While, FAMU

seniors perceived lower levels of engagement with respect to Quality of Interactions the majority of the


respondent sample were satisfied with their overall experiences at FAMU. The results suggest that much more

must be done in an effort to address areas of opportunity especially with respect to activities aligned with the

following engagement indicators: Discussion with Diverse Others, Effective Teaching Practices, and Quality of

Interactions.

Recommendations for Improvement

The Office of University Assessment continues to play a critical role in enhancing institutional effectiveness and

contributing to an environment of continuous improvement and excellence in the operations and outcomes of

the university. The following recommendations are advanced based on outcomes of the assessment processes at

Florida Agricultural and Mechanical University.

• Require all students in capstone courses to take the ETS Proficiency Profile as a component of the course

grade.

• Recommend all programs to emphasize the competencies and skills measured by the ETS Proficiency

Profile in the curriculum with the goal of improving seniors’ performance at or beyond the national

average.

• Units with low satisfaction ratings on the Exit Survey need to use their findings in developing and

implementing programmatic strategies to make programmatic improvements. It is strongly recommended

that these units employ the FAMU STARS assessment process to document their efforts.

• Students benefit and are more satisfied in supportive settings that cultivate positive relationships among

students, faculty and staff. Based on this premise, it is strongly recommended that the university develop

Point of Service (POS) instruments to capture in real time students perceptions of their experiences

interfacing with key administrative offices of the university to include Financial Aid, Registrar Office,

Parking Services, and the Housing Office.

• Develop a program, in collaboration with the writing center, to enhance student writing skills. From

freshman level to senior level, student writing continues to reflect deficiencies in communication.

• Develop and implement strategies to improve the quality of student interactions with academic advisors.

Develop mechanisms to track the quality of these interactions.

• Degree programs for which students are required to take licensure examinations should review and revise

curriculum and assessment mapping for tighter alignment between instruction and the competencies and

skills being tested on examinations.


Message from the Director

Greetings,

The Office of University Assessment staff has prepared the 2013-2014 Annual Assessment Report for your review.

It is my hope that you will find this report to be informative, enlightening, and empowering. The Office of

University Assessment was established in 1994 and has coordinated the University’s assessment activities since

that time. January, 2015 marks the second year of my tenure as the Director of University Assessment, and this

last year has been a very exciting one. During the course of the 2013-2014 academic year, the OUA welcomed a

new Coordinator for Computer Applications (Mr. George Pinkney IV). In our ongoing effort to provide the highest

quality of services to the university community, we continue to review and revise the assessment processes and

procedures. We have strengthened systems and galvanized the collaborative efforts with our university partners

to promote and maintain a culture of evidence across the entire institution.

This year we are proud to have introduced to the university community the new and improved approach to

assessment which has been labeled the FAMU STARS Assessment Approach. You will find information on STARS

and other key and important institutional level data summarized in this report as a way of quantifying and

qualifying the impact of our instructional programs and support services on the academic experience of FAMU

students. The STARS Assessment Approach is designed to promote more effective alignment of assessment

practices to the University Strategic Plan, President’s goals, and other strategic initiatives while promoting

excellence in the operation of all programs and services. Our wish is to maintain an assessment process that

empower decision makers with key performance data that promote the true “excellence with caring” experience

that FAMU students deserve.

Please take the opportunity to review the 2013-2014 Annual Assessment Report with special attention to the

elements that are most directly related to the work of your unit(s). I also implore you to use information from

this report to drive conversations about effective operation and to inform strategies for ongoing continuous

improvement.

The Office of University Assessment stands ready to support you and your assessment activities in any way that

we can. Do not hesitate to call on us.

Sincerely,

Mark Howse, Ph.D.


Introduction

Overview of Annual Report

The 2013-2014 Office of University Assessment (OUA) Institutional Assessment Report is designed to provide you

key institutional level data as a way of quantifying and qualifying the impact of our instructional programs and

support services on the academic experience of FAMU students. Over the course of this year, the OUA continued

to provide leadership and expertise in assessment as a way of helping units to build capacity for effectively

assessing themselves based on industry standards and best practices for meaningful assessment. This report is

divided into several key areas which will provide you insights into the findings of key institutional level assessment

activities to include the Exit Survey, the ETS Proficiency Profile and the 2014 administration of the National

Survey of Student Engagement (NSSE). Also discussed in this annual report are trends in Instructional Programs

(IPs) and Administrative and Educational Support Units (ADESUs) submission rates for assessment reports and

plans. You will also find a section which provides an overview of the recently implemented FAMU STARS

assessment approach. We discuss our goals and objectives for the 2014-2015 assessment cycle in addition to

providing summary data on workshops and training sessions conducted over the course of this year. There is

great deal of information presented in this annual report and it is our hope that you will find this report

meaningful and empowering in efforts to facilitate the true “excellence with caring” experience that our

students not only deserve.

OUA Mission Statement

The mission of the Office of University Assessment is to promote a culture of evidence at FAMU through the

implementation of a manageable and sustainable assessment process that leads to quality program and service

improvement.

In support of the University’s mission, the Office of University Assessment at FAMU promotes excellence in

student learning and services across academic programs, administrative, and educational support units. Through

ongoing training and assessment support services, the Office operationalizes the development of a university-

wide culture of evidence wherein intentional, effective, and efficient programs and services are established and

improved in a manner that embodies the University's motto of “excellence with caring.”

FAMU Assessment Philosophy

The philosophy of assessment at FAMU is "to foster a culture of continuous improvement of program offerings

and support services to ensure student success and institutional effectiveness.” Assessment is grounded in the

institutional values set forth in the mission statement and focuses primarily on improving student learning both

within and outside the classroom. Our approach is rooted in a comprehensive and collaborative process that

utilizes data to improve the overall student experience.

FAMU defines assessment as “a systematic process of gathering, analyzing, interpreting and documenting the

effectiveness of the teaching and learning processes to ensure that expectations and standards are met in

fulfilling the mission of the university. The process also includes monitoring and enhancing the administrative


and educational support structure of the university that leads to the continuous improvement of FAMU’s academic

programs and administrative and educational support services”.

OUA Goals and Objectives

Goal 1: Academic Improvement –Strengthen the distinctive quality of academic programs, producing graduates

that demonstrate the 21st-century skills needed to compete in an increasingly diverse and global society.

Objective 1.1 Enhance the capacity for development of student learning outcomes aligned to

appropriate measures and reflective of best practices for each academic discipline.

Objective 1.2 Analyze and summarize key student learning data for review by decision makers and other

stakeholders.

Goal 2: Service Improvement – Strengthen the quality of administrative and educational support services

through the development of appropriate performance outcomes and measures.

Objective 2.1 Support the development of performance-based strategies and measures to monitor and

document efficiency and effectiveness of all support services.

Objective 2.2 Summarize key performance data related to administrative and educational support unit

for review by decision makers and other stakeholders.

Goal 3: Accountability – Maintain effective and efficient assessment processes that comply with all internal

and external assessment requirements.

Objective 3.1 Facilitate the implementation of assessment practices that meet or exceed the

accountability requirements set forth by the Florida Legislature, Board of Trustees, and other external

accountability bodies.

Objective 3.2 Promote alignment between assessment planning and University strategic initiatives.

Goal 4: Communication - Raise stakeholder awareness of student learning and key performance outcomes

through the dissemination of institutional assessment data and recommendations for improvement.

Objective 4.1 Communicate student learning outcomes and other assessment information to students,

faculty, staff, administrators, and other stakeholders.

Objective 4.2 Maintain a framework for meaningful, timely feedback and recommendations related to

key assessment information.

Instructional Programs Assessment Activities

This section of the 2013-2014 OUA Annual Report highlights trends in the submission rates for assessment reports

from Instructional Programs and Colleges and Schools relative to Research and Community/Public Service. Also

discussed in this section is a status on the Academic Learning Compacts (ALCs) which are state-mandated

requirements for baccalaureate degree programs offered by public universities in the State of Florida University

System.


Trends in Instructional Programs Assessment Report Submissions

This section highlights assessment activities for instructional programs, colleges and schools. Assessment reports

are designed to highlight progress made in assessment efforts at the program and college/school level. The OUA

advocates that each unit use their assessment report and results to celebrate achievements as well as to identify

potential areas for future improvement. The Office of University Assessment annually reviews and provides

feedback on submitted reports.

For instructional areas, assessment reports are collected for three general areas: Instructional Programs,

Research, and Community/Public Service. In 2013-2014, assessment reports were collected for (n=111)

programs. Three of the programs were newly established. For research and community/public service,

assessment reports are collected for all (n=13) colleges/schools. Table 1 provides a summary of the submission

rates for assessment reports for instructional programs, colleges/schools. Please note that the collection of

assessment reports for Research and Community/Public Service did not begin until 2008. As can be discerned

from these results, there is much variability in submission rates.

Table 1

Assessment Report Submission Rates for Instructional Programs, Colleges and Schools

2005

-

2006

2006

-

2007

2007

-

2008

2008

-

2009

2009

-

2010

2010

-

2011

2011

-

2012

2012

-

2013

2013

-

2014

Submission Rate for Instructional Programs

(IPs) Assessment Reports 94% 94% 100% 67% 97% 76% 96% 99% 80%

Number of Programs 108 106 108 108 107 107 112 105 111

Submission Rate for Colleges/Schools

Assessment Reports for Research 92% 67% 83% 46% 100% 69%

Number of Colleges/Schools 12 12 12 13 13 13

Submission Rate for Colleges/Schools

Assessment Reports for Community/Public

Service

92% 67% 83% 54% 92% 62%

Number of Colleges/Schools 12 12 12 13 13 13

Note: The percentage represented for 2013-2014 are as of December 10, 2014.

Figures 1-3 provide a graphical representation of these results. It should be noted that timely submissions of

assessment reports for Instructional Programs was highest in (2007-2008) when the university was preparing for

the SACS-COC re-accreditation visit. The OUA promotes assessment as a tool for continuous improvement and

not as a mere compliance obligation for re-accreditation. As such, the OUA will continue to hold all units

accountable for timely submissions and encourage units to make assessment a natural part of their operation.


Figure 1

Trends in Assessment Report Submission Rates for Instructional Programs (IPs)

Figure 2

Trends in Assessment Report Submission Rates for Research

94%94%

100%

67%

97%

76%

96%

99%

61%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

92%

67%

83%

46%

100%

54%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2008-2009 2009-2010 2010-2011 2011-2012 2012-2013 2013-2014


Figure 3

Trends in Assessment Report Submission Rates for Community/Public Service

Academic Learning Compacts

Academic Learning Compacts (ALCs) are state-mandated requirements for baccalaureate degree programs

offered by public universities in the State of Florida University System. The ALCs serve to document the expected

learning outcomes to be achieved by every student matriculating through his/her program. The ALCs are

required, at a minimum, to determine the expected student learning outcomes for baccalaureate program

graduates in the areas of:

Content/Discipline knowledge and skills

Communication skills

Critical thinking skills

Documentation of the ALCs and the related results at FAMU are based upon the FAMOUS assessment planning and

implementation model. Periodic measurements of the extent to which student learning has taken place are

accomplished by the use of tools, such as rubrics, products of student work, and other course-embedded

measures. The Student Learning Assessment Report (formerly - ALC Status Report) was submitted on January 24,

2014 to the Board of Governors (BOG).

The BOG regulation 8.016 requires universities to submit a status report on student learning outcomes assessment

for each baccalaureate program. The following elements are key components of the report:

(1) Identification of expected core student learning outcomes.

(2) Posting of expected core student learning outcomes on the Web.

(3) Identification of corresponding assessment tools.

92%

67%

83%

54%

92%

46%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2008-2009 2009-2010 2010-2011 2011-2012 2012-2013 2013-2014


(4) Posting of examples of corresponding assessment tools on the Web.

(5) Development of a system of program assessment/evaluation.

(6) Use of information to improve student achievement and program effectiveness.

The BOG expects each of the before mentioned elements be reviewed for possible improvement with the

exception of new programs early in their review cycles or programs undergoing significant modifications. Table

2 provides a summary report on student learning outcomes submitted to the Board of Governors for 2013-2014.

Table 2 Student Learning Outcomes Progress Report to the Board of Governors (2013-2014)

ALC Status Report

Process/Product Totals

1. Identification of expected core

student learning

outcomes.

2. Core learning expectations posted on the

Web.

3. Identification of corresponding

assessment tools

4. Examples of

corresponding assessment

tools posted on the Web.

5. Development of

a system of program

assessment/ evaluation.

6. Use of information to improve student achievement and

program effectiveness.

CI 54 54 54 53 53 52

PC 0 0 0 0 0 1

NS 0 0 0 1 1 1

CI = Evidence That Process/Product is Completed and in Continuous Improvement Mode. PC = Evidence That Process/Product Is Partially Completed. NS = No Evidence That Process/Product is Started.

Based on the status of programs and product 6 of BOG regulation 8.016 (Use of information to improve student

achievement and program effectiveness) a total of (n=52) out of (N=54) programs have shown evidence that

information is used to improve student learning. This information is gathered and determined from Steps 5&6 in

the FAMOUS Assessment Approach. One program showed evidence that the program has partially started using

information for improvement. This program had only been in existence for one year and as such, only baseline

data was available for use. Next year, they will have comparable data to analyze. As can be discerned from Table

2, one program had no evidence of continuous improvement. This program is new and was approved by the BOG,

however, students will not be accepted until the upcoming fall semester. As such, no data was available to utilize

for program improvement. Figure 4 provides a graphical representation of the use of assessment data to improve

student learning outcomes and overall program quality and effectiveness.

Figure 4

Use of Information to Improve Student Achievement and Program Effectiveness

96%

2%2%CI

PC

NS


Administrative and Educational Support Units Activities

This section of the 2013-2014 OUA Annual Report highlights trends in the submission rates for assessment reports

from Administrative and Educational Support Units (ADESUs).

Trends in Administrative and Educational Support Units Assessment Report Submissions

Information highlighted in this section includes the status of submissions of assessment reports. Full assessment

reports can be found at www.famu.edu/assessment. Assessment reports are meant to highlight progress made

in assessment efforts. It is recommended that each unit use their assessment report and results to celebrate

achievements as well as to identify potential areas for future improvement. The Office of University Assessment

will review your reports to provide constructive feedback, as well as identify particular units for potential

assessment recognition.

Administrative and Educational Support Units are inclusive of departments that offer administrative services and

departments that offer educational support services. Table 3 provides a summary of the submission rates for

assessment reports from years 2005-2014. The percentage represented for 2013-2014 is as of January 5, 2014.

Table 3

Assessment Report Submission Rates for Administrative and Educational Support Units (ADESUs)

2005-

2006

2006-

2007

2007-

2008

2008-

2009

2009-

2010

2010-

2011

2011-

2012

2012-

2013

2013-

2014

Submission Rate for

Administrative Units and

Educational Support

Services

17% 86% 97% 79% 89% 80% 82% 83% 57%

Number of ADESUs 60 58 64 57 55 52 56 52 53

In 2013-2014, assessment reports were collected from 53 departments. Figure 5 provides a graphical

representation of the data reported in Table 3. Consistent with the findings for Instructional Programs, timely

submission of assessment reports tend to increase in years that the university is preparing for accreditation visits.

The OUA promotes assessment for continuous improvement, not for re-accreditation. We will continue to push

for 100% submission and encourage units to make assessment a natural part of their daily routine.

http://www.famu.edu/assessment


Figure 5

Trends in Assessment Report Submission Rates for Administrative and Educational Support Units

Institutional Level Assessment Activities

This section of the 2013-2014 OUA Annual Report highlights key findings from a variety of inventories designed

to provide deeper insights into the experiences, disposition, skills and abilities of FAMU students. Discussed

within this section are key findings from the following instruments: FAMU Exit Survey, ETS Proficiency Profile

and the National Survey of Student Engagement (NSSE). For a comprehensive report of the findings from these

inventories, please contact the Office of University Assessment at [email protected].

Exit Survey

Florida A&M University (FAMU) exit survey is designed to capture graduating students’ perceptions of their

collegiate experiences and future plans. The survey specifically seeks respondents’ perceptions of their

experiences as it relates to learning outcomes, student support services, facilities, availability of services, major

field of study, among other items deemed important to the university. This report provides a summary of data

collected for the 2013-2014 academic year. Over the course of this time period (N=2,197) students graduated

from the university. Of this population, (n=2,042) students responded to the survey representing an overall

response rate of 92.9%. Table 4 provides a comprehensive summary of select items from the exit survey. The

rows highlighted in gray reflect the summary results for each category in the survey. These numbers represent

an average for all items within that category. Items in which average scores are ≤ 60% are italicized in Table 4.

17%

86%

97%

79%

89%

80% 82% 83%

47%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%


Table 4

Summary of Exit Survey Results for 2013-2014

Select Items FAMU Results

Total Respondents

(n)

% Very Satisfied &

Somewhat Satisfied

Student Support Services* 1996 63.25%

Registration Process 2001 76.06%

Financial Aid Process 1996 32.31%

Grade Reporting Process 1994 79.59%

Advising Process 1991 65.04%

Facilities* 1998 78.48%

Classroom facilities 2003 83.08%

Computing facilities 1999 77.39%

Dining facilities 2001 65.27%

Recreational facilities 1998 76.93%

University bookstore 1997 79.67%

University libraries 1995 87.97%

Fitness/Exercise facilities 1996 79.11%

Departments at FAMU* 1997 60.41%

Campus Police 2001 66.62%

Office of Financial Aid 1995 33.63%

Office of Food Services 2000 55.00%

Housing Office 1997 44.97%

Office of Parking Services 1995 42.76%

Registrar's Office 1996 73.40%

Your major program's department office 1999 79.14%

Library Services 1992 87.75%

Availability of Services* 1998 63.17%

Availability of desired classes 2001 69.57%

Availability of parking 1996 26.35%

Size of classes 1997 83.73%

Quality of the food served on campus 2000 59.45%

Student Government Association 2001 52.92%

Campus e-mail account 1992 61.50%

Campus radio station 1993 62.52%

The value of your FAMU degree 2002 89.26%

Educational Outcomes* 1980 92.66%

Communication 1884 94.16%

Critical Thinking 1996 95.34%

Technology literacy 1996 86.47%

Collaboration 1999 94.90%

Ethical Values 1997 91.99%

Life-long Learning 1998 93.09%

Cultural Diversity 1990 92.76%


Select Items FAMU Results

Total Respondents

(n)

% Strongly Agree &

Somewhat Agree

Major Field of Study* 1985 89.23%

I have obtained a sense of competence in my major

field of study. 1989 96.08%

I have acquired the ability to apply theoretical

knowledge to a practical situation. 1985 96.68%

I feel my major field of study was intellectually

challenging. 1985 94.76%

I feel my major field of study was easily done

academically. 1981 69.36%

Item 29 from the exit survey asked respondents – “Please indicate to what extent the following

items were important in your decision to attend FAMU:” Results are summarized below.

Total Respondents

(n)

% Very Important &

Somewhat Important

Collegiate Experience* 1993 55.67%

Intercollegiate athletics (such as Football,

Basketball, etc.). 1996 30.01%

Marching 100 Band. 1995 41.30%

Total Respondents

(n)

% Very Important &

Somewhat Important

On-Campus Social Events. 1995 44.81%

Student Clubs and Org. 1985 50.83%

Reputation of the Degree Programs. 1991 83.22%

Reputation of Florida A&M University. 1996 83.87%

Item 34 from the exit survey asked respondents – “How confident are you about your ability to find a

job related to your major area of study after earning your degree from FAMU?” Results are

summarized below.

Employment

I have already accepted a job offer 1984 12.00%

Very confident 1984 35.64%

Confident 1984 23.64%

Somewhat confident 1984 23.89%

Not confident at all. 1984 4.84%

Total Respondents

(n)

% Very Important &

Somewhat Important

Item 31 from the exit survey asked respondents – “Would you recommend Florida A&M University

(FAMU) to others who may be interested in your field of study?” Results are summarized below.

Likelihood of Recommending FAMU to Others

Yes 1747 87.31%

No 254 12.69%

Note: (*) Represents the average number of students and their corresponding scores for topical area.


ETS Proficiency Profile (EPP)

The ETS Proficiency Profile (EPP) test is a General Education knowledge and skills test and one of the tests used

by the Voluntary System of Accountability (VSA). The test measures critical thinking and college-level reading,

writing, and mathematics skills in the contexts of the humanities, social sciences, and natural sciences. In an

effort to evaluate General Education learning outcomes, the University currently requires students to take the

ETS EPP test at entry (freshmen) and exit (senior) levels. Additional information about the EPP can be found at

http://www.ets.org/proficiencyprofile/about.

FALL 2013

In an effort to encourage student participation in the test, the decision was made in Fall 2013 to incentivize

participation. The student who scored the highest on the test received an IPAD. All students who scored a 470 or

better (the score range is 400-500) were entered into a raffle for an IPOD, from which three winners were chosen.

In an effort to bring about awareness for the EPP the Office of University Assessment engaged both freshmen and

seniors in dialogue relative to how the test is used as a mechanism for continuous improvement.

As a result of these efforts, in Fall 2013, (n=334) incoming freshmen attempted to take the EPP test. Results are

calculated for (n=308) of those students who completed 75% or more of the test items. The results showed that

one student had a perfect score of 500 among the five top performers on the test. A total of 10 students scored

470 or above. Table 5 provides a summary of the EPP top performers, their intended major field of study and

scores.

Table 5

ETS Top Scorers

Name Intended Major Total Score

Fralin, Phylesia, A Accounting 500

McGowan, Calvin, E Biological Sciences 493

Roy, Rais Health & Medical Sciences 493

Gonzalez, Jorge Biological Sciences 490

Onubogu, Ugoma, C Biological Sciences 486

The national average for graduating seniors and first year students sitting the ETS Proficiency Profile was

(M=447.89) and (M=435.31) respectively. The national mean provided was taken from the annual comparative

data guide provided by ETS. The report provides descriptive statistics based on the number of students that have

completed an un-proctored version of the ETS Proficiency Profile between July 1, 2008 and June 30, 2013. A

five year rolling average is utilized to normalize the scores. The anonymity of each institution’s performance is

maintained by reporting only the aggregate performance of the selected reference group. As a way of gaining

insights into FAMU Freshman and Seniors’ overall performance on the EPP, Tables 6 and 7 provides a historical

overview of performance. As can be discerned from Table 6, the performance of FAMU Freshmen has been

relatively flat since 2011. Relative to the performance of FAMU Seniors, there was much variability in overall

mean scores since 2009. Please refer to Tables 6 and 7 for a comprehensive overview of these results. Figures 6

and 7 provide a graphical representation of the overall mean performance of FAMU students.

http://www.ets.org/proficiencyprofile/about


Table 6

Trends in Incoming Freshmen Performance on the ETS Proficiency Profile (2006-2013)

2006 2007 2008 2009 2010 2011 2012 2013

Mean Score 431.19 436.16 434.8 430.41 429.67 431.61 431.26 431.68

Skill Dimension 2006 2007 2008 2009 2010 2011 2012 2013

Critical Thinking 108.42 109.38 108.85 108.08 107.56 108.28 107.65 107.97

Reading 113.98 116.24 115.4 113.59 113.07 113.52 113.54 113.57

Writing 112.01 112.91 113.67 112.17 111.95 112.36 112.12 112.43

Mathematics 110.09 111.22 110.04 109.88 110.49 110.77 111.15 110.97

Humanities 111.29 113.37 112.5 111.62 112.43 112.57 112.64 113.17

Social Sciences 110.78 111.88 111.37 110.51 110.03 110.72 110.42 110.29

Natural Sciences 112.21 113.48 113.01 111.96 111.17 111.91 111.05 111.54

Percent Proficient 2006 2007 2008 2009 2010 2011 2012 2013

Reading, Level 1 41% 55% 51% 34% 29% 32% 32% 31%

Reading, Level 2 13% 23% 20% 12% 10% 12% 12% 13%

Critical Thinking 1% 2% 3% 1% 1% 1% 3% 2%

Writing, Level 1 43% 52% 53% 39% 35% 34% 37% 40%

Writing, Level 2 7% 10% 19% 8% 9% 9% 10% 11%

Writing, Level 3 2% 3% 7% 2% 3% 4% 3% 3%

Mathematics, Level 1 37% 42% 29% 28% 28% 28% 29% 32%




Figure 6 Trends in Mean Score of Incoming Freshmen on ETS Proficiency Profile

SPRING 2014

In an effort to evaluate the General Education Program and the flow of knowledge within degree programs, all

students graduating with a bachelor’s degree are asked to take the ETS Proficiency Profile (EPP). In Spring 2014,

(n=600) out of (n=981) graduating seniors took the EPP test. Results are calculated for (n=537) students who

completed 75% or more of the test. Table 7 provides a comprehensive overview of the performance of FAMU

seniors covering the last six years.

Table 7

Trends in Graduating Seniors Performance on the ETS Proficiency Profile (2009-2013)

2009 2010 2011 2012 2013 2014

Mean Score 438.08 432.97 430.84 434.64 434.28 430.56

Skill dimension 2009 2010 2011 2012 2013 2014

Critical Thinking 109.88 108.55 108.28 109.07 109.18 108.09

Reading 115.81 114.8 113.79 114.85 114.53 113.69

Writing 113.48 112.14 111.54 112.35 112.61 111.45

Mathematics 112.4 111.04 110.75 111.53 111.2 110.9

Humanities 113.47 113.74 112.67 114.03 113.51 112.88

Social Sciences 111.88 111.11 111.01 111.39 111.34 110.55

Natural Sciences 113.72 112.34 112.02 112.56 112.98 111.76

Percent Proficient 2009 2010 2011 2012 2013 2014

2006 2007 2008 2009 2010 2011 2012 2013 2014

Mean Score* 431.19 436.16 434.8 430.41 429.67 431.61 431.26 431.68 429.13

National Avg. 440.50 440.50 440.50 440.50 440.50 440.50 435.31 435.31 435.31

422

424

426

428

430

432

434

436

438

440

442

Me

an S

cale

d S

core


2009 2010 2011 2012 2013 2014

Reading, Level 1 51% 39% 32% 37% 37% 35%

Reading, Level 2 24% 18% 13% 17% 16% 13%

Critical Thinking 2% 1% 0% 3% 4% 1%

Writing, Level 1 54% 36% 32% 39% 41% 32%

Writing, Level 2 15% 11% 10% 11% 13% 10%

Writing, Level 3 5% 4% 3% 4% 4% 2%

Mathematics, Level 1 47% 31% 31% 33% 31% 31%



Figure 7 Trends in Mean Score of Graduating Seniors on ETS Proficiency Profile

Item Analysis Results

Areas of Strength: The following areas were found to be consistently above the national average for FAMU

students. Note that a majority of the content areas identified fell within the Mathematics category.

Freshmen:

Mathematics - Apply formula

2009 2010 2011 2012 2013 2014

Mean Score* 438.08 432.97 430.84 434.64 434.28 430.56

National Avg. 448.46 448.46 448.46 448.46 448.46 447.89

420

425

430

435

440

445

450

Me

an S

cale

d S

core


Mathematics - Data interpretation- read information

Mathematics - Exponential functions

Mathematics - Negative and positive integers- average

Mathematics - Number line

Reading - Recognize explicit information

Writing - Recast an existing sentence

Writing - Recognize correct construction

Seniors:

Mathematics - Word problem- logarithmic function

Mathematics - Apply formula

Mathematics - Word problem- algebraic equation

Areas of Weakness: The following areas were found to be significantly below the national average score for

FAMU students.

Freshmen:

Critical Thinking - Discern purpose of a reference

Critical Thinking - Evaluate an argument

Critical Thinking - Evaluate data for consistency

Critical Thinking - Evaluate Hypothesis

Critical Thinking - Recognize a valid inference

Critical Thinking - Recognize an assumption

Mathematics - Translation to algebraic expression

Reading - Discern facts from passage

Reading - Discern primary purpose

Reading - Recognize a valid inference


Writing - Recognize agreement

Seniors:

Critical Thinking - Discern purpose of a reference

Critical Thinking - Evaluate an argument

Critical Thinking - Evaluate Hypothesis

Critical Thinking - Recognize assumption

Reading - Discern facts from passage

Reading - Discern primary purpose


Reading - Recognize valid inference

Reading - Synthesize material


Writing - Appropriate Connector

Writing - Recast an existing sentence

Writing - Recognize coordination

Writing - Recognize grammatical correction

The results continue to confirm that our students are entering the university with a deficiency in critical thinking

skills. This year, an emerging concern is students writing and reading skill. It is important that more emphasis is

placed on these skills and its relationship with students’ critical thinking. For the seniors, more development is

essential in the area of writing. By improving writing and reading across the curriculum, this will enhance

students’ critical thinking skills, which in turn affect their reading comprehension and writing relationship skills.

Finally, the focus to improve critical thinking needs to naturally flow into the programs and not just in the general

education sequence. Additional information about the EPP can be found at

www.ets.org/proficiencyprofile/about.

National Survey of Student Engagement

The National Survey of Student Engagement (NSSE) is designed to capture students’ perceptions of their

collegiate experiences to include how students spend their free time, what they have gained form classes and

their interactions with faculty and peers. The NSSE is administered annually at many colleges and universities

nationwide. The NSSE was first administered at Florida A&M University (FAMU) in 2005 and was subsequently

administered in 2008 and then in 2011. The 2014 administration of NSSE represents the fourth time that this

instrument has been administered at FAMU. FAMU results for each indicator assessed on the instrument are

analyzed and compared with that of the group of institutions in the State University System (SUS) i.e. (n=9),

Carnegie Classification of similar institutions (n=54) and the NSSE 2013 & 2014 group of institutions.

The instrument is administered to first-year and senior students to gain their perspectives on programs and

activities that the institution provides for their learning and personal development. The results provide an

estimate of how undergraduates spend their time and what they gain from attending college. The 2014 NSSE was

launched at FAMU on February 13, 2014. There were four reminders sent to non-responders. The survey closed

on April 15, 2014 with an overall response rate of 25% for first year students and 23% for seniors. Taken as a

group, there were (n=835) students who participated in the survey. When compared to the average response

rate for the State University System (SUS) in Florida, FAMU’s response rate was four percentage points higher for

first-year students and two percentage points lower for seniors. Following is a summary of key findings from

the 2014 administration of NSSE:

FAMU First Year students and seniors perceived higher levels of engagement with faculty when

compared to other students from institutions in the SUS, Carnegie Classification and NSSE 2013 & 2014

taken as a group.

FAMU Seniors perceived higher levels of learning with peers as described by the engagement indicator

“Collaborative Learning” when compared to other students from institutions in the SUS, Carnegie

Classification and NSSE 2013 & 2014 taken as a group.


FAMU First Year students perceived lower levels of engagement in the areas of “Discussions with

Diverse Other”, “Effective Teaching Practices” and “Quality of Interactions” when compared to other

students from institutions in the SUS, Carnegie Classification and NSSE 2013 & 2014 taken as a group.

FAMU Seniors perceived lower levels of engagement with respect to “Quality of Interactions” when


taken as a group.

The majority of the respondent sample rated their overall experience at FAMU as excellent or good

(First year students – 70% and Senior – 82%).

Seventy-nine percent of seniors as compared to 82% for the Carnegie Classification of students

stated that they would definitely or probably attend FAMU again.

While the majority of the respondent sample were satisfied with their overall experiences at FAMU, the results

suggest that much more must be done in an effort to address areas of opportunity especially with respect to

activities aligned with the following engagement indicators: Discussion with Diverse Others, Effective

Teaching Practices, Quality of Interactions.

Students Assessment of Their Experiences at FAMU

Students' perceptions of their cognitive and affective development, as well as their overall satisfaction with the

institution provide useful evidence of their educational experiences. This section of the report highlights key

findings relative to FAMU students’ perceived gains as a result of attending FAMU. Table 8 provides a summary

of FAMU students’ perceptions of their experiences and how FAMU contributed to their knowledge, skills, and

personal development in ten areas. As can be discerned from this data, 70% or more of graduating seniors felt

that FAMU contributed “very much or “quite a bit” to their knowledge, skills and personal development. Please

refer to Table 8 for a summary of these results.

Table 8 Perceived College Gains of FAMU Seniors

Perceived Gains (Sorted highest to lowest) Percentage of Seniors Responding "Very much" or

"Quite a bit"

Thinking critically and analytically 85%

Working effectively with others 83%

Speaking clearly and effectively 78%

Developing or clarifying a personal code of values and ethics

78%

Acquiring job- or work-related knowledge and skills

77%

Being an informed and active citizen 75%

Writing clearly and effectively 74%


Perceived Gains (Sorted highest to lowest) Percentage of Seniors Responding "Very much" or

"Quite a bit"

Solving complex real-world problems 73%

Analyzing numerical and statistical information 72%

Understanding people of other backgrounds (econ., racial/ethnic, polit., relig., nation., etc.)

70%

Academic Advising

The Academic Advising module on the NSSE examines students' experiences with academic advising, including

frequency, accessibility, and types of information provided. It also asks students to identify their primary source

of advice. This section of the report summarizes FAMU students’ experiences with academic advising and

compares the results to the comparison group in the 2014 administration of NSSE. As can be discerned from

Tables 9 and 10, for both first-year and seniors, the mean difference between FAMU students responses and the

comparison group relative to the following item (During the current school year, about how many times have you

and an academic advisor discussed your academic interests, course selections, or academic performance?) was

statistically significant with a small to medium effect size. For the majority of items measured within this section

for both first-year and seniors, there were no meaningful differences between FAMU students and that of the

comparison group. This would suggest that while FAMU students spend more time on average with their academic

advisor, there is room for improvement relative to the quality of those interactions. The majority of FAMU

students (both first-year and seniors) also indicated that the academic advisor assigned to them has been their

primary source of advice relative to their academic plans. Please refer to Tables 9 and 10 for a summary of

these results.

Table 9 Comparative Overview of FAMU First –Year Students Experiences with Academic Advising

Variable FAMU

M

Comparison

Group M

p Effect Size

(Cohen’s d)

During the current school year, about how

many times have you and an academic advisor

discussed your academic interests, course

selections, or academic performance?

Note: The response scale for this item ranged

from 0- 6 or more.

2.80 2.11 .000 .45

During the current school year, to what extent have your academic advisors done the following?

Note: The response scale for this item was as follows: Very Little (1), Some (2), Quite a bit (3), Very

much (4).

Been available when needed 2.88 2.98 .138 -.10

Listened closely to your concerns and questions 3.00 3.00 .933 .01


Variable FAMU

M

Comparison

Group M

p Effect Size

(Cohen’s d)

Informed you of important deadlines 2.87 2.81 .435 .05

Helped you understand academic rules and

policies

2.85 2.77 .289 .07

Informed you of academic support options

(tutoring, study groups, help with writing,

etc.)

2.69 2.70 .881 -.01

Provided useful information about courses 2.87 2.85 .700 .03

Helped you when you had academic difficulties 2.64 2.64 .924 -.01

Helped you get information on special

opportunities (study abroad, internship,

research projects, etc.)

2.59 2.52 .369 .06

Discussed your career interests and post-

graduation plans

2.59 2.50 .217 .09

Significant at p<0.05 level

Table 10 Comparative Overview of FAMU Seniors Students Experiences with Academic Advising

Variable FAMU

M

Comparison

Group M

p Effect Size

(Cohen’s d)

During the current school year, about how

many times have you and an academic advisor

discussed your academic interests, course

selections, or academic performance?

Note: The response scale for this item ranged

from 0- 6 or more.

2.84 2.26 .000 .33

During the current school year, to what extent have your academic advisors done the following?

Note: The response scale for this item was as follows: Very Little (1), Some (2), Quite a bit (3), Very

much (4).

Been available when needed 2.56 2.95 .000 -.39

Listened closely to your concerns and questions 2.77 2.96 .000 -.18

Informed you of important deadlines 2.68 2.73 .368 -.04

Helped you understand academic rules and

policies

2.64 2.64 .954 .00

Informed you of academic support options

(tutoring, study groups, help with writing,

etc.)

2.37 2.39 .730 -.02

Provided useful information about courses 2.58 2.69 .037 -.10

Helped you when you had academic difficulties 2.54 2.58 .505 -.03


Variable FAMU

M

Comparison

Group M

p Effect Size

(Cohen’s d)

Helped you get information on special

opportunities (study abroad, internship,

research projects, etc.)

2.42 2.45 .642 -.02

Discussed your career interests and post-

graduation plans

2.52 2.47 .324 .05

Significant at p<0.05 level

Office of University Assessment Activities/Updates

This section of the 2013-2014 OUA Annual Report highlights the newly introduced FAMU STARS Assessment Model

in addition to providing an overview of the Office of University Assessment focused objectives for 2014-2015.

The section concludes with a summary of data from workshops and training sessions conducted over the course

of the year.

STARS Assessment Model

In the 2013-2014 academic year, the Office of University Assessment (OUA) began to transition the university

from the FAMOUS assessment approach to the STARS assessment approach, a five step approach to quality

assessment designed to provide tighter alignment of outcomes to the University’s Strategic Initiatives and

President’s goals. The FAMU STARS assessment approach was also designed to allow for the aggregation and

disaggregation of outcomes and outcome types at the program, unit and division level. During the Spring and

Summer of 2014, the OUA embarked on a university-wide initiative geared towards socializing assessment

coordinators and others responsible for assessment in their units to the new approach to quality assessment.

Figure 8 provides a graphical representation of the FAMU STARS assessment process.


Figure 8

FAMU STARS Assessment Process

OUA Focused Objectives for 2014-2015 Assessment Cycle

Consistent with its main goal of strengthening institutional assessment, the OUA objectives for the 2014-2015

academic year follows:

Strengthen the Office of University Assessment through the use of performance evaluation measures for

instructional and administrative units.

Increase university awareness of assessment best practices, including the understanding of curriculum

and assessment mapping.

Enhance the electronic monitoring database of assessment reports in an effort to improve the review

and communication process relative to submitted documents.

Maintain accreditation standards by assisting programs in preparing for re-accreditation or program

review in an effort to maintain regional/specialized accreditation and enhance quality.


Increase data and recommendations for improvements provided to university administrators based on

analysis of surveys administered to the university community.

Workshops/Training Sessions

In an effort to build capacity for assessment that matters, the OUA facilitated Assessment training sessions

with all Assessment Coordinators and others responsible for assessment within their units across the university.

The purpose of these training sessions were as follows:

a) Reinforce the mission of the Office of University Assessment

b) Getting back to the basics with quality assessment

c) Gain a deeper understanding of the planning process for the development of quality outcomes and

measures

d) Close the loop on the assessment process

e) Introduce STARS, a re-engineered approach to assessment

These training sessions afforded participants the opportunity to engage in meaningful ways with the OUA as we

work to broaden our impact on assessment for the university. The targeted training sessions were well received.

Over the course of the 2013-2014 academic year a total of (35) training sessions were conducted. There was a

general consensus among the participants that the sessions were helpful in increasing their understanding of best

practices in assessment. Participants also expressed that among others, the most beneficial aspect of the training

was the transition to the FAMU STARS assessment approach. Based on the success of this approach in engaging

the university community around quality assessment, the OUA will continue this format during the course of the

next assessment cycle. Table 11 provides a summary of participants’ perceptions of their experiences relative

to the training in addition to their experience interfacing with the OUA.

Table 11 Summary of 2014 University Wide Assessment Training

Survey Item n % Strongly Agree/Agree

Items relative to training session The presentation materials provided will be useful in helping me with assessment in my unit.

135 97.8%

The speaker(s) seemed knowledgeable of the discussion topics. 135 100% The discussion topics were relevant to assessment. 135 99% I found the topics discussed during the session to be applicable to my work and responsibilities.

135 99%

The knowledge and skills I learned in this training session will inform the assessment process in my unit.

135 96%

Overall, I am satisfied with the quality of today's training session. 135 99% Items relative to services provided by the OUA


Survey Item n % Strongly Agree/Agree

The OUA staff provides timely response to requests for information and feedback.

100 99%

The OUA staff addressed any questions or concerns to my satisfaction. 115 99% I am satisfied with the quality of services (i.e., assessment trainings and survey design/administration) provided by the OUA.

113 96%

I am satisfied with the quality of support (i.e., accreditation preparation and data analysis) offered by the OUA.

107 98%

Overall, I am satisfied with the quality of the services provided by the OUA.

111 99%

n M How likely is it that you would recommend this assessment training to your colleagues? (Note: This item is rated on a ten point scale with anchored responses ranging from Not at all likely to Extremely Likely)

121 9.30

Note: Percentages are rounded to the nearest whole number. Items relative to services provided by the OUA was scored on a five point Likert scale with responses ranging from Strongly Agree to Strongly Disagree. The fifth point on the scale was labeled n/a, to accommodate for individuals who could not assess these items because of a lack of interface with the OUA. Individuals selecting this response category were removed from the final analysis.

Assessment Committees Activities/Updates

This section of the 2013-2014 OUA Annual Report provides an update on activities carried out by several university

committees to include the General Education Assessment Committee (GEAC), the Quality Enhancement Plan

Committee (QEP) and the Institutional Level Assessment Committee (ILAC).

General Education Assessment Committee Activities/Updates

The General Education Assessment Committee (GEAC) continues to fulfill its charge with a particular focus in

developing, implementing, and monitoring the General Education assessment processes for the University. During

the academic year 2013-2014, the main focus in General Education Assessment was to finalize all changes to the

current master list of general education courses. The following summarizes the activities of the GEAC.

Determined university general education requirements as mandated by recent legislation (BOG

Regulation 8.005)

Evaluated and revised master list of courses for General Education

Reviewed recommendation for additional student learning outcomes

Reviewed strategic suggestions for communications outcomes (as provided by faculty from the

Department of English and Modern Languages)

Completed Focus Groups (April 2013 and December 2014)

Collected student artifacts and conducted evaluation for assessment purposes

Provided updates to the Institutional Effectiveness Committee December 2014


Provided regular GEAC updates/ reports to the Institutional Level Assessment Committee during the 2013

– 2014 academic year

Perception of General Education/Liberal Arts Education

Results of a focus group exercise conducted with FAMU students revealed that our students associate the general

education curriculum with specific subject areas i.e. the humanities, natural sciences, etc. It was also suggested

that the general education curriculum is a body of knowledge that can be applied to one’s academic career

regardless of one’s chosen major. Students see the inherent value in the general education curriculum; however

they, question its compulsory nature given the associated cost and the potential negative impact on the student’s

GPA if he/she fails the course. One student argued, “if the general education curriculum is liberal arts let me be

liberal (the individual right to choose)” about his/her choice of general education courses. The university has

the responsibility to provide the option of taking general education courses as opposed to forcing students to

take courses unrelated to their major. A general education curriculum should only be mandatory under a system

of higher education that is “public” in the purest sense. If students are responsible for paying for their education

they should have the right to choose.

Recommendations

Student responses from a focus group revealed that students believe:

Beyond writing, both speaking and interacting effectively are important communication skills that should

be taught in the general education curriculum.

The pervasiveness of social media technology undermines the ability of students to communicate

(writing, verbal communication, etc.) effectively.

Communication courses are absolutely necessary to combat the negative impact of social media.

In an effort to use feedback to further improve programmatic elements of the English and Modern Languages

Department, and to address recent mandates that impact how the communication requirement is satisfied, the

following recommendations were provided by faculty from the English and Modern Languages Department.

1. All students enrolled at FAMU should take the 4 writing intensive courses as stipulated by the Gordon

Rule. It seems that currently all students are not fulfilling the 24,000 word requirement for writing

competency.

a) Perusal of the web reveals that some departments require only 1 additional course for their

majors beyond the 2 required English composition courses.

b) Also, speech is listed as an additional course for some departments, and our students

certainly can benefit from taking the course, but enhancement of oral communication skills

cannot be substituted for enhancement of written expression.

2. The courses in General Education be monitored more closely to ensure intense instruction is provided,

and required quantity of writing is completed.


Institutional Level Assessment Committee Activities/Updates

The Institutional Level Assessment Committee (ILAC) continued to monitor the assessment activities of the

university at the institutional level during the 2013-2014 academic year. The committee successfully held

meetings during which they discussed, decided and/or made propositions on important subjects regarding the

University's assessment activities and processes. The committee met during the fall and spring semester. The

committee took time to reflect and revisit its purpose/role and began the process of re-establishing its role in

the institutional level assessment activities of the university. Table 12 provides a list of tasks that were addressed

during the 2013-2014 academic year. Please be advised that some activities will be on-going and will continue

beyond the current year:

Table 12

ILAC Tasks and Outcomes for 2013-2014

Task Purpose of Task Action Item Outcomes

ETS

Purpose is to assess student basic skills and critical thinking. Revise policy statement to reflect percentages of schools and colleges that will identify student participants.

Updated – October 2013 and reviewed by ILAC for resubmission to ILAC, IE, and policy committee.

Senior Exit Survey

Purpose is to update Senior Exit Survey and create alignment with the goals of the university concerning student academic outcomes and activities – link results to tangible outcomes.

Updated – October 2013, April 2014 and presented to ILAC.by Mr. Francis. Further recommendations made concerning changes needed. Further review will occur by ILAC.in Fall 2014

SAC’s 5th-year Report

The fifth-year interim report was developed to respond to the U.S. Department of Education’s requirements (1) that accrediting bodies continuously monitor institutions to ensure complaints and (2) that accrediting bodies have a mechanism for reviewing multiple sites initiated since the last reaffirmation. ILAC will be responsible for responding to CR 2.10 (Student Support Services, CS 3.3.1.1 (Institutional Effectiveness Educational Programs, and CS 3.1.1.3 (Physical Facilities).

Reviewed by ILAC – Subcommittee formed. October 2013. No further action taken waiting for directive from IE. Will pursue after direction.

Campus Police Survey/FAMOUS Assessment Request

To assist the Campus in obtaining assessments on student’s view of safety on campus and this department’s effectiveness in addressing campus-wide needs concerning safety.

Update – October 2013


Task Purpose of Task Action Item Outcomes

The Police Department will now be required to complete the ADESU FAMOUS Assessment Plan to document levels of safety of students.

FAMOUS Assessment Training

Linking Assessment results to outcomes – A focus on rows 5 and 6 of the FAMOUS Model.

TBA

Linking Assessment to Student Assessment

Increasing oral and written reports documenting a closed loop of student’s assessment results in final outcomes including resources and etc. Assessment Briefs in FAMUAN

Provided non-fiscal examples and recommendations to document closing the loop for student outcomes to ILAC.

Pursue Closing the Loop Funding

Identify other funding opportunities to assist programs in financing data driven recommendations.

Recommendations submitted to IE. No further action taken

Quality Enhancement Plan Committee Activities/Updates

The 2013-2014 academic year marked the final year of the Quality Enhancement Plan (QEP) introduced five years

earlier. The Office of the Quality Enhancement Program maintained and forged new collaborations with various

units across campus to execute various student and faculty-oriented activities. Some of the student activities

included the Freshman Summer Reading Program, the Freshman Critical Thinking Seminar Series, and the First

Year Experience Course. In addition to student activities, the QEP also hosted various faculty-oriented

professional development opportunities that were instrumental in changing the campus culture. Below are

highlights of activities sponsored by the Quality Enhancement Program.

Year 5 of the Freshman Summer Reading Program

This co-curricular activity promotes student learning by requiring all incoming freshman students to read

a common book and complete a writing assignment during the summer preceding the fall term.

Losing My Cool was selected as the 2014 Common Book selection.

This year the QEP and the Division of Housing sponsored “Dorm Dialogues” in seven residential halls

across campus. The designated day for dialogue allowed students to have intellectual and critical

discussions about the common book selection. All incoming freshmen received a copy of Losing My Cool

during the summer prior to coming to campus.

Students completed written responses on the common reading selection and submitted these essays in

their First Year Experience Course (SLS 1101). More than 400 students participated in the Dorm

Dialogues.

QEP provided instructional training to FYE faculty on how to utilize the book through the use of critical

thinking pedagogy strategies.


The author of the 2014 book selection (Losing My Cool) visited the campus during the fall semester to

facilitate in discussions with the University community. Six hundred sixty-three (663) freshman students

attended the Author’s Seminar.

Year 5 of the Freshman Critical Thinking Seminar Series

This co-curricular activity is designed to enhance students’ critical thinking skills through the

participation in interactive dialogue sessions addressing life skills and academic issues.

Approximately 1,700 students participated in critical thinking seminars during the 2013 – 2014 academic

year.

Topics of the 2013-2014 critical thinking seminar sessions included: Critical Thinking, Financial Literacy,

Race Relations, and Entrepreneurship.

Year 5 of the Faculty Development Activity (Program)

Through this activity (program), faculty are provided opportunities to attend on-campus training sessions,

travel to professional conferences, and engage in curriculum development activities.

Thirty-seven (37) on-campus workshops and two (2) off-campus workshops (Law School) on critical

thinking, course redesign, and active learning were held during the academic year.

Two hundred seventy-three (273) faculty and staff attended at least one of the workshop sessions.

Five (5) faculty trips to off-campus conferences/workshops, specific to teaching, assessment and student

learning were funded by the QEP during the academic year.

QEP co-sponsored a Faculty Learning Community (FLC) with the Innovative Academic Instructional

Program (IAIP) during the summer of 2014. Nine (9) faculty were selected to participate in a year-long

interactive course redesign exercise to enhance pedagogy and instructional approaches.

Faculty participating in the FLC also received training to teach in new scale-up classroom.

The QEP/ Faculty Development Activity sponsored a spring faculty planning conference with themed

sessions that addressed active learning and the flipped classroom concept.

The QEP/ Faculty Development Activity launched the Innovative Teaching Award, which recognizes junior

and senior faculty members who are utilizing unique, innovative teaching approaches that enhance

student performance. The nominees and winners of this teaching award were announced and recognized

at the 2014 spring Faculty Planning Conference.

The QEP served as the impetus for the new Teaching and Learning Center, which hosted its first

workshops in October 2014. This Center (TLC) serves a resource venue for faculty, as well as a central

location for training and faculty development workshops for the main campus and all site campuses.

Year 5 of the CAT Test and Assessment

The QEP continued the development and implementation of a uniform approach for assessing the critical

thinking skills of freshman students. The CAT test was administered in the First Year Experience courses.

Preliminary results indicate that 38% (n=138) of freshman students who completed the CAT showed at

least a 5% mean score gain between pre and post-tests scores.

The QEP partnered with the General Education Assessment Committee (GEAC) and the two academic

units to further assess the critical thinking skills of freshman students. Results were reported in the QEP

Impact Report and the annual assessment report for GEAC.


The College of Science and Technology has adopted the CAT test as their commercial instrument of

choice to also evaluate students’ critical thinking skills.

The Critical Thinking Assessment Test (CAT), critical essays, and case study analyses were administered

to students during the course of the freshman year to evaluate their critical thinking skills. The

assessment results are being used to guide future improvements in faculty instruction and assessment of

students’ critical thinking skills in the classroom setting.

The QEP Report will be submitted as a part of the Fifth Year Interim Report in March 2015.

Licensure/Certification Examination Pass Rates

This section of the report highlights pass rates on licensure/certification examinations. The Board of Governors

(BOG), State University System (SUS), monitors the progress of performance on licensing and certification exams

for specific bachelor’s and professional programs in which licensure is required to practice in the field. In addition

to progress monitored by the BOG, FAMU Board of Trustees (BOT) has established increasing pass rates on

licensure examinations and set target pass rates for the programs in which passing licensure or certification is a

condition of employment in the field. To meet the BOT’s goals, each academic program developed detailed

plans with various strategies on how it plans to achieve the stated goals. Table 13 provides a summary of FAMU

Licensure/Certification pass rates by program. Figures 9 – 14 provide a graphical overview of these results.

As can be discerned from the data collected from respective academic programs, first-time pass rates for the

College of Law have been steadily increasing since 2010, with the highest pass rate earned in 2013. In 2014, a

first-time pass rate in Nursing was equal to the pass rates nationally and surpassed first-time pass rates at the

State level. The College of Law first-time pass rates (i.e. 72% was just under the State level (73%) by 1%). In 2013

and 2014, Cardiopulmonary Science surpassed the national average with 100% of its graduates passing the

licensure examination the first time. Other programs in which the pass rate is below the national or state overall

pass rate are instituting procedures to improve pass rates.

Areas requiring Licensure/Certification Examination

College of Education

o Counselor Education (M.S./M.Ed.)

o Early Childhood Education (B.S.)

o Educational Leadership (M.S.)

o Elementary Education (B.S.)

o Social Science Education

o Theatre Education

o Biology Education

o English Education

o Math Education

o Music Education

o Physical Education (B.S.)

College of Law


o Law

College of Pharmacy and Pharmaceutical Sciences

o Pharmacy (Pharm. D.)

School of Nursing

o Nursing (B.S.)

School of Allied Health

o Health Informatics and Information Management (B.S.)

o Cardiopulmonary Science (B.S.)

o Occupational Therapy (M.S.)

o Physical Therapy (DPT)

Table 13 Longitudinal Overview of FAMU Students’ Licensing / Certification Examination Pass Rates by Program Nursing: National Council Licensure Examination for Registered Nurses

2010 2011 2012 2013 2014*

Examinees 72 73 84 82 36

First-time Pass Rate 85% 85% 90% 74% 89%

State Benchmark 89% 88% 86% 77% 77%

National Benchmark 89% 89% 92% 89% 89%

1st and 2nd Quarter Results: January 01, 2014 – June 30, 2014

Law: Florida Bar Exam

2010 2011 2012 2013 2014

Examinees 108 144 123 175 36



Pharmacy: North American Pharmacist Licensure Exam

2010 2011 2012 2013 2014*

Examinees 133 142 130 123 -

First-time Pass Rate 72% 87% 88% 85% -

National Benchmark 94% 96% 97% 96% -

*2014 Results Not Available.

Physical Therapy: National Physical Therapy Examinations

2010 2011 2012 2013 2014

Examinees - - 19 26 16


State Benchmark - - 77% 85% 94%



Occupational Therapy: National Board for Certification in Occupational Therapy Exam

2010 2011 2012 2013 2014*

Examinees - - - - -

First-time Pass Rate 50% 33% 60% 90% -

National Benchmark 81% 81% 81% 81% -

*2014 Occupational Results Not Available.

Cardiopulmonary Science: National Board for Respiratory Care Examination

2010 2011 2012 2013 2014

Examinees 11 8 23 21 10



Source: FAMU Office of Program Quality

Figure 9

Trends in Nursing Pass Rates: National Council Licensure Examination for Registered Nurses

2010 2011 2012 2013 2014*




0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%


Figure 10 Trends in Law Pass Rates: Florida Bar Exam

Figure 11 Trends in Pharmacy Pass Rates: North American Pharmacist Licensure Exam

2010 2011 2012 2013 2014


State Benchmark* 78% 88% 79% 78% 73%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2010 2011 2012 2013

First-time Pass Rate 72% 87% 88% 85%

National Benchmark 94% 96% 97% 96%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%


Figure 12

Trends in Physical Therapy Pass Rates: National Physical Therapy Examination

2010 2011 2012 2013 2014


State Benchmark 77% 85% 94%


0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2010 2011 2012 2013

First-time Pass Rate 50% 33% 60% 90%

National Benchmark 81% 81% 81% 81%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%


Figure 13

Trends in Occupational Therapy Pass Rates: National Board for Certification in Occupational Therapy Exam

Figure 14

Trends in Cardiopulmonary Science Pass Rates: National Board for Respiratory Care Examination

Summary of Key University Assessment Findings & Recommendations

This section of the 2013-2014 OUA Annual Report provides a summary of key university assessment findings.

Exit Survey

92.9% or (n=2,042) graduating students completed the exit survey.

Students are overall satisfied with their educational outcomes and services related to their major field

of Study (M=89.23%).

Parking and Financial Aid continue to be problem areas with low satisfaction ratings (Parking M=26.35%,

Financial Aid M=33.63%).

Units should use results, specifically results that indicate low student satisfaction to inform

improvement efforts.

All departments are encouraged to conduct Point of Service (POS) surveys in order to gauge students’

perceptions of their experiences.

ETS Proficiency Profile

(n=334) incoming freshmen attempted the EPP test. Results were calculated for (n=308) of those

students who completed 75% or more of the test items.

2010 2011 2012 2013 2014



0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%


(n=600) graduating seniors attempted the EPP test. Results were calculated for (n=537) of those

students who completed 75% or more of the test items.

Results suggest that freshmen are entering the university with deficiencies in critical thinking and

writing. Focused efforts are needed to support student learning in these areas.

Results suggest that for seniors, more development is essential in the area of writing.

To address the deficiency in critical thinking, intervention efforts should be focused across the

curriculum and not solely in the general education sequence.

Incentivize student participation and support efforts to encourage competition among and within

Colleges/Schools.

Require all students in capstone courses to take the test for credit.

Recommend all programs to emphasize the skills measured by the test in the curriculum with the goal

of improving seniors’ performance at or beyond the national average.

NSSE

25% response rate for this year’s administration which was significantly higher than that of the last

administration of the instrument in 2011 (approximately 10% response rate).

FAMU First Year students and seniors perceived higher levels of engagement with faculty when


taken as a group.

FAMU Seniors perceived higher levels of learning with peers as described by the engagement

indicator “Collaborative Learning” when compared to other students from institutions in the SUS,

Carnegie Classification and NSSE 2013 & 2014 taken as a group.

FAMU First Year students perceived lower levels of engagement in the areas of “Discussions with

Diverse Other”, “Effective Teaching Practices” and “Quality of Interactions” when compared to

other students from institutions in the SUS, Carnegie Classification and NSSE 2013 & 2014 taken as a

group.

FAMU Seniors perceived lower levels of engagement with respect to “Quality of Interactions” when


taken as a group.

The majority of the respondent sample rated their overall experience at FAMU as excellent or good

(First year students – 70% and Senior – 82%).

Seventy nine percent of seniors as compared to 82% for the Carnegie Classification of students

stated that they would definitely or probably attend FAMU again.

Students benefit and are more satisfied in supportive settings that cultivate positive relationships

among students, faculty and staff.

Develop a Center for Teaching Excellence that would be charged with the responsibility of engaging

new and existing faculty in effective pedagogy.

Continue to recruit diverse faculty, staff and students to the university.

Recommended a POS system to capture in real time students’ perceptions of their experiences

interfacing with key administrative offices of the University.


Assessment Report Submissions

Assessment reports were collected for (n=111) degree programs, (n=13) colleges/schools, and (n=53)

administrative departments.

Submission rates for 2013-2014 are lower than they have been in previous years.

Timely submissions tend to increase in years that the university is preparing for accreditation visits.

OUA will continue to push for 100% submission and reinforce the importance of assessment for

continuous improvement and institutional effectiveness.

ALCs

Based on assessment reports submitted and BOG criteria, a majority of undergraduate programs are in

continuous improvement mode.

OUA will continue to provide support to new programs in developing measurable outcomes and

gathering baseline data to use for continuous improvement.

OUA Workshops and Training Sessions

A total of 35 training sessions were conducted.

Overall, 99% of participants were satisfied with the quality of the training sessions.

Participants felt that the presentation and presenters were very professional and proficient in

Assessment.

Attendees expressed that the training sessions helped them to better understand the importance of

assessment and how the process can help document the ‘good works’ being done in their area.

OUA will continue to offer 1 hour sessions with the option to extend the length of training if it is

required.


Office of University Assessment Team

Mark Howse, Ph.D.

Director

Office of University Assessment

Florida A&M University

[email protected]

Phone (850) 412-5265

Franz H. Reneau, Ph.D. Assistant Director



[email protected]

Phone (850) 412-5263

Brandi Newkirk, MBA

Coordinator Academic Support Services



[email protected]

Phone (850) 412-5266

George L. Pinkney IV, M.Ed.

Coordinator Computer Applications



[email protected]

Phone (850) 412-5262

Angela Jordan

Administrative Assistant



[email protected]

Phone (850) 412-5713

mailto:[email protected]





Documents

Florida A & M Office of University Assessment 2013-2014 ... 2013-2014 Annual...the OUA realized several key accomplishments over the course of the 2013-2014 academic year. Of significance,