View
215
Download
0
Category
Preview:
Citation preview
Florida A & M
University
Office of University
Assessment
2013-2014 Annual
Institutional
Assessment Report
A Title III Co-Sponsored Activity
Pending Approval
December 2014 page 2
Table of Contents Executive Summary .............................................................................................................. 3
Message from the Director .................................................................................................... .6
Introduction ...................................................................................................................... .7
Overview of Annual Report ................................................................................................... 7
OUA Mission Statement ....................................................................................................... 7
FAMU Assessment Philosophy ................................................................................................ 7
OUA Goals and Objectives ................................................................................................... 8
Instructional Programs Assessment Activities ............................................................................... 8
Trends in Instructional Programs Assessment Plans/Reports Submissions .......................................... 9
Academic Learning Compacts .............................................................................................. 11
Administrative and Educational Support Units Activities ................................................................ 13
Trend in Administrative and Educational Support Units Plans/Reports Submission ............................ 13
Institutional Level Assessment Activities ................................................................................... 14
Exit Survey .................................................................................................................... 14
ETS Proficiency Profile ..................................................................................................... 17
National Survey of Student Engagement ................................................................................ 22
Office of University Assessment Activities/Updates ..................................................................... 26
STARS Assessment Model ................................................................................................... 26
OUA Focused Objectives for 2014-2015 Assessment Cycle ........................................................... 27
Workshops/Training Sessions .............................................................................................. 28
Assessment Committees Activities/Updates ............................................................................... 29
General Education Assessment Committee Activities/Updates ..................................................... 29
Institutional Level Assessment Committee Activities/Updates ..................................................... 31
Quality Enhancement Plan Committee Activities/Updates .......................................................... 32
Licensure/Certification Examination Pass Rates .......................................................................... 34
Summary of Key University Assessment Findings and Recommendations ........................................... 39
Office of University Assessment Team ..................................................................................... 42
December 2014 page 3
Executive Summary
The Office of University Assessment (OUA) at Florida Agricultural and Mechanical University is committed to
promoting a culture of evidence at FAMU through the implementation of a manageable and sustainable
assessment process that leads to quality program and service improvement. The OUA achieves this mission by
providing support to all academic programs and administrative and educational support units by building capacity
for the use of best practices in assessment and continuous quality improvement. Through our concerted efforts,
the OUA realized several key accomplishments over the course of the 2013-2014 academic year. Of significance,
is the improvement in the timeliness’ and quality of the assessment reports and plans submitted by Instructional
Programs (IPs) and Administrative and Educational Support Units (ADESUs) for the 2012-2013 and 2013-2014
academic year. As of February 2014, the OUA collected and documented 100% (n=104) of the assessment reports
and plans from (IPs) and 92% (n=45) and 94% (n=47) of the assessment reports and plans from (ADESUs)
respectively, and 100% (n=13) of the assessment reports and plans for both research and community service. The
success relative to the submissions of reports and plans is largely attributed to targeted efforts over the course
of the year (i.e. site visitations, workshops etc.) in creating more awareness of the important role that
assessment plays in creating a culture of evidence and continuous improvement for the university. During the
2013-2014 academic year, the OUA transitioned from the FAMOUS Assessment Approach to the FAMU STARS
Assessment Approach, a five step approach to quality assessment designed to provide tighter alignment of
outcomes to the University’s Strategic Initiatives and Presidents goals. The FAMU STARS Assessment Approach
was also designed to allow for the aggregation and disaggregation of outcomes and outcome types at the program,
unit and division level.
Collaborative Efforts
In an effort to build capacity for assessment that matters, the OUA facilitated Assessment training sessions with
all Assessment Coordinators and others responsible for assessment within their units across the university. These
training sessions afforded participants the opportunity to engage in meaningful ways with the OUA as we work
to broaden our impact on assessment for the university. Over the course of the 2013-2014 academic year a total
of (35) training sessions were conducted. There was a general consensus among the participants that the sessions
were helpful in increasing their understanding of best practices in assessment. Based on the success of this
approach in engaging the university community around quality assessment, the OUA will continue this format
during the course of the next assessment cycle. Following is a summary of key institutional level assessment
activities and recommendations for improvement.
Institutional Level Assessment
The assessment activities conducted at the institutional level included the administration of the ETS Proficiency
Profile (EPP) for entering freshmen and graduating seniors, the Exit Survey to graduating seniors, the National
Survey of Student Engagement (NSSE) and focus group exercises conducted with current students relative to
general education outcomes. Assessment activities also included the review and submission of Academic Learning
Compacts to the Board of Governors. The OUA conducted a comprehensive review of these results and advanced
recommendations relative to each of the aforementioned areas. A sample of the results from the assessment
tools and activities follow.
December 2014 page 4
Exit Survey
Over the course of the 2013-2014 academic year, (N=2,197) students graduated from the university. Of this
population, (n=2,042) students responded to the survey representing an overall response rate of 92.9%. Consistent
with the findings of previous years, graduating students have consistently provided low ratings relative to their
level of satisfaction with the Availability of Parking (26.35%) and Financial Aid Processes (32.31%). Areas with
the highest level of satisfaction were the value of your FAMU degree (89.26%), University Libraries (87.97%), and
Size of Classes (83.73%). Considering these findings, the respective departments and units within the university
need to identify and implement programmatic strategies to combat these areas of opportunity. While the
majority of graduating students were overall satisfied with their educational outcomes and their major field of
study, in keeping with a culture of continuous improvement, we need to continually assess these areas in an
effort to stay current and competitive in an environment that is constantly changing.
ETS Proficiency Profile
In an effort to evaluate General Education learning outcomes, FAMU students are required to take the ETS
Proficiency Profile at entry (freshmen) and exit (senior) levels. In the Fall of 2013, (n=334) incoming freshmen
attempted to take the EPP test. Results were calculated for (n=308) of those students who completed 75% or
more of the test items. Of those students that completed the EPP, (n=1) student had a perfect score. The mean
score for incoming freshmen was 431.26 (SD=17.75) based on a 400-500 range. In the Spring 2014, (n=600)
graduating seniors took the EPP test. Results were calculated for (n=537) of those students. Sixty-three students
completed less than 75% of the test and as such, their scores were excluded from the analysis. Results show that
the total mean score for the graduating seniors was 430.56 (SD=18.31) based on a 400-500 range. As can be
discerned from these results, graduating seniors performed marginally below their freshman counterparts.
Further analysis of the results also confirms that our students are entering the university with deficiency in
critical thinking skills, especially in the area of ‘Evaluate Hypothesis’. Additionally, reading has become an area
of concern. It is important that more emphasis is placed on these skills in an effort to transform students through
the FAMU experience.
National Survey of Student Engagement
The 2014 administration of the National Survey of Student Engagement (NSSE) represents the third time that this
instrument has been administered at FAMU. The instrument was administered to first-year and senior students'
participating in programs and activities that the institutions provide for their learning and personal development.
The 2014 NSSE was launched on February 13, 2014 and closed on April 15, 2014 with an overall response rate of
25% or (n=835) respondents. When compared to the average response rate for the State University System (SUS)
in Florida, FAMU’s response rate was four percentage points higher. Based on analysis of the findings, FAMU
first-year students and seniors perceived higher levels of engagement with faculty when compared to other
students from institutions in the SUS, Carnegie Classification and NSSE 2013 & 2014 taken as a group. FAMU
seniors also perceived higher levels of learning with peers as described by the engagement indicator
(Collaborative Learning). Conversely, FAMU first-year students perceived lower levels of engagement in the
areas of Discussions with Diverse Other, Effective Teaching Practices and Quality of Interactions. While, FAMU
seniors perceived lower levels of engagement with respect to Quality of Interactions the majority of the
December 2014 page 5
respondent sample were satisfied with their overall experiences at FAMU. The results suggest that much more
must be done in an effort to address areas of opportunity especially with respect to activities aligned with the
following engagement indicators: Discussion with Diverse Others, Effective Teaching Practices, and Quality of
Interactions.
Recommendations for Improvement
The Office of University Assessment continues to play a critical role in enhancing institutional effectiveness and
contributing to an environment of continuous improvement and excellence in the operations and outcomes of
the university. The following recommendations are advanced based on outcomes of the assessment processes at
Florida Agricultural and Mechanical University.
• Require all students in capstone courses to take the ETS Proficiency Profile as a component of the course
grade.
• Recommend all programs to emphasize the competencies and skills measured by the ETS Proficiency
Profile in the curriculum with the goal of improving seniors’ performance at or beyond the national
average.
• Units with low satisfaction ratings on the Exit Survey need to use their findings in developing and
implementing programmatic strategies to make programmatic improvements. It is strongly recommended
that these units employ the FAMU STARS assessment process to document their efforts.
• Students benefit and are more satisfied in supportive settings that cultivate positive relationships among
students, faculty and staff. Based on this premise, it is strongly recommended that the university develop
Point of Service (POS) instruments to capture in real time students perceptions of their experiences
interfacing with key administrative offices of the university to include Financial Aid, Registrar Office,
Parking Services, and the Housing Office.
• Develop a program, in collaboration with the writing center, to enhance student writing skills. From
freshman level to senior level, student writing continues to reflect deficiencies in communication.
• Develop and implement strategies to improve the quality of student interactions with academic advisors.
Develop mechanisms to track the quality of these interactions.
• Degree programs for which students are required to take licensure examinations should review and revise
curriculum and assessment mapping for tighter alignment between instruction and the competencies and
skills being tested on examinations.
December 2014 page 6
Message from the Director
Greetings,
The Office of University Assessment staff has prepared the 2013-2014 Annual Assessment Report for your review.
It is my hope that you will find this report to be informative, enlightening, and empowering. The Office of
University Assessment was established in 1994 and has coordinated the University’s assessment activities since
that time. January, 2015 marks the second year of my tenure as the Director of University Assessment, and this
last year has been a very exciting one. During the course of the 2013-2014 academic year, the OUA welcomed a
new Coordinator for Computer Applications (Mr. George Pinkney IV). In our ongoing effort to provide the highest
quality of services to the university community, we continue to review and revise the assessment processes and
procedures. We have strengthened systems and galvanized the collaborative efforts with our university partners
to promote and maintain a culture of evidence across the entire institution.
This year we are proud to have introduced to the university community the new and improved approach to
assessment which has been labeled the FAMU STARS Assessment Approach. You will find information on STARS
and other key and important institutional level data summarized in this report as a way of quantifying and
qualifying the impact of our instructional programs and support services on the academic experience of FAMU
students. The STARS Assessment Approach is designed to promote more effective alignment of assessment
practices to the University Strategic Plan, President’s goals, and other strategic initiatives while promoting
excellence in the operation of all programs and services. Our wish is to maintain an assessment process that
empower decision makers with key performance data that promote the true “excellence with caring” experience
that FAMU students deserve.
Please take the opportunity to review the 2013-2014 Annual Assessment Report with special attention to the
elements that are most directly related to the work of your unit(s). I also implore you to use information from
this report to drive conversations about effective operation and to inform strategies for ongoing continuous
improvement.
The Office of University Assessment stands ready to support you and your assessment activities in any way that
we can. Do not hesitate to call on us.
Sincerely,
Mark Howse, Ph.D.
December 2014 page 7
Introduction
Overview of Annual Report
The 2013-2014 Office of University Assessment (OUA) Institutional Assessment Report is designed to provide you
key institutional level data as a way of quantifying and qualifying the impact of our instructional programs and
support services on the academic experience of FAMU students. Over the course of this year, the OUA continued
to provide leadership and expertise in assessment as a way of helping units to build capacity for effectively
assessing themselves based on industry standards and best practices for meaningful assessment. This report is
divided into several key areas which will provide you insights into the findings of key institutional level assessment
activities to include the Exit Survey, the ETS Proficiency Profile and the 2014 administration of the National
Survey of Student Engagement (NSSE). Also discussed in this annual report are trends in Instructional Programs
(IPs) and Administrative and Educational Support Units (ADESUs) submission rates for assessment reports and
plans. You will also find a section which provides an overview of the recently implemented FAMU STARS
assessment approach. We discuss our goals and objectives for the 2014-2015 assessment cycle in addition to
providing summary data on workshops and training sessions conducted over the course of this year. There is
great deal of information presented in this annual report and it is our hope that you will find this report
meaningful and empowering in efforts to facilitate the true “excellence with caring” experience that our
students not only deserve.
OUA Mission Statement
The mission of the Office of University Assessment is to promote a culture of evidence at FAMU through the
implementation of a manageable and sustainable assessment process that leads to quality program and service
improvement.
In support of the University’s mission, the Office of University Assessment at FAMU promotes excellence in
student learning and services across academic programs, administrative, and educational support units. Through
ongoing training and assessment support services, the Office operationalizes the development of a university-
wide culture of evidence wherein intentional, effective, and efficient programs and services are established and
improved in a manner that embodies the University's motto of “excellence with caring.”
FAMU Assessment Philosophy
The philosophy of assessment at FAMU is "to foster a culture of continuous improvement of program offerings
and support services to ensure student success and institutional effectiveness.” Assessment is grounded in the
institutional values set forth in the mission statement and focuses primarily on improving student learning both
within and outside the classroom. Our approach is rooted in a comprehensive and collaborative process that
utilizes data to improve the overall student experience.
FAMU defines assessment as “a systematic process of gathering, analyzing, interpreting and documenting the
effectiveness of the teaching and learning processes to ensure that expectations and standards are met in
fulfilling the mission of the university. The process also includes monitoring and enhancing the administrative
December 2014 page 8
and educational support structure of the university that leads to the continuous improvement of FAMU’s academic
programs and administrative and educational support services”.
OUA Goals and Objectives
Goal 1: Academic Improvement –Strengthen the distinctive quality of academic programs, producing graduates
that demonstrate the 21st-century skills needed to compete in an increasingly diverse and global society.
Objective 1.1 Enhance the capacity for development of student learning outcomes aligned to
appropriate measures and reflective of best practices for each academic discipline.
Objective 1.2 Analyze and summarize key student learning data for review by decision makers and other
stakeholders.
Goal 2: Service Improvement – Strengthen the quality of administrative and educational support services
through the development of appropriate performance outcomes and measures.
Objective 2.1 Support the development of performance-based strategies and measures to monitor and
document efficiency and effectiveness of all support services.
Objective 2.2 Summarize key performance data related to administrative and educational support unit
for review by decision makers and other stakeholders.
Goal 3: Accountability – Maintain effective and efficient assessment processes that comply with all internal
and external assessment requirements.
Objective 3.1 Facilitate the implementation of assessment practices that meet or exceed the
accountability requirements set forth by the Florida Legislature, Board of Trustees, and other external
accountability bodies.
Objective 3.2 Promote alignment between assessment planning and University strategic initiatives.
Goal 4: Communication - Raise stakeholder awareness of student learning and key performance outcomes
through the dissemination of institutional assessment data and recommendations for improvement.
Objective 4.1 Communicate student learning outcomes and other assessment information to students,
faculty, staff, administrators, and other stakeholders.
Objective 4.2 Maintain a framework for meaningful, timely feedback and recommendations related to
key assessment information.
Instructional Programs Assessment Activities
This section of the 2013-2014 OUA Annual Report highlights trends in the submission rates for assessment reports
from Instructional Programs and Colleges and Schools relative to Research and Community/Public Service. Also
discussed in this section is a status on the Academic Learning Compacts (ALCs) which are state-mandated
requirements for baccalaureate degree programs offered by public universities in the State of Florida University
System.
December 2014 page 9
Trends in Instructional Programs Assessment Report Submissions
This section highlights assessment activities for instructional programs, colleges and schools. Assessment reports
are designed to highlight progress made in assessment efforts at the program and college/school level. The OUA
advocates that each unit use their assessment report and results to celebrate achievements as well as to identify
potential areas for future improvement. The Office of University Assessment annually reviews and provides
feedback on submitted reports.
For instructional areas, assessment reports are collected for three general areas: Instructional Programs,
Research, and Community/Public Service. In 2013-2014, assessment reports were collected for (n=111)
programs. Three of the programs were newly established. For research and community/public service,
assessment reports are collected for all (n=13) colleges/schools. Table 1 provides a summary of the submission
rates for assessment reports for instructional programs, colleges/schools. Please note that the collection of
assessment reports for Research and Community/Public Service did not begin until 2008. As can be discerned
from these results, there is much variability in submission rates.
Table 1
Assessment Report Submission Rates for Instructional Programs, Colleges and Schools
2005
-
2006
2006
-
2007
2007
-
2008
2008
-
2009
2009
-
2010
2010
-
2011
2011
-
2012
2012
-
2013
2013
-
2014
Submission Rate for Instructional Programs
(IPs) Assessment Reports 94% 94% 100% 67% 97% 76% 96% 99% 80%
Number of Programs 108 106 108 108 107 107 112 105 111
Submission Rate for Colleges/Schools
Assessment Reports for Research 92% 67% 83% 46% 100% 69%
Number of Colleges/Schools 12 12 12 13 13 13
Submission Rate for Colleges/Schools
Assessment Reports for Community/Public
Service
92% 67% 83% 54% 92% 62%
Number of Colleges/Schools 12 12 12 13 13 13
Note: The percentage represented for 2013-2014 are as of December 10, 2014.
Figures 1-3 provide a graphical representation of these results. It should be noted that timely submissions of
assessment reports for Instructional Programs was highest in (2007-2008) when the university was preparing for
the SACS-COC re-accreditation visit. The OUA promotes assessment as a tool for continuous improvement and
not as a mere compliance obligation for re-accreditation. As such, the OUA will continue to hold all units
accountable for timely submissions and encourage units to make assessment a natural part of their operation.
December 2014 page 10
Figure 1
Trends in Assessment Report Submission Rates for Instructional Programs (IPs)
Figure 2
Trends in Assessment Report Submission Rates for Research
94%94%
100%
67%
97%
76%
96%
99%
61%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
92%
67%
83%
46%
100%
54%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
2008-2009 2009-2010 2010-2011 2011-2012 2012-2013 2013-2014
December 2014 page 11
Figure 3
Trends in Assessment Report Submission Rates for Community/Public Service
Academic Learning Compacts
Academic Learning Compacts (ALCs) are state-mandated requirements for baccalaureate degree programs
offered by public universities in the State of Florida University System. The ALCs serve to document the expected
learning outcomes to be achieved by every student matriculating through his/her program. The ALCs are
required, at a minimum, to determine the expected student learning outcomes for baccalaureate program
graduates in the areas of:
Content/Discipline knowledge and skills
Communication skills
Critical thinking skills
Documentation of the ALCs and the related results at FAMU are based upon the FAMOUS assessment planning and
implementation model. Periodic measurements of the extent to which student learning has taken place are
accomplished by the use of tools, such as rubrics, products of student work, and other course-embedded
measures. The Student Learning Assessment Report (formerly - ALC Status Report) was submitted on January 24,
2014 to the Board of Governors (BOG).
The BOG regulation 8.016 requires universities to submit a status report on student learning outcomes assessment
for each baccalaureate program. The following elements are key components of the report:
(1) Identification of expected core student learning outcomes.
(2) Posting of expected core student learning outcomes on the Web.
(3) Identification of corresponding assessment tools.
92%
67%
83%
54%
92%
46%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
2008-2009 2009-2010 2010-2011 2011-2012 2012-2013 2013-2014
December 2014 page 12
(4) Posting of examples of corresponding assessment tools on the Web.
(5) Development of a system of program assessment/evaluation.
(6) Use of information to improve student achievement and program effectiveness.
The BOG expects each of the before mentioned elements be reviewed for possible improvement with the
exception of new programs early in their review cycles or programs undergoing significant modifications. Table
2 provides a summary report on student learning outcomes submitted to the Board of Governors for 2013-2014.
Table 2 Student Learning Outcomes Progress Report to the Board of Governors (2013-2014)
ALC Status Report
Process/Product Totals
1. Identification of expected core
student learning
outcomes.
2. Core learning expectations posted on the
Web.
3. Identification of corresponding
assessment tools
4. Examples of
corresponding assessment
tools posted on the Web.
5. Development of
a system of program
assessment/ evaluation.
6. Use of information to improve student achievement and
program effectiveness.
CI 54 54 54 53 53 52
PC 0 0 0 0 0 1
NS 0 0 0 1 1 1
CI = Evidence That Process/Product is Completed and in Continuous Improvement Mode. PC = Evidence That Process/Product Is Partially Completed. NS = No Evidence That Process/Product is Started.
Based on the status of programs and product 6 of BOG regulation 8.016 (Use of information to improve student
achievement and program effectiveness) a total of (n=52) out of (N=54) programs have shown evidence that
information is used to improve student learning. This information is gathered and determined from Steps 5&6 in
the FAMOUS Assessment Approach. One program showed evidence that the program has partially started using
information for improvement. This program had only been in existence for one year and as such, only baseline
data was available for use. Next year, they will have comparable data to analyze. As can be discerned from Table
2, one program had no evidence of continuous improvement. This program is new and was approved by the BOG,
however, students will not be accepted until the upcoming fall semester. As such, no data was available to utilize
for program improvement. Figure 4 provides a graphical representation of the use of assessment data to improve
student learning outcomes and overall program quality and effectiveness.
Figure 4
Use of Information to Improve Student Achievement and Program Effectiveness
96%
2%2%CI
PC
NS
December 2014 page 13
Administrative and Educational Support Units Activities
This section of the 2013-2014 OUA Annual Report highlights trends in the submission rates for assessment reports
from Administrative and Educational Support Units (ADESUs).
Trends in Administrative and Educational Support Units Assessment Report Submissions
Information highlighted in this section includes the status of submissions of assessment reports. Full assessment
reports can be found at www.famu.edu/assessment. Assessment reports are meant to highlight progress made
in assessment efforts. It is recommended that each unit use their assessment report and results to celebrate
achievements as well as to identify potential areas for future improvement. The Office of University Assessment
will review your reports to provide constructive feedback, as well as identify particular units for potential
assessment recognition.
Administrative and Educational Support Units are inclusive of departments that offer administrative services and
departments that offer educational support services. Table 3 provides a summary of the submission rates for
assessment reports from years 2005-2014. The percentage represented for 2013-2014 is as of January 5, 2014.
Table 3
Assessment Report Submission Rates for Administrative and Educational Support Units (ADESUs)
2005-
2006
2006-
2007
2007-
2008
2008-
2009
2009-
2010
2010-
2011
2011-
2012
2012-
2013
2013-
2014
Submission Rate for
Administrative Units and
Educational Support
Services
17% 86% 97% 79% 89% 80% 82% 83% 57%
Number of ADESUs 60 58 64 57 55 52 56 52 53
In 2013-2014, assessment reports were collected from 53 departments. Figure 5 provides a graphical
representation of the data reported in Table 3. Consistent with the findings for Instructional Programs, timely
submission of assessment reports tend to increase in years that the university is preparing for accreditation visits.
The OUA promotes assessment for continuous improvement, not for re-accreditation. We will continue to push
for 100% submission and encourage units to make assessment a natural part of their daily routine.
December 2014 page 14
Figure 5
Trends in Assessment Report Submission Rates for Administrative and Educational Support Units
Institutional Level Assessment Activities
This section of the 2013-2014 OUA Annual Report highlights key findings from a variety of inventories designed
to provide deeper insights into the experiences, disposition, skills and abilities of FAMU students. Discussed
within this section are key findings from the following instruments: FAMU Exit Survey, ETS Proficiency Profile
and the National Survey of Student Engagement (NSSE). For a comprehensive report of the findings from these
inventories, please contact the Office of University Assessment at assessment@famu.edu.
Exit Survey
Florida A&M University (FAMU) exit survey is designed to capture graduating students’ perceptions of their
collegiate experiences and future plans. The survey specifically seeks respondents’ perceptions of their
experiences as it relates to learning outcomes, student support services, facilities, availability of services, major
field of study, among other items deemed important to the university. This report provides a summary of data
collected for the 2013-2014 academic year. Over the course of this time period (N=2,197) students graduated
from the university. Of this population, (n=2,042) students responded to the survey representing an overall
response rate of 92.9%. Table 4 provides a comprehensive summary of select items from the exit survey. The
rows highlighted in gray reflect the summary results for each category in the survey. These numbers represent
an average for all items within that category. Items in which average scores are ≤ 60% are italicized in Table 4.
17%
86%
97%
79%
89%
80% 82% 83%
47%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
December 2014 page 15
Table 4
Summary of Exit Survey Results for 2013-2014
Select Items FAMU Results
Total Respondents
(n)
% Very Satisfied &
Somewhat Satisfied
Student Support Services* 1996 63.25%
Registration Process 2001 76.06%
Financial Aid Process 1996 32.31%
Grade Reporting Process 1994 79.59%
Advising Process 1991 65.04%
Facilities* 1998 78.48%
Classroom facilities 2003 83.08%
Computing facilities 1999 77.39%
Dining facilities 2001 65.27%
Recreational facilities 1998 76.93%
University bookstore 1997 79.67%
University libraries 1995 87.97%
Fitness/Exercise facilities 1996 79.11%
Departments at FAMU* 1997 60.41%
Campus Police 2001 66.62%
Office of Financial Aid 1995 33.63%
Office of Food Services 2000 55.00%
Housing Office 1997 44.97%
Office of Parking Services 1995 42.76%
Registrar's Office 1996 73.40%
Your major program's department office 1999 79.14%
Library Services 1992 87.75%
Availability of Services* 1998 63.17%
Availability of desired classes 2001 69.57%
Availability of parking 1996 26.35%
Size of classes 1997 83.73%
Quality of the food served on campus 2000 59.45%
Student Government Association 2001 52.92%
Campus e-mail account 1992 61.50%
Campus radio station 1993 62.52%
The value of your FAMU degree 2002 89.26%
Educational Outcomes* 1980 92.66%
Communication 1884 94.16%
Critical Thinking 1996 95.34%
Technology literacy 1996 86.47%
Collaboration 1999 94.90%
Ethical Values 1997 91.99%
Life-long Learning 1998 93.09%
Cultural Diversity 1990 92.76%
December 2014 page 16
Select Items FAMU Results
Total Respondents
(n)
% Strongly Agree &
Somewhat Agree
Major Field of Study* 1985 89.23%
I have obtained a sense of competence in my major
field of study. 1989 96.08%
I have acquired the ability to apply theoretical
knowledge to a practical situation. 1985 96.68%
I feel my major field of study was intellectually
challenging. 1985 94.76%
I feel my major field of study was easily done
academically. 1981 69.36%
Item 29 from the exit survey asked respondents – “Please indicate to what extent the following
items were important in your decision to attend FAMU:” Results are summarized below.
Total Respondents
(n)
% Very Important &
Somewhat Important
Collegiate Experience* 1993 55.67%
Intercollegiate athletics (such as Football,
Basketball, etc.). 1996 30.01%
Marching 100 Band. 1995 41.30%
Total Respondents
(n)
% Very Important &
Somewhat Important
On-Campus Social Events. 1995 44.81%
Student Clubs and Org. 1985 50.83%
Reputation of the Degree Programs. 1991 83.22%
Reputation of Florida A&M University. 1996 83.87%
Item 34 from the exit survey asked respondents – “How confident are you about your ability to find a
job related to your major area of study after earning your degree from FAMU?” Results are
summarized below.
Employment
I have already accepted a job offer 1984 12.00%
Very confident 1984 35.64%
Confident 1984 23.64%
Somewhat confident 1984 23.89%
Not confident at all. 1984 4.84%
Total Respondents
(n)
% Very Important &
Somewhat Important
Item 31 from the exit survey asked respondents – “Would you recommend Florida A&M University
(FAMU) to others who may be interested in your field of study?” Results are summarized below.
Likelihood of Recommending FAMU to Others
Yes 1747 87.31%
No 254 12.69%
Note: (*) Represents the average number of students and their corresponding scores for topical area.
December 2014 page 17
ETS Proficiency Profile (EPP)
The ETS Proficiency Profile (EPP) test is a General Education knowledge and skills test and one of the tests used
by the Voluntary System of Accountability (VSA). The test measures critical thinking and college-level reading,
writing, and mathematics skills in the contexts of the humanities, social sciences, and natural sciences. In an
effort to evaluate General Education learning outcomes, the University currently requires students to take the
ETS EPP test at entry (freshmen) and exit (senior) levels. Additional information about the EPP can be found at
http://www.ets.org/proficiencyprofile/about.
FALL 2013
In an effort to encourage student participation in the test, the decision was made in Fall 2013 to incentivize
participation. The student who scored the highest on the test received an IPAD. All students who scored a 470 or
better (the score range is 400-500) were entered into a raffle for an IPOD, from which three winners were chosen.
In an effort to bring about awareness for the EPP the Office of University Assessment engaged both freshmen and
seniors in dialogue relative to how the test is used as a mechanism for continuous improvement.
As a result of these efforts, in Fall 2013, (n=334) incoming freshmen attempted to take the EPP test. Results are
calculated for (n=308) of those students who completed 75% or more of the test items. The results showed that
one student had a perfect score of 500 among the five top performers on the test. A total of 10 students scored
470 or above. Table 5 provides a summary of the EPP top performers, their intended major field of study and
scores.
Table 5
ETS Top Scorers
Name Intended Major Total Score
Fralin, Phylesia, A Accounting 500
McGowan, Calvin, E Biological Sciences 493
Roy, Rais Health & Medical Sciences 493
Gonzalez, Jorge Biological Sciences 490
Onubogu, Ugoma, C Biological Sciences 486
The national average for graduating seniors and first year students sitting the ETS Proficiency Profile was
(M=447.89) and (M=435.31) respectively. The national mean provided was taken from the annual comparative
data guide provided by ETS. The report provides descriptive statistics based on the number of students that have
completed an un-proctored version of the ETS Proficiency Profile between July 1, 2008 and June 30, 2013. A
five year rolling average is utilized to normalize the scores. The anonymity of each institution’s performance is
maintained by reporting only the aggregate performance of the selected reference group. As a way of gaining
insights into FAMU Freshman and Seniors’ overall performance on the EPP, Tables 6 and 7 provides a historical
overview of performance. As can be discerned from Table 6, the performance of FAMU Freshmen has been
relatively flat since 2011. Relative to the performance of FAMU Seniors, there was much variability in overall
mean scores since 2009. Please refer to Tables 6 and 7 for a comprehensive overview of these results. Figures 6
and 7 provide a graphical representation of the overall mean performance of FAMU students.
December 2014 page 18
Table 6
Trends in Incoming Freshmen Performance on the ETS Proficiency Profile (2006-2013)
2006 2007 2008 2009 2010 2011 2012 2013
Mean Score 431.19 436.16 434.8 430.41 429.67 431.61 431.26 431.68
Skill Dimension 2006 2007 2008 2009 2010 2011 2012 2013
Critical Thinking 108.42 109.38 108.85 108.08 107.56 108.28 107.65 107.97
Reading 113.98 116.24 115.4 113.59 113.07 113.52 113.54 113.57
Writing 112.01 112.91 113.67 112.17 111.95 112.36 112.12 112.43
Mathematics 110.09 111.22 110.04 109.88 110.49 110.77 111.15 110.97
Humanities 111.29 113.37 112.5 111.62 112.43 112.57 112.64 113.17
Social Sciences 110.78 111.88 111.37 110.51 110.03 110.72 110.42 110.29
Natural Sciences 112.21 113.48 113.01 111.96 111.17 111.91 111.05 111.54
Percent Proficient 2006 2007 2008 2009 2010 2011 2012 2013
Reading, Level 1 41% 55% 51% 34% 29% 32% 32% 31%
Reading, Level 2 13% 23% 20% 12% 10% 12% 12% 13%
Critical Thinking 1% 2% 3% 1% 1% 1% 3% 2%
Writing, Level 1 43% 52% 53% 39% 35% 34% 37% 40%
Writing, Level 2 7% 10% 19% 8% 9% 9% 10% 11%
Writing, Level 3 2% 3% 7% 2% 3% 4% 3% 3%
Mathematics, Level 1 37% 42% 29% 28% 28% 28% 29% 32%
Mathematics, Level 2 11% 16% 14% 8% 9% 9% 11% 12%
Mathematics, Level 3 2% 2% 4% 2% 2% 2% 3% 2%
December 2014 page 19
Figure 6 Trends in Mean Score of Incoming Freshmen on ETS Proficiency Profile
SPRING 2014
In an effort to evaluate the General Education Program and the flow of knowledge within degree programs, all
students graduating with a bachelor’s degree are asked to take the ETS Proficiency Profile (EPP). In Spring 2014,
(n=600) out of (n=981) graduating seniors took the EPP test. Results are calculated for (n=537) students who
completed 75% or more of the test. Table 7 provides a comprehensive overview of the performance of FAMU
seniors covering the last six years.
Table 7
Trends in Graduating Seniors Performance on the ETS Proficiency Profile (2009-2013)
2009 2010 2011 2012 2013 2014
Mean Score 438.08 432.97 430.84 434.64 434.28 430.56
Skill dimension 2009 2010 2011 2012 2013 2014
Critical Thinking 109.88 108.55 108.28 109.07 109.18 108.09
Reading 115.81 114.8 113.79 114.85 114.53 113.69
Writing 113.48 112.14 111.54 112.35 112.61 111.45
Mathematics 112.4 111.04 110.75 111.53 111.2 110.9
Humanities 113.47 113.74 112.67 114.03 113.51 112.88
Social Sciences 111.88 111.11 111.01 111.39 111.34 110.55
Natural Sciences 113.72 112.34 112.02 112.56 112.98 111.76
Percent Proficient 2009 2010 2011 2012 2013 2014
2006 2007 2008 2009 2010 2011 2012 2013 2014
Mean Score* 431.19 436.16 434.8 430.41 429.67 431.61 431.26 431.68 429.13
National Avg. 440.50 440.50 440.50 440.50 440.50 440.50 435.31 435.31 435.31
422
424
426
428
430
432
434
436
438
440
442
Me
an S
cale
d S
core
December 2014 page 20
2009 2010 2011 2012 2013 2014
Reading, Level 1 51% 39% 32% 37% 37% 35%
Reading, Level 2 24% 18% 13% 17% 16% 13%
Critical Thinking 2% 1% 0% 3% 4% 1%
Writing, Level 1 54% 36% 32% 39% 41% 32%
Writing, Level 2 15% 11% 10% 11% 13% 10%
Writing, Level 3 5% 4% 3% 4% 4% 2%
Mathematics, Level 1 47% 31% 31% 33% 31% 31%
Mathematics, Level 2 25% 15% 12% 15% 16% 15%
Mathematics, Level 3 7% 4% 2% 4% 3% 4%
Figure 7 Trends in Mean Score of Graduating Seniors on ETS Proficiency Profile
Item Analysis Results
Areas of Strength: The following areas were found to be consistently above the national average for FAMU
students. Note that a majority of the content areas identified fell within the Mathematics category.
Freshmen:
Mathematics - Apply formula
2009 2010 2011 2012 2013 2014
Mean Score* 438.08 432.97 430.84 434.64 434.28 430.56
National Avg. 448.46 448.46 448.46 448.46 448.46 447.89
420
425
430
435
440
445
450
Me
an S
cale
d S
core
December 2014 page 21
Mathematics - Data interpretation- read information
Mathematics - Exponential functions
Mathematics - Negative and positive integers- average
Mathematics - Number line
Reading - Recognize explicit information
Writing - Recast an existing sentence
Writing - Recognize correct construction
Seniors:
Mathematics - Word problem- logarithmic function
Mathematics - Apply formula
Mathematics - Word problem- algebraic equation
Areas of Weakness: The following areas were found to be significantly below the national average score for
FAMU students.
Freshmen:
Critical Thinking - Discern purpose of a reference
Critical Thinking - Evaluate an argument
Critical Thinking - Evaluate data for consistency
Critical Thinking - Evaluate Hypothesis
Critical Thinking - Recognize a valid inference
Critical Thinking - Recognize an assumption
Mathematics - Translation to algebraic expression
Reading - Discern facts from passage
Reading - Discern primary purpose
Reading - Recognize a valid inference
Reading - Recognize explicit information
Writing - Recognize agreement
Seniors:
Critical Thinking - Discern purpose of a reference
Critical Thinking - Evaluate an argument
Critical Thinking - Evaluate Hypothesis
Critical Thinking - Recognize assumption
Reading - Discern facts from passage
Reading - Discern primary purpose
Reading - Recognize explicit information
Reading - Recognize valid inference
Reading - Synthesize material
December 2014 page 22
Writing - Appropriate Connector
Writing - Recast an existing sentence
Writing - Recognize coordination
Writing - Recognize grammatical correction
The results continue to confirm that our students are entering the university with a deficiency in critical thinking
skills. This year, an emerging concern is students writing and reading skill. It is important that more emphasis is
placed on these skills and its relationship with students’ critical thinking. For the seniors, more development is
essential in the area of writing. By improving writing and reading across the curriculum, this will enhance
students’ critical thinking skills, which in turn affect their reading comprehension and writing relationship skills.
Finally, the focus to improve critical thinking needs to naturally flow into the programs and not just in the general
education sequence. Additional information about the EPP can be found at
www.ets.org/proficiencyprofile/about.
National Survey of Student Engagement
The National Survey of Student Engagement (NSSE) is designed to capture students’ perceptions of their
collegiate experiences to include how students spend their free time, what they have gained form classes and
their interactions with faculty and peers. The NSSE is administered annually at many colleges and universities
nationwide. The NSSE was first administered at Florida A&M University (FAMU) in 2005 and was subsequently
administered in 2008 and then in 2011. The 2014 administration of NSSE represents the fourth time that this
instrument has been administered at FAMU. FAMU results for each indicator assessed on the instrument are
analyzed and compared with that of the group of institutions in the State University System (SUS) i.e. (n=9),
Carnegie Classification of similar institutions (n=54) and the NSSE 2013 & 2014 group of institutions.
The instrument is administered to first-year and senior students to gain their perspectives on programs and
activities that the institution provides for their learning and personal development. The results provide an
estimate of how undergraduates spend their time and what they gain from attending college. The 2014 NSSE was
launched at FAMU on February 13, 2014. There were four reminders sent to non-responders. The survey closed
on April 15, 2014 with an overall response rate of 25% for first year students and 23% for seniors. Taken as a
group, there were (n=835) students who participated in the survey. When compared to the average response
rate for the State University System (SUS) in Florida, FAMU’s response rate was four percentage points higher for
first-year students and two percentage points lower for seniors. Following is a summary of key findings from
the 2014 administration of NSSE:
FAMU First Year students and seniors perceived higher levels of engagement with faculty when
compared to other students from institutions in the SUS, Carnegie Classification and NSSE 2013 & 2014
taken as a group.
FAMU Seniors perceived higher levels of learning with peers as described by the engagement indicator
“Collaborative Learning” when compared to other students from institutions in the SUS, Carnegie
Classification and NSSE 2013 & 2014 taken as a group.
December 2014 page 23
FAMU First Year students perceived lower levels of engagement in the areas of “Discussions with
Diverse Other”, “Effective Teaching Practices” and “Quality of Interactions” when compared to other
students from institutions in the SUS, Carnegie Classification and NSSE 2013 & 2014 taken as a group.
FAMU Seniors perceived lower levels of engagement with respect to “Quality of Interactions” when
compared to other students from institutions in the SUS, Carnegie Classification and NSSE 2013 & 2014
taken as a group.
The majority of the respondent sample rated their overall experience at FAMU as excellent or good
(First year students – 70% and Senior – 82%).
Seventy-nine percent of seniors as compared to 82% for the Carnegie Classification of students
stated that they would definitely or probably attend FAMU again.
While the majority of the respondent sample were satisfied with their overall experiences at FAMU, the results
suggest that much more must be done in an effort to address areas of opportunity especially with respect to
activities aligned with the following engagement indicators: Discussion with Diverse Others, Effective
Teaching Practices, Quality of Interactions.
Students Assessment of Their Experiences at FAMU
Students' perceptions of their cognitive and affective development, as well as their overall satisfaction with the
institution provide useful evidence of their educational experiences. This section of the report highlights key
findings relative to FAMU students’ perceived gains as a result of attending FAMU. Table 8 provides a summary
of FAMU students’ perceptions of their experiences and how FAMU contributed to their knowledge, skills, and
personal development in ten areas. As can be discerned from this data, 70% or more of graduating seniors felt
that FAMU contributed “very much or “quite a bit” to their knowledge, skills and personal development. Please
refer to Table 8 for a summary of these results.
Table 8 Perceived College Gains of FAMU Seniors
Perceived Gains (Sorted highest to lowest) Percentage of Seniors Responding "Very much" or
"Quite a bit"
Thinking critically and analytically 85%
Working effectively with others 83%
Speaking clearly and effectively 78%
Developing or clarifying a personal code of values and ethics
78%
Acquiring job- or work-related knowledge and skills
77%
Being an informed and active citizen 75%
Writing clearly and effectively 74%
December 2014 page 24
Perceived Gains (Sorted highest to lowest) Percentage of Seniors Responding "Very much" or
"Quite a bit"
Solving complex real-world problems 73%
Analyzing numerical and statistical information 72%
Understanding people of other backgrounds (econ., racial/ethnic, polit., relig., nation., etc.)
70%
Academic Advising
The Academic Advising module on the NSSE examines students' experiences with academic advising, including
frequency, accessibility, and types of information provided. It also asks students to identify their primary source
of advice. This section of the report summarizes FAMU students’ experiences with academic advising and
compares the results to the comparison group in the 2014 administration of NSSE. As can be discerned from
Tables 9 and 10, for both first-year and seniors, the mean difference between FAMU students responses and the
comparison group relative to the following item (During the current school year, about how many times have you
and an academic advisor discussed your academic interests, course selections, or academic performance?) was
statistically significant with a small to medium effect size. For the majority of items measured within this section
for both first-year and seniors, there were no meaningful differences between FAMU students and that of the
comparison group. This would suggest that while FAMU students spend more time on average with their academic
advisor, there is room for improvement relative to the quality of those interactions. The majority of FAMU
students (both first-year and seniors) also indicated that the academic advisor assigned to them has been their
primary source of advice relative to their academic plans. Please refer to Tables 9 and 10 for a summary of
these results.
Table 9 Comparative Overview of FAMU First –Year Students Experiences with Academic Advising
Variable FAMU
M
Comparison
Group M
p Effect Size
(Cohen’s d)
During the current school year, about how
many times have you and an academic advisor
discussed your academic interests, course
selections, or academic performance?
Note: The response scale for this item ranged
from 0- 6 or more.
2.80 2.11 .000 .45
During the current school year, to what extent have your academic advisors done the following?
Note: The response scale for this item was as follows: Very Little (1), Some (2), Quite a bit (3), Very
much (4).
Been available when needed 2.88 2.98 .138 -.10
Listened closely to your concerns and questions 3.00 3.00 .933 .01
December 2014 page 25
Variable FAMU
M
Comparison
Group M
p Effect Size
(Cohen’s d)
Informed you of important deadlines 2.87 2.81 .435 .05
Helped you understand academic rules and
policies
2.85 2.77 .289 .07
Informed you of academic support options
(tutoring, study groups, help with writing,
etc.)
2.69 2.70 .881 -.01
Provided useful information about courses 2.87 2.85 .700 .03
Helped you when you had academic difficulties 2.64 2.64 .924 -.01
Helped you get information on special
opportunities (study abroad, internship,
research projects, etc.)
2.59 2.52 .369 .06
Discussed your career interests and post-
graduation plans
2.59 2.50 .217 .09
Significant at p<0.05 level
Table 10 Comparative Overview of FAMU Seniors Students Experiences with Academic Advising
Variable FAMU
M
Comparison
Group M
p Effect Size
(Cohen’s d)
During the current school year, about how
many times have you and an academic advisor
discussed your academic interests, course
selections, or academic performance?
Note: The response scale for this item ranged
from 0- 6 or more.
2.84 2.26 .000 .33
During the current school year, to what extent have your academic advisors done the following?
Note: The response scale for this item was as follows: Very Little (1), Some (2), Quite a bit (3), Very
much (4).
Been available when needed 2.56 2.95 .000 -.39
Listened closely to your concerns and questions 2.77 2.96 .000 -.18
Informed you of important deadlines 2.68 2.73 .368 -.04
Helped you understand academic rules and
policies
2.64 2.64 .954 .00
Informed you of academic support options
(tutoring, study groups, help with writing,
etc.)
2.37 2.39 .730 -.02
Provided useful information about courses 2.58 2.69 .037 -.10
Helped you when you had academic difficulties 2.54 2.58 .505 -.03
December 2014 page 26
Variable FAMU
M
Comparison
Group M
p Effect Size
(Cohen’s d)
Helped you get information on special
opportunities (study abroad, internship,
research projects, etc.)
2.42 2.45 .642 -.02
Discussed your career interests and post-
graduation plans
2.52 2.47 .324 .05
Significant at p<0.05 level
Office of University Assessment Activities/Updates
This section of the 2013-2014 OUA Annual Report highlights the newly introduced FAMU STARS Assessment Model
in addition to providing an overview of the Office of University Assessment focused objectives for 2014-2015.
The section concludes with a summary of data from workshops and training sessions conducted over the course
of the year.
STARS Assessment Model
In the 2013-2014 academic year, the Office of University Assessment (OUA) began to transition the university
from the FAMOUS assessment approach to the STARS assessment approach, a five step approach to quality
assessment designed to provide tighter alignment of outcomes to the University’s Strategic Initiatives and
President’s goals. The FAMU STARS assessment approach was also designed to allow for the aggregation and
disaggregation of outcomes and outcome types at the program, unit and division level. During the Spring and
Summer of 2014, the OUA embarked on a university-wide initiative geared towards socializing assessment
coordinators and others responsible for assessment in their units to the new approach to quality assessment.
Figure 8 provides a graphical representation of the FAMU STARS assessment process.
December 2014 page 27
Figure 8
FAMU STARS Assessment Process
OUA Focused Objectives for 2014-2015 Assessment Cycle
Consistent with its main goal of strengthening institutional assessment, the OUA objectives for the 2014-2015
academic year follows:
Strengthen the Office of University Assessment through the use of performance evaluation measures for
instructional and administrative units.
Increase university awareness of assessment best practices, including the understanding of curriculum
and assessment mapping.
Enhance the electronic monitoring database of assessment reports in an effort to improve the review
and communication process relative to submitted documents.
Maintain accreditation standards by assisting programs in preparing for re-accreditation or program
review in an effort to maintain regional/specialized accreditation and enhance quality.
December 2014 page 28
Increase data and recommendations for improvements provided to university administrators based on
analysis of surveys administered to the university community.
Workshops/Training Sessions
In an effort to build capacity for assessment that matters, the OUA facilitated Assessment training sessions
with all Assessment Coordinators and others responsible for assessment within their units across the university.
The purpose of these training sessions were as follows:
a) Reinforce the mission of the Office of University Assessment
b) Getting back to the basics with quality assessment
c) Gain a deeper understanding of the planning process for the development of quality outcomes and
measures
d) Close the loop on the assessment process
e) Introduce STARS, a re-engineered approach to assessment
These training sessions afforded participants the opportunity to engage in meaningful ways with the OUA as we
work to broaden our impact on assessment for the university. The targeted training sessions were well received.
Over the course of the 2013-2014 academic year a total of (35) training sessions were conducted. There was a
general consensus among the participants that the sessions were helpful in increasing their understanding of best
practices in assessment. Participants also expressed that among others, the most beneficial aspect of the training
was the transition to the FAMU STARS assessment approach. Based on the success of this approach in engaging
the university community around quality assessment, the OUA will continue this format during the course of the
next assessment cycle. Table 11 provides a summary of participants’ perceptions of their experiences relative
to the training in addition to their experience interfacing with the OUA.
Table 11 Summary of 2014 University Wide Assessment Training
Survey Item n % Strongly Agree/Agree
Items relative to training session The presentation materials provided will be useful in helping me with assessment in my unit.
135 97.8%
The speaker(s) seemed knowledgeable of the discussion topics. 135 100% The discussion topics were relevant to assessment. 135 99% I found the topics discussed during the session to be applicable to my work and responsibilities.
135 99%
The knowledge and skills I learned in this training session will inform the assessment process in my unit.
135 96%
Overall, I am satisfied with the quality of today's training session. 135 99% Items relative to services provided by the OUA
December 2014 page 29
Survey Item n % Strongly Agree/Agree
The OUA staff provides timely response to requests for information and feedback.
100 99%
The OUA staff addressed any questions or concerns to my satisfaction. 115 99% I am satisfied with the quality of services (i.e., assessment trainings and survey design/administration) provided by the OUA.
113 96%
I am satisfied with the quality of support (i.e., accreditation preparation and data analysis) offered by the OUA.
107 98%
Overall, I am satisfied with the quality of the services provided by the OUA.
111 99%
n M How likely is it that you would recommend this assessment training to your colleagues? (Note: This item is rated on a ten point scale with anchored responses ranging from Not at all likely to Extremely Likely)
121 9.30
Note: Percentages are rounded to the nearest whole number. Items relative to services provided by the OUA was scored on a five point Likert scale with responses ranging from Strongly Agree to Strongly Disagree. The fifth point on the scale was labeled n/a, to accommodate for individuals who could not assess these items because of a lack of interface with the OUA. Individuals selecting this response category were removed from the final analysis.
Assessment Committees Activities/Updates
This section of the 2013-2014 OUA Annual Report provides an update on activities carried out by several university
committees to include the General Education Assessment Committee (GEAC), the Quality Enhancement Plan
Committee (QEP) and the Institutional Level Assessment Committee (ILAC).
General Education Assessment Committee Activities/Updates
The General Education Assessment Committee (GEAC) continues to fulfill its charge with a particular focus in
developing, implementing, and monitoring the General Education assessment processes for the University. During
the academic year 2013-2014, the main focus in General Education Assessment was to finalize all changes to the
current master list of general education courses. The following summarizes the activities of the GEAC.
Determined university general education requirements as mandated by recent legislation (BOG
Regulation 8.005)
Evaluated and revised master list of courses for General Education
Reviewed recommendation for additional student learning outcomes
Reviewed strategic suggestions for communications outcomes (as provided by faculty from the
Department of English and Modern Languages)
Completed Focus Groups (April 2013 and December 2014)
Collected student artifacts and conducted evaluation for assessment purposes
Provided updates to the Institutional Effectiveness Committee December 2014
December 2014 page 30
Provided regular GEAC updates/ reports to the Institutional Level Assessment Committee during the 2013
– 2014 academic year
Perception of General Education/Liberal Arts Education
Results of a focus group exercise conducted with FAMU students revealed that our students associate the general
education curriculum with specific subject areas i.e. the humanities, natural sciences, etc. It was also suggested
that the general education curriculum is a body of knowledge that can be applied to one’s academic career
regardless of one’s chosen major. Students see the inherent value in the general education curriculum; however
they, question its compulsory nature given the associated cost and the potential negative impact on the student’s
GPA if he/she fails the course. One student argued, “if the general education curriculum is liberal arts let me be
liberal (the individual right to choose)” about his/her choice of general education courses. The university has
the responsibility to provide the option of taking general education courses as opposed to forcing students to
take courses unrelated to their major. A general education curriculum should only be mandatory under a system
of higher education that is “public” in the purest sense. If students are responsible for paying for their education
they should have the right to choose.
Recommendations
Student responses from a focus group revealed that students believe:
Beyond writing, both speaking and interacting effectively are important communication skills that should
be taught in the general education curriculum.
The pervasiveness of social media technology undermines the ability of students to communicate
(writing, verbal communication, etc.) effectively.
Communication courses are absolutely necessary to combat the negative impact of social media.
In an effort to use feedback to further improve programmatic elements of the English and Modern Languages
Department, and to address recent mandates that impact how the communication requirement is satisfied, the
following recommendations were provided by faculty from the English and Modern Languages Department.
1. All students enrolled at FAMU should take the 4 writing intensive courses as stipulated by the Gordon
Rule. It seems that currently all students are not fulfilling the 24,000 word requirement for writing
competency.
a) Perusal of the web reveals that some departments require only 1 additional course for their
majors beyond the 2 required English composition courses.
b) Also, speech is listed as an additional course for some departments, and our students
certainly can benefit from taking the course, but enhancement of oral communication skills
cannot be substituted for enhancement of written expression.
2. The courses in General Education be monitored more closely to ensure intense instruction is provided,
and required quantity of writing is completed.
December 2014 page 31
Institutional Level Assessment Committee Activities/Updates
The Institutional Level Assessment Committee (ILAC) continued to monitor the assessment activities of the
university at the institutional level during the 2013-2014 academic year. The committee successfully held
meetings during which they discussed, decided and/or made propositions on important subjects regarding the
University's assessment activities and processes. The committee met during the fall and spring semester. The
committee took time to reflect and revisit its purpose/role and began the process of re-establishing its role in
the institutional level assessment activities of the university. Table 12 provides a list of tasks that were addressed
during the 2013-2014 academic year. Please be advised that some activities will be on-going and will continue
beyond the current year:
Table 12
ILAC Tasks and Outcomes for 2013-2014
Task Purpose of Task Action Item Outcomes
ETS
Purpose is to assess student basic skills and critical thinking. Revise policy statement to reflect percentages of schools and colleges that will identify student participants.
Updated – October 2013 and reviewed by ILAC for resubmission to ILAC, IE, and policy committee.
Senior Exit Survey
Purpose is to update Senior Exit Survey and create alignment with the goals of the university concerning student academic outcomes and activities – link results to tangible outcomes.
Updated – October 2013, April 2014 and presented to ILAC.by Mr. Francis. Further recommendations made concerning changes needed. Further review will occur by ILAC.in Fall 2014
SAC’s 5th-year Report
The fifth-year interim report was developed to respond to the U.S. Department of Education’s requirements (1) that accrediting bodies continuously monitor institutions to ensure complaints and (2) that accrediting bodies have a mechanism for reviewing multiple sites initiated since the last reaffirmation. ILAC will be responsible for responding to CR 2.10 (Student Support Services, CS 3.3.1.1 (Institutional Effectiveness Educational Programs, and CS 3.1.1.3 (Physical Facilities).
Reviewed by ILAC – Subcommittee formed. October 2013. No further action taken waiting for directive from IE. Will pursue after direction.
Campus Police Survey/FAMOUS Assessment Request
To assist the Campus in obtaining assessments on student’s view of safety on campus and this department’s effectiveness in addressing campus-wide needs concerning safety.
Update – October 2013
December 2014 page 32
Task Purpose of Task Action Item Outcomes
The Police Department will now be required to complete the ADESU FAMOUS Assessment Plan to document levels of safety of students.
FAMOUS Assessment Training
Linking Assessment results to outcomes – A focus on rows 5 and 6 of the FAMOUS Model.
TBA
Linking Assessment to Student Assessment
Increasing oral and written reports documenting a closed loop of student’s assessment results in final outcomes including resources and etc. Assessment Briefs in FAMUAN
Provided non-fiscal examples and recommendations to document closing the loop for student outcomes to ILAC.
Pursue Closing the Loop Funding
Identify other funding opportunities to assist programs in financing data driven recommendations.
Recommendations submitted to IE. No further action taken
Quality Enhancement Plan Committee Activities/Updates
The 2013-2014 academic year marked the final year of the Quality Enhancement Plan (QEP) introduced five years
earlier. The Office of the Quality Enhancement Program maintained and forged new collaborations with various
units across campus to execute various student and faculty-oriented activities. Some of the student activities
included the Freshman Summer Reading Program, the Freshman Critical Thinking Seminar Series, and the First
Year Experience Course. In addition to student activities, the QEP also hosted various faculty-oriented
professional development opportunities that were instrumental in changing the campus culture. Below are
highlights of activities sponsored by the Quality Enhancement Program.
Year 5 of the Freshman Summer Reading Program
This co-curricular activity promotes student learning by requiring all incoming freshman students to read
a common book and complete a writing assignment during the summer preceding the fall term.
Losing My Cool was selected as the 2014 Common Book selection.
This year the QEP and the Division of Housing sponsored “Dorm Dialogues” in seven residential halls
across campus. The designated day for dialogue allowed students to have intellectual and critical
discussions about the common book selection. All incoming freshmen received a copy of Losing My Cool
during the summer prior to coming to campus.
Students completed written responses on the common reading selection and submitted these essays in
their First Year Experience Course (SLS 1101). More than 400 students participated in the Dorm
Dialogues.
QEP provided instructional training to FYE faculty on how to utilize the book through the use of critical
thinking pedagogy strategies.
December 2014 page 33
The author of the 2014 book selection (Losing My Cool) visited the campus during the fall semester to
facilitate in discussions with the University community. Six hundred sixty-three (663) freshman students
attended the Author’s Seminar.
Year 5 of the Freshman Critical Thinking Seminar Series
This co-curricular activity is designed to enhance students’ critical thinking skills through the
participation in interactive dialogue sessions addressing life skills and academic issues.
Approximately 1,700 students participated in critical thinking seminars during the 2013 – 2014 academic
year.
Topics of the 2013-2014 critical thinking seminar sessions included: Critical Thinking, Financial Literacy,
Race Relations, and Entrepreneurship.
Year 5 of the Faculty Development Activity (Program)
Through this activity (program), faculty are provided opportunities to attend on-campus training sessions,
travel to professional conferences, and engage in curriculum development activities.
Thirty-seven (37) on-campus workshops and two (2) off-campus workshops (Law School) on critical
thinking, course redesign, and active learning were held during the academic year.
Two hundred seventy-three (273) faculty and staff attended at least one of the workshop sessions.
Five (5) faculty trips to off-campus conferences/workshops, specific to teaching, assessment and student
learning were funded by the QEP during the academic year.
QEP co-sponsored a Faculty Learning Community (FLC) with the Innovative Academic Instructional
Program (IAIP) during the summer of 2014. Nine (9) faculty were selected to participate in a year-long
interactive course redesign exercise to enhance pedagogy and instructional approaches.
Faculty participating in the FLC also received training to teach in new scale-up classroom.
The QEP/ Faculty Development Activity sponsored a spring faculty planning conference with themed
sessions that addressed active learning and the flipped classroom concept.
The QEP/ Faculty Development Activity launched the Innovative Teaching Award, which recognizes junior
and senior faculty members who are utilizing unique, innovative teaching approaches that enhance
student performance. The nominees and winners of this teaching award were announced and recognized
at the 2014 spring Faculty Planning Conference.
The QEP served as the impetus for the new Teaching and Learning Center, which hosted its first
workshops in October 2014. This Center (TLC) serves a resource venue for faculty, as well as a central
location for training and faculty development workshops for the main campus and all site campuses.
Year 5 of the CAT Test and Assessment
The QEP continued the development and implementation of a uniform approach for assessing the critical
thinking skills of freshman students. The CAT test was administered in the First Year Experience courses.
Preliminary results indicate that 38% (n=138) of freshman students who completed the CAT showed at
least a 5% mean score gain between pre and post-tests scores.
The QEP partnered with the General Education Assessment Committee (GEAC) and the two academic
units to further assess the critical thinking skills of freshman students. Results were reported in the QEP
Impact Report and the annual assessment report for GEAC.
December 2014 page 34
The College of Science and Technology has adopted the CAT test as their commercial instrument of
choice to also evaluate students’ critical thinking skills.
The Critical Thinking Assessment Test (CAT), critical essays, and case study analyses were administered
to students during the course of the freshman year to evaluate their critical thinking skills. The
assessment results are being used to guide future improvements in faculty instruction and assessment of
students’ critical thinking skills in the classroom setting.
The QEP Report will be submitted as a part of the Fifth Year Interim Report in March 2015.
Licensure/Certification Examination Pass Rates
This section of the report highlights pass rates on licensure/certification examinations. The Board of Governors
(BOG), State University System (SUS), monitors the progress of performance on licensing and certification exams
for specific bachelor’s and professional programs in which licensure is required to practice in the field. In addition
to progress monitored by the BOG, FAMU Board of Trustees (BOT) has established increasing pass rates on
licensure examinations and set target pass rates for the programs in which passing licensure or certification is a
condition of employment in the field. To meet the BOT’s goals, each academic program developed detailed
plans with various strategies on how it plans to achieve the stated goals. Table 13 provides a summary of FAMU
Licensure/Certification pass rates by program. Figures 9 – 14 provide a graphical overview of these results.
As can be discerned from the data collected from respective academic programs, first-time pass rates for the
College of Law have been steadily increasing since 2010, with the highest pass rate earned in 2013. In 2014, a
first-time pass rate in Nursing was equal to the pass rates nationally and surpassed first-time pass rates at the
State level. The College of Law first-time pass rates (i.e. 72% was just under the State level (73%) by 1%). In 2013
and 2014, Cardiopulmonary Science surpassed the national average with 100% of its graduates passing the
licensure examination the first time. Other programs in which the pass rate is below the national or state overall
pass rate are instituting procedures to improve pass rates.
Areas requiring Licensure/Certification Examination
College of Education
o Counselor Education (M.S./M.Ed.)
o Early Childhood Education (B.S.)
o Educational Leadership (M.S.)
o Elementary Education (B.S.)
o Social Science Education
o Theatre Education
o Biology Education
o English Education
o Math Education
o Music Education
o Physical Education (B.S.)
College of Law
December 2014 page 35
o Law
College of Pharmacy and Pharmaceutical Sciences
o Pharmacy (Pharm. D.)
School of Nursing
o Nursing (B.S.)
School of Allied Health
o Health Informatics and Information Management (B.S.)
o Cardiopulmonary Science (B.S.)
o Occupational Therapy (M.S.)
o Physical Therapy (DPT)
Table 13 Longitudinal Overview of FAMU Students’ Licensing / Certification Examination Pass Rates by Program Nursing: National Council Licensure Examination for Registered Nurses
2010 2011 2012 2013 2014*
Examinees 72 73 84 82 36
First-time Pass Rate 85% 85% 90% 74% 89%
State Benchmark 89% 88% 86% 77% 77%
National Benchmark 89% 89% 92% 89% 89%
1st and 2nd Quarter Results: January 01, 2014 – June 30, 2014
Law: Florida Bar Exam
2010 2011 2012 2013 2014
Examinees 108 144 123 175 36
First-time Pass Rate 61% 63% 68% 73% 72%
State Benchmark 78% 88% 79% 78% 73%
Pharmacy: North American Pharmacist Licensure Exam
2010 2011 2012 2013 2014*
Examinees 133 142 130 123 -
First-time Pass Rate 72% 87% 88% 85% -
National Benchmark 94% 96% 97% 96% -
*2014 Results Not Available.
Physical Therapy: National Physical Therapy Examinations
2010 2011 2012 2013 2014
Examinees - - 19 26 16
First-time Pass Rate 54% 39% 53% 46% 81%
State Benchmark - - 77% 85% 94%
National Benchmark 87% 89% 88% 90% 92%
December 2014 page 36
Occupational Therapy: National Board for Certification in Occupational Therapy Exam
2010 2011 2012 2013 2014*
Examinees - - - - -
First-time Pass Rate 50% 33% 60% 90% -
National Benchmark 81% 81% 81% 81% -
*2014 Occupational Results Not Available.
Cardiopulmonary Science: National Board for Respiratory Care Examination
2010 2011 2012 2013 2014
Examinees 11 8 23 21 10
First-time Pass Rate 73% 38% 70% 86% 100%
National Benchmark 88% 76% 85% 79% 85%
Source: FAMU Office of Program Quality
Figure 9
Trends in Nursing Pass Rates: National Council Licensure Examination for Registered Nurses
2010 2011 2012 2013 2014*
First-time Pass Rate 85% 85% 90% 74% 89%
State Benchmark 89% 88% 86% 77% 77%
National Benchmark 89% 89% 92% 89% 89%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
December 2014 page 37
Figure 10 Trends in Law Pass Rates: Florida Bar Exam
Figure 11 Trends in Pharmacy Pass Rates: North American Pharmacist Licensure Exam
2010 2011 2012 2013 2014
First-time Pass Rate 61% 63% 68% 73% 72%
State Benchmark* 78% 88% 79% 78% 73%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
2010 2011 2012 2013
First-time Pass Rate 72% 87% 88% 85%
National Benchmark 94% 96% 97% 96%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
December 2014 page 38
Figure 12
Trends in Physical Therapy Pass Rates: National Physical Therapy Examination
2010 2011 2012 2013 2014
First-time Pass Rate 54% 39% 53% 46% 81%
State Benchmark 77% 85% 94%
National Benchmark 87% 89% 88% 90% 92%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
2010 2011 2012 2013
First-time Pass Rate 50% 33% 60% 90%
National Benchmark 81% 81% 81% 81%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
December 2014 page 39
Figure 13
Trends in Occupational Therapy Pass Rates: National Board for Certification in Occupational Therapy Exam
Figure 14
Trends in Cardiopulmonary Science Pass Rates: National Board for Respiratory Care Examination
Summary of Key University Assessment Findings & Recommendations
This section of the 2013-2014 OUA Annual Report provides a summary of key university assessment findings.
Exit Survey
92.9% or (n=2,042) graduating students completed the exit survey.
Students are overall satisfied with their educational outcomes and services related to their major field
of Study (M=89.23%).
Parking and Financial Aid continue to be problem areas with low satisfaction ratings (Parking M=26.35%,
Financial Aid M=33.63%).
Units should use results, specifically results that indicate low student satisfaction to inform
improvement efforts.
All departments are encouraged to conduct Point of Service (POS) surveys in order to gauge students’
perceptions of their experiences.
ETS Proficiency Profile
(n=334) incoming freshmen attempted the EPP test. Results were calculated for (n=308) of those
students who completed 75% or more of the test items.
2010 2011 2012 2013 2014
First-time Pass Rate 73% 38% 70% 86% 100%
National Benchmark 88% 76% 85% 79% 85%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
December 2014 page 40
(n=600) graduating seniors attempted the EPP test. Results were calculated for (n=537) of those
students who completed 75% or more of the test items.
Results suggest that freshmen are entering the university with deficiencies in critical thinking and
writing. Focused efforts are needed to support student learning in these areas.
Results suggest that for seniors, more development is essential in the area of writing.
To address the deficiency in critical thinking, intervention efforts should be focused across the
curriculum and not solely in the general education sequence.
Incentivize student participation and support efforts to encourage competition among and within
Colleges/Schools.
Require all students in capstone courses to take the test for credit.
Recommend all programs to emphasize the skills measured by the test in the curriculum with the goal
of improving seniors’ performance at or beyond the national average.
NSSE
25% response rate for this year’s administration which was significantly higher than that of the last
administration of the instrument in 2011 (approximately 10% response rate).
FAMU First Year students and seniors perceived higher levels of engagement with faculty when
compared to other students from institutions in the SUS, Carnegie Classification and NSSE 2013 & 2014
taken as a group.
FAMU Seniors perceived higher levels of learning with peers as described by the engagement
indicator “Collaborative Learning” when compared to other students from institutions in the SUS,
Carnegie Classification and NSSE 2013 & 2014 taken as a group.
FAMU First Year students perceived lower levels of engagement in the areas of “Discussions with
Diverse Other”, “Effective Teaching Practices” and “Quality of Interactions” when compared to
other students from institutions in the SUS, Carnegie Classification and NSSE 2013 & 2014 taken as a
group.
FAMU Seniors perceived lower levels of engagement with respect to “Quality of Interactions” when
compared to other students from institutions in the SUS, Carnegie Classification and NSSE 2013 & 2014
taken as a group.
The majority of the respondent sample rated their overall experience at FAMU as excellent or good
(First year students – 70% and Senior – 82%).
Seventy nine percent of seniors as compared to 82% for the Carnegie Classification of students
stated that they would definitely or probably attend FAMU again.
Students benefit and are more satisfied in supportive settings that cultivate positive relationships
among students, faculty and staff.
Develop a Center for Teaching Excellence that would be charged with the responsibility of engaging
new and existing faculty in effective pedagogy.
Continue to recruit diverse faculty, staff and students to the university.
Recommended a POS system to capture in real time students’ perceptions of their experiences
interfacing with key administrative offices of the University.
December 2014 page 41
Assessment Report Submissions
Assessment reports were collected for (n=111) degree programs, (n=13) colleges/schools, and (n=53)
administrative departments.
Submission rates for 2013-2014 are lower than they have been in previous years.
Timely submissions tend to increase in years that the university is preparing for accreditation visits.
OUA will continue to push for 100% submission and reinforce the importance of assessment for
continuous improvement and institutional effectiveness.
ALCs
Based on assessment reports submitted and BOG criteria, a majority of undergraduate programs are in
continuous improvement mode.
OUA will continue to provide support to new programs in developing measurable outcomes and
gathering baseline data to use for continuous improvement.
OUA Workshops and Training Sessions
A total of 35 training sessions were conducted.
Overall, 99% of participants were satisfied with the quality of the training sessions.
Participants felt that the presentation and presenters were very professional and proficient in
Assessment.
Attendees expressed that the training sessions helped them to better understand the importance of
assessment and how the process can help document the ‘good works’ being done in their area.
OUA will continue to offer 1 hour sessions with the option to extend the length of training if it is
required.
December 2014 page 42
Office of University Assessment Team
Mark Howse, Ph.D.
Director
Office of University Assessment
Florida A&M University
Mark.howse@famu.edu
Phone (850) 412-5265
Franz H. Reneau, Ph.D. Assistant Director
Office of University Assessment
Florida A&M University
Franz.reneau@famu.edu
Phone (850) 412-5263
Brandi Newkirk, MBA
Coordinator Academic Support Services
Office of University Assessment
Florida A&M University
Brandi.newkirk@famu.edu
Phone (850) 412-5266
George L. Pinkney IV, M.Ed.
Coordinator Computer Applications
Office of University Assessment
Florida A&M University
George.pinkney@famu.edu
Phone (850) 412-5262
Angela Jordan
Administrative Assistant
Office of University Assessment
Florida A&M University
Angela.jordan@famu.edu
Phone (850) 412-5713
Recommended