O C T O B E R 2 0 1 1 V O L U M E 3 , I S S U E 5

S P E C I A L

P O I N T S O F

I N T E R E S T :

Blackboard

Outcomes Users

Appreciated for

Their Efforts

Assessment In

and Around UK

UK CLA Results:

A Longitudinal

Study of the fall

2007 Cohort

UAC Member

Profiles

SIZZLE What’s Hot in Assessment

On September 15, the Office of Institutional Re-search, Planning, and Effectiveness (IRPE) hosted a luncheon for Blackboard (Bb) Out-comes users. Over 200 Bb Outcomes users were invited. The individual efforts represented the 18 colleges, 200 educational and administra-tive units, and more than 250 academic pro-grams. This luncheon was to acknowledge those who have worked diligently within their colleges and divisions to learn the new Bb Out-comes system. Users entered reporting informa-tion on assessment activities, which include stu-dent learning and/or strategic planning. During the luncheon, attendees were addressed by President Eli Capilouto and Interim Vice Presi-dent for IRPE, Heidi Anderson. President Capilouto spoke about the importance of strategic planning and assessment. In his speech, he noted the value of a quality technol-ogy system to help with these processes. He mentioned his experience using a cumbersome system in preparation for SACS reaffirmation in his previous position as Provost at the University of Alabama Birmingham, and compared that to the system currently employed at UK. The Uni-versity of Kentucky uses Bb Outcomes as its primary tool for capturing assessment and strate-gic planning information for use in planning and documentation for SACS. The Outcomes sys-tem was rolled out to faculty and staff in the fall of 2010 for use across campus. President Capilouto acknowledged that there have likely been some growing pains for some who have been tasked to learn the new technology, but he admired their tenacity to take on this charge and to do it well. Bb Outcomes is a technology for the assessment of student learning that was purchased as an add-on to the current Bb Learn system that UK uses for course management. The Bb Out-

Blackboard Outcomes Users Appreciated for Their Efforts By Leah Simpson, Assessment Specialist

comes technology aided the University in solving a variety of assessment and institutional effective-ness problems. UK wanted a technology that could: 1) house strategic plans and annual up-dates; 2) store all assessment of student learning results and Improvement Action Plans; and 3) facili-tate the evaluation of student artifacts without hav-ing to use paper. In addition, Outcomes has the ability to create program curriculum maps, create and utilize rubrics, and send surveys. The University of Kentucky is not the first university to utilize Blackboard Outcomes for assessment purposes, but it is the first university to make use of a variety of tools offered in the system. Most uni-versities and colleges that have adopted Outcomes have limited their use of the system to only one tool. The University of Kentucky has chosen to make use of a variety of tools offered by the sys-tem. To date, the system has been used to:

build strategic plans complete annual strategic plan updates

(Continued on page 2)

President Capilouto with the College of Social Work’s Director of Un-dergraduate Studies, Dr. Karen Badger during the Bb Outcomes Luncheon. .

P A G E 2 V O L U M E 3 , I S S U E 5

Blackboard Outcomes Users Appreciated for Their Efforts

The Bb Outcomes Resources web page offers a variety of tutorials.

Rubrics can be used to evaluate student work outside of the classroom environment for program assessment purposes.

and improvement action plans for student learning assess-ment

conduct course evaluations send surveys house curriculum maps house all program student learning outcomes house all college and division metrics and standards align program outcomes to rubrics and courses use rubrics for the evaluation of student artifacts separate

from the classroom grading create artifact templates for assessment purposes complete separate assessment process necessary for pro-

gram-specific accreditation

To better assist Bb Outcomes users to learn about the various ways that the system might be useful to them, the Office of Assessment has created an Outcomes Resources web page that provides a variety of tutorials, training and workshop dates, as well as other useful informa-tion about Outcomes. The site can be accessed directly at www4.uky.edu/outcomes_resources. Outcomes users are asked to check the site regularly for updates on any training events and to find out how others are using Outcomes to aid their assessment process on campus.

(Continued from page 1)

Blackboard Outcomes Training Schedule - Fall 2011

DATE TOPIC

October 17, Monday Associating Courses in Blackboard Outcomes

October 17 – 28 Consultation Sessions with individual colleges on Student Learning Assessment and Strategic Planning

October 19, Wednesday Curriculum Mapping in Bb Outcomes-Part 1: Building

November 7, Monday Creating Rubrics using Bb Outcomes for Programmatic Assessment

November 9, Wednesday Evaluating Student Artifacts using Bb Outcomes

NEW

NEW

P A G E 3

Assessment In and Around UK

Using Blackboard Outcomes for Student Artifact Assessment

McClure, N., Rose, T., & Simpson, L. (July, 2011). Using Blackboard Out-comes for Student Artifact Assessment. The Blackboard World 2011 Confer-ence in Las Vegas, Nevada. The session provided an overview of the University of Kentucky’s process from identifying Student Learning Outcomes, artifact collection, and scoring artifacts at an Institutional level, as well as working with evaluators from around the university. Pitfalls in both technology and business process that we had to overcome during our pilot were also discussed.

Using Blackboard Outcomes for Student Artifact

Assessment

Nick J. McClure

Tara A. Rose

Leah P. Simpson

University of Kentucky

Using Rubrics in the Information Literacy Assessment Process: A Conversation

Sharp, D., & Wiza, J. (June, 2011). Using Rubrics in the Information Liter-acy Assessment Process: A Conversation. Association for the Assessment of Learning in Higher Education (AALHE) Conference in Lexington, KY. Although the UK libraries have been assessing information literacy learning outcomes for some time, it is only recently that rubrics have been applied to this assessment process. The conversation explored the uses and limita-tions of rubrics for assessing information literacy. Discussions included the VALUE rubric and lessons learned.

By Debbie Sharp and Judy Wiza, UK LIbraries

By Nick McClure, UKIT, Tara Rose, and Leah Simpson, Office of Assessment

The Office of Assessment is asking for article contributions to Assessment In and Around UK. In this section, we highlight college and department assessment activities. We focus on the variety and strength of assessment methods, strategies, plans implemented, and other assessment-related infor-mation of interest to your colleagues in UK’s colleges and departments. If you would like to contribute an article highlighting the assessment activities in your college or depart-ment in future issues, please your submission to the Office of Assessment or to tara.rose@uky.edu. Deadline for the November 2011 issue is on October 31, 2011. Thank you.

P A G E 4

The Colligate Learning Assessment (CLA) is a nationally–normed, authentic assessment developed by the Council for Aid to Education (CAE) to measure institutional achievements in student learning in four key general education core skills: critical thinking, analytic reasoning, problem solving, and written communication. CLA re-sults are controlled for incoming academic ability. In an effort to assess UK’s general education learning, and as a requirement by the Voluntary System of Ac-countability, the Office of Assessment randomly sampled UK students to participate in a longitudinal study. A longitudinal design of the CLA required that it be administered to the same cohort three times: first in fall 2007, next in spring 2009, and finally, in spring 2011. The summary results are provided in the tables below.

S I Z Z L E

UK CLA Results: A Longitudinal Study of the Fall 2007 Cohort

P A G E 5 V O L U M E 3 , I S S U E 5

The CLA Longitudinal Study was carried out in three phases in various institutions. In phase 1 (fall 2007), a total of 11, 437 freshman students from 48 schools participated in the assessment. In phase 2 (spring 2009), 3,327 students from 32 schools took the assess-ment. Then, in the final phase (spring 2011), 2, the CLA was administered to 2,374 students from 31 schools. The CLA does not control for students effort. Nevertheless, researchers looked into the amount of time spent on the assessment and the students’ scores. They found that a positive correlation exists between CLA scores and the time students spent on tasks. Most students need 90 minutes on a performance task and 75 minutes on an analytic writing task. To complete a CLA full battery of tasks (performance task and analytic writ-ing task), students are given a total time of 165 minutes.

(Continued from page 4)

UK CLA Results: A Longitudinal Study of the Fall 2007 Cohort

No. of Institutions: 48 Total No. of Students: 11,437 No. of UK Students: 308 Lowest Score: 759 (63 minutes spent) Highest Score: 1375 (153 minutes spent)

No. of Institutions: 32 Total No. of Students: 3,327 No. of UK Students: 146 Lowest Score: 828 (63 minutes spent) Highest Score: 1461 (121 minutes spent)

No. of Institutions: 31 Total No. of Students: 2,374 No. of UK Students: 110 Lowest Score: 897 (19 minutes spent) Highest Score: 1489 (70 minutes spent)

PHASE 1: Fall 2007 PHASE 2: Spring 2009 PHASE 3: Spring 2011

Percentage of Students Percentage of Students Percentage of Students

The CLA results of the first cohort have been shared with the University Assessment Council (UAC) members and the Office of Under-graduate Education. Together with a faculty committee, the Office of Undergraduate Education will review the results and create an im-provement action plan. A new cohort of freshman students has been assessed this fall 2011 for the second longitudinal study that ends in spring 2015. The students in this cohort were not given the full battery of tests. Instead, CAE had decided to phase to a 90-minute version of the assessment, wherein students will be given either a performance task or an analytic writing task. Results of the fall 2011 assessment are expected in spring 2012.

P A G E 6 V O L U M E 3 , I S S U E 5

S I Z Z L E

Note: Total enrollment includes bachelor’s, master’s, specialists, PhDs in research and professional practice (degree and non-degree), post-doctorates, house staff, certificates, undergraduate (non-degree), and graduate (non-degree). Source: University of Kentucky Institutional Research, Planning, and Effectiveness, http://www.uky.edu/IRPE/students/enrollment.html

University Assessment Council (UAC) UAC Member Profiles

Sandra D. Challman has worked in the College of Dentistry since 1997 initially as the Manager of In-structional Technology and most recently as Director of Curriculum Development. Her current responsi-bility in the College involves the areas of curricular management, program assessment, accreditation, on-line course evaluation, faculty development in IT, and leadership in the use of technology for teach-ing and learning. Sandy’s background includes instructional design and development, information man-agement and informatics. She has an M.S. in Library and Information Science from the University of Kentucky. Working nearly 14 years in the College of Dentistry was preceded by 7 years at the Human Develop-ment Institute and additional years in the Colleges of Health Science and Medicine. Special projects and special people along the way have kept Sandy’s skills sharp and her motivation to contribute high. Favorite projects that come to mind are: learning and using new technologies including Second Life, lecture capture, library databases and web design; attending national conferences such as Educause and American Dental Education Association (ADEA), participating in grants; and collaborating on com-mittees. Sandy was born and raised in Danbury, Connecticut , but has enjoyed her Kentucky home since gradu-ate school. Nothing is more important to her than family. Her husband, Don, is also a career UK em-ployee. They have two adult children-- Lauren, a graduate of Indiana University, now lives and works in Manhattan, and Greg, is an undergraduate here at UK. Sandy is grateful for all the opportunities the University of Kentucky has presented her and her family.

Amie Ellis joined the University of Kentucky in April 2011 as Director of Assessment of the Gatton College of Business and Economics. Prior to UK, Amie was a full time lecturer at Clemson University in the College of Business and Behavioral Sciences. At Clemson, Amie taught Operations Management, Advanced Statistical Quality Control, Lean Operations, and Business Statistics. During this time, Amie was a member of a faculty college committee charged with establishing an AACSB compliant learning of assurance process for the De-partment of Management. Preceding Amie’s career in academia, she worked in the automo-tive industry for General Motors Corporation and Delphi Corporation as an Industrial Engi-neer for nearly 20 years. In her manufacturing leadership positions, Amie facilitated lean transformation workshops, optimized and scheduled injection mold operations, led imple-mentation of error-proofing systems, coordinated divisional metric reports for executive re-view, chaired the divisional ergonomics team, and worked to select and train new employ-ees. As a lean facilitator, Amie worked with multiple manufacturing processes to achieve multi-million dollar savings through productivity improvement, inventory reduction, improved yield, and reduced ergonomic risk. More re-cently, Amie worked as a Senior Engineer for Group Schneider in Seneca, SC, where she led a strategic lean transformation project for a high volume, high variation motor control center assembly. Since joining Gatton, Amie’s efforts have been focused on the preparation of the College’s Fifth Year Maintenance of Accreditation Report that is required by AACSB International, the accrediting body for business schools. The report serves to provide a variety of information about the College, including perceived strengths and weaknesses, challenges, strategic management, financial strategies, and learning of assurance. The College will be hosting both Business and Accounting Peer Review Teams on campus in November. Amie earned her M.S. in Manufacturing Management from Kettering University (formerly GMI Engineering & Management Institute) and her B.S. in Industrial Engineering from the State University of New York at Buffalo. Most recently, Amie earned the Supply Chain Opera-tions Reference (SCOR) Certification from the Supply-Chain Council. She currently resides in Lexington, KY, with her husband and two daughters.

SACS Reaffirmation of Accreditation Timeline

September 2012 Compliance Certification Due: A written review and analysis of

UK’s compliance to the Core Requirements and Compre-hensive Standards (replaces the Self Study).

November 2012

Off-Site Peer Review Conducted: A SACS-appointed team meets off-campus to review UK’s Compliance Certification and support documentation. This team will make a prelimi-nary determination regarding compliance with the Core Requirements and Comprehensive Standards. A focused report is written.

January, 2013

Quality Enhancement Plan Due: A clear and succinct review of the Quality Enhancement Plan is delivered to the Commis-sion on Colleges. This document includes a review of the process used to develop the QEP, the determination of the topic, the desired student learning outcomes, a literature

review and best practices, the actions to be implemented, a timeline, the QEP organizational structure, a listing of re-sources, and an assessment plan.

March – April 2013

Onsite Peer Review Conducted: A SACS-appointed team will visit UK to make the final decision on the university’s compli-ance with the Core Requirements and Comprehensive Stan-dards. They will also determine the acceptability of UK’s Quality Enhancement Plan. This group will provide a report to UK with analysis and advice concerning UK’s reaffirma-tion.

May 2013

Receipt of Affirmation Report from SACS

December 2013 Reaffirmation announcement at the SACS Annual Meeting

Academic Year Blackboard Outcomes Entry Required by:

2010-2011 October 31, 2011

2011-2012 May 25, 2012*

2012-2013 October 31, 2013

Institutional Effectiveness Due Dates: Annual Progress Reports and Annual Assessment Improvement Action Plans

*Due to the University’s reaffirmation timeline for SACS the due date for AY 2011-2012 is earlier than previous years. This date cannot be extended.

P A G E 7

DATE TIME LOCATION

October 14 12:00 - 1:30PM 359 Student Center

November 11 12:00 - 1:30PM 363 Student Center

December 9 12:00 - 1:30PM 359 Student Center

UAC Meeting Schedule - Fall 2011

Mission

The mission of the Office of Assessment is to provide univer-sity-wide support for assessment of student learning, planning, and continuous improvement activities at course, program, and institutional levels, and to develop and sustain across the uni-versity community a culture of assessment.

Values

Continuous improvement of student learning

Shared responsibility, shared resources, and collaboration

Use of technology to achieve efficiency

Responsiveness and communication

Rigorous, authentic, and useful assessment data

Assessment Workshops in 2011

www.uky.edu/IRPE/assessment.html

For more information, please contact us at: Office of Assessment UK Federal Credit Union Building 1080 Export Street Suite 280, 2nd Floor Lexington, KY 40504 Phone 859.257.6394 Fax 859.323.3999

Or email us: Tara Rose, Director of Assessment tara.rose@uky.edu Leah Simpson, Assessment Specialist I leah.simpson@uky.edu Natasha Mamaril, Graduate Assistant tashmamaril@uky.edu Jill Priesmeyer, Graduate Assistant jill.priesmeyer@uky.edu Daniel Cleland, Graduate Assistant dan.cleland@uky.edu

Assessment workshops in 2011 will be scheduled as requested. The Office of Assessment plans to conduct workshops and presentations at the college and/or departmental level. For workshop requests, please contact Leah Simpson, Assessment Specialist at leah.simpson@uky.edu or fill out a request form at our web-site: http://www.uky.edu/IRPE/assessment/assessmentrequest.html