SLO Assessment at UKPresentation to the College of Nursing

October 20, 2015

Office of University Assessment Tara Rose, Director

Brandon Combs, Assistant Director

Presentation OverviewO SACSCOC RequirementsO UK RequirementsO Implementing Assessment O RubricsO Integrating Assessment

SACSCOC Core Requirement

O The University will engage in ongoing, integrated, and institution-wide evidence-based planning and evaluation processes that:

O incorporate a systematic review of institutional mission, goals, and outcomes;

O result in continuing improvement in institutional quality; and

O demonstrate the institution is effectively accomplishing its mission.

Note: If an institution fails to document compliance with Core Requirements at the time of reaffirmation or at the time of any review, the Commission will place the institution on sanction or take adverse action.

SACSCOC Core Standard

O 3.3.1 The institution identifies expected outcomes, assesses the extent to which it achieves these outcomes, and provides evidence of improvement based on analysis of the results in each of the following areas: (Institutional Effectiveness)

O 3.3.1.1. educational programs, to include student learning outcomes

O 3.3.1.2. administrative support servicesO 3.3.1.3. academic and student support servicesO 3.3.1.4. research within its mission, if appropriateO 3.3.1.5. community/public service within its mission, if

appropriate

SACSCOC Guidelines for Compliance

O Institutions are expected to use multiple assessments in each area.

O Grades alone for the assessment of educational programs or student learning outcomes are insufficient.

O An institution will be able to demonstrate institutional effectiveness for all its diplomas, certificates, and undergraduate and graduate educational degree programs.

O The institution is responsible for producing mature data. Mature data can be defined as sufficient information used as a basis for sound decision making.

O The institution is responsible for providing evidence of improvement based on the analysis of the assessment results, as opposed to a plan for improvement.

SACSCOC Guidelines for Compliance

Educational Programs O the institution will engage in on-going planning and assessment to ensure that

for each academic program, the institution develops and assesses student learning outcomes.

O program and learning outcomes specify the knowledge, skills, values, and attitudes students are expected to attain in courses or in a program.

O methods for assessing the extent to which students achieve these outcomes are appropriate to the nature of the discipline, and consistent over time to enable the institution to evaluate cohorts of students who complete courses or a program.

O shared widely within and across programs, the results of this assessment can affirm the institution’s success at achieving its mission and can be used to inform decisions about curricular and programmatic revisions.

O at appropriate intervals, program and learning outcomes and assessment methods are evaluated and revised.

UK Requirements Administrative Regulation 1:4

O Planning, Assessment, and Budgeting Cycle (Cycle)O The “Cycle” is the ongoing and institution-wide planning and evaluation process through which

the University monitors the effectiveness of its units, programs, and services to inform decision making in the areas of resource allocation and budgeting to support the University’s mission.

O PlanningO University Strategic PlanningO Unit Strategic Planning

O AssessmentO Periodic ReviewsO Progress ReportingO SLO Reporting: Student Learning Outcomes Report is produced annually to document the

continuous improvement of student learning and are required by all educational programs. This report shall focus on student learning outcomes; however, units may also include assessments of program curricula and teaching effectiveness.

O CAO ReviewsO Budgeting

O The budgeting process is aligned with the University’s planning and assessment activities to allocate fiscal resources, foster accountability, and translate plans into action.

UK Requirements Office of University Assessment

O Assessment PlansO To be aligned with program’s Periodic ReviewO Include criteria outlined in the Assessment Plan

Template O Reviewed by UAC Members or the OUA using

approved Rubric O Annual SLO Reporting

O Submitted by October 31 each yearO SLO Assessment should align with assessment plan O All outcomes must be assessed within a 3-year cycleO Reviewed by UAC Members or the OUA using

approved Rubric

SLO Assessment PLANO Criteria

O Mission Statement O Program-level Student Learning Outcomes O Curriculum Map/Artifact Map O Assessment ToolsO Assessment Oversight O Assessment Cycle/Data Collection O Appendices O Teacher Effectiveness (Educational Programs Only)

O Post Graduate Success

SLO Assessment REPORT

O Criteria O Student Learning Outcome O Relationship between assessment tools and

outcomes O Data collection and research integrity O Specification of desired benchmark/target/goal O Results O Interpretation of results O Improvement actions O Reflection on previous years report

Cycle Academic Year Report Due

SACSCOC Reporting

UK SLO Reporting Cycle: 2009-2012 *data used for SACSCOC Compliance Report

Year 1 2009-2010 February 2011 Year 2 2010-2011 October 2011 Year 3 2011-2012 May 2012 Compliance Report Submitted 2013

UK SLO Reporting Cycle: 2012-2015 Year 1 2012-2013 October 31, 2013 Year 2 2013-2014 October 31, 2014 Year 3 (current cycle)

2014-2015 October 31, 2015 Planning Begins

UK SLO Reporting Cycle: 2015-2018 *data used for SACSCOC 5th year Interim Report

Year 1 2015-2016 October 31, 2016 Year 2 2016-2017 October 31, 2017 Year 3 2017-2018 October 31, 2018 NO extensions. UAC review to occur 2nd week in

November. Results incorporated into SACSCOC report by December 2018. ***Year 3 reporting cycle may change pending President’s 5th year interim report review timeline.

5th Year Interim Report DUE March 2019

SLO Assessment Reporting Cycles at UK

Implementing your Assessment Plan

Mechanics of Assessment O Plan

O Mission O purpose

O Outcomes O what you intend to achieve

O MethodsO how you plan to get there

O Measure O how well are you doing

O ActO Analyze your results O Plan changesO Take actionO Assess impact

Rubrics

Improvement

MEASURE O How, Who, What, When, Where

O Apply most, if not all of these questions to the following:O MapsO ToolsO Collection of DataO Preliminary AnalysisO Dissemination of results and formulation or an

improvement action plan

Rubrics are…O An objective method of gathering quantitative

data on complex performances of learningO A clear articulation of how student performance

is linked to specific course & program outcomesO A means of decreasing time spent on grading,

while increasing consistency in grading across sections, courses, programs, colleges

O A method for translating qualitative information into quantitative data

Rubric vs Scoring GuideO Scoring guides generally provide a scale, but

not performance indicatorsO Unacceptably low inter-rater reliability

Criteria Score 1-5

Context and purpose for writing

Content development

Genre and disciplinary conventions

Sources and evidenc e

Control of syntax and mechanics

Rating Scale 4 3 2 1

1. Demonstrates ability to perform basic skills previously learned.

Consistently demonstrates ability to perform skills previously learned with minimal or no instructor assistance or cueing.

Demonstrates ability to perform skills previously learned. Requires moderate instructor assistance and cueing.

Inconsistently demonstrates ability to perform skills previously learned. Requires maximum instructor assistance and cueing.

Inconsistently demonstrates ability to perform skills previously learned. Has demonstrated unsafe patient care.

2. Demonstrates knowledge of principles of sterile technique and ability to maintain sterility. (E.g. Injections, dressing changes, etc.)

Consistently demonstrates knowledge of sterile technique. Able to verbalize and perform technique with minimal or no instructor cueing or feedback.

Demonstrates knowledge of sterile technique. Able to verbalize and perform technique with moderate instructor cueing or feedback.

Inconsistently demonstrates knowledge of sterile technique. Not able to verbalize or perform technique, requires maximum instructor feedback and cueing

Inconsistently demonstrates knowledge of sterile technique. Has no recollection of technique or procedure. Has demonstrated unsafe patient care, below level of safety.

3. Applies principles of safety.(E.g. S.R., personal safety, check ID band

Consistently demonstrates knowledge of safety. Keeps the patients safety in the forefront of patient care with minimal or no instructor reminders or cueing.

Demonstrates knowledge of safety. Keeps the patients safety in the forefront of patient care with moderate instructor reminders or cueing.

Inconsistently demonstrates knowledge of patient safety. Requires maximum instructor reminders and cueing about patient safety.

Inconsistently demonstrates knowledge of patient safety. Has no recollection of patient safety issues. Has demonstrated unsafe patient care, below level of safety.

4. Utilizes professional therapeutic communication techniques with patients, staff, family, instructor, and other students.

Consistently demonstrates knowledge of therapeutic communication with patients, staff, etc. with minimal or no instructor cueing.

Demonstrates knowledge of therapeutic communication with patients, staff, etc. moderate instructor cueing.

Inconsistently demonstrates knowledge of therapeutic communication. Requires maximum instructor involvement and cueing.

Inconsistently demonstrates knowledge of therapeutic communication. Has demonstrated unsafe patient care, below level of safety.

5. Demonstrates a sense of responsibility in initiating and completing patient care needs.

Consistently demonstrates knowledge & responsibility in initiating and completing patient cares. Requires a minimal or no amount of instructor intervention and

Demonstrates knowledge & responsibility in initiating and completing patient cares. Requires a moderate amount of instructor intervention and cueing.

Inconsistently demonstrates knowledge & responsibility in initiating and completing patient cares. Requires maximum instructor intervention and cueing.

Inconsistently demonstrates & responsibility in initiating and completing patient cares. Consistently demonstrates unsafe and dangerous patient care.

Clinical Rubric for Nursing

Holistic vs Analytic RubricsO Analytic rubric

O Each criterion is scored independentlyO Most commonly used for evaluating course

work

O Holistic rubricO Each student performance or work is

assigned a single scoreO Most commonly used for evaluating program

outcomes

Program-Level vs. Course-Level

Program Course • Broad• Focused on the

overall achievement of an outcome

• Can be used across the curriculum with a variety of assignments

• Specific • Focused on the

achievement of an outcome’s sub-component

• Includes assignment-specific criteria

Hybrid RubricO “Hybrid” rubric: both analytic and holistic

elementsO Specific O Broad

O A way to assess course-level and program-level at the same timeO Add department approved program-level

criteria at the bottom of the rubric

ACT O “Action” implies that something is happening.

O Are you taking action based on evidence?O Does your action align to your results and

analysis?O Will you change your curriculum or services?

O If so, how?O If not, what other improvements will you make?

O If you are happy with your results then you must:O Investigate your benchmarksO Explain why you choose to only maintain

What Does “Improvement” Mean?

O Better student learning experiencesO Increased student learningO Increased program, department, college,

institutional efficiencyO Increased program, department, college,

institutional effectiveness

Integrate assessment and improvement processes

BUDGETING

Questions?

Tara Rose, Director tara.rose@uky.edu

Brandon Combs, Assistant Director bjcomb2@uky.edu

ReferencesO SACSCOC Core Requirement 2.5 and

Comprehensive Standard 3.3.1, The Principles of Accreditation: Foundations for Quality Enhancement, 2012