Upload
stefan
View
29
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Assessment 101 An Introduction for Administrative and Support Units. Presented by Horace D. (“H.D.”) Stearman, Ph.D. Director of Institutional Effectiveness August 14, 2008. Module One: Overview. What Is Assessment? Why Do We Do Assessment? What The Accreditors Say - PowerPoint PPT Presentation
Citation preview
Assessment 101
An Introduction for Administrative and Support Units
Presented by Horace D. (“H.D.”) Stearman, Ph.D.
Director of Institutional Effectiveness
August 14, 2008
2
Module One: Overview
What Is Assessment?
Why Do We Do Assessment?
What The Accreditors Say
Assessment Misconceptions
The Assessment Cycle Step 1: Mission Statement and Core Functions/Services
Step 2: Expected Outcomes
Step 3: Measures
Step 4: Target Levels
Step 5: Findings
Step 6: Action Plans
Potential Assessment Pitfalls
Relation of Assessment to Planning
3
What Is Assessment?
Assessment is a systematic, ongoing process of collecting, reviewing, and using data about an institution’s academic, administrative, and support units to improve the effectiveness of these units.
We carry out assessment of administrative and support units by Creating a mission statement for our unit that briefly describes our purpose;
Identifying the core functions and services we provide that enable us to meet our mission;
Determining expected outcomes to describe the results we expect to see in the way we perform our core functions and services;
Selecting measures that will quantify the achievement of our expected outcomes;
Setting target levels of performance on the expected outcomes;
Interpreting the levels of performance that we attain on our expected outcomes to judge how well our actual performance matches our target levels; and
Using the results to document, explain, and improve the effectiveness of our unit in accomplishing our mission.
4
Why Do We Do Assessment?
To improve what we do To inform decision-making, budgeting, and planning To focus our efforts To show students, alumni, employers, and others that we’re doing a good job To satisfy the expectations of our accreditors
5
What The Accreditors Say
The institution engages in ongoing planning to achieve its missions and goals. It also evaluates how well, and in what ways, it is accomplishing its mission and goals and uses the results for broad-based, continuous planning and evaluation. Through its planning process, the institution asks questions, seeks answers, analyzes itself, and revises its goals, policies, procedures, and resource allocation.
Northwest Commission on Colleges and Universities
“Standard 1.B – Planning and Effectiveness”
6
Assessment Misconceptions
It’s A Passing Fad
It’s Not My Job
You Can’t Assess What I Do
I Don’t Have Time For Assessment
I Don’t Have The Expertise To Do Assessment
Assessment Might Reveal Shortcomings Or Failures
7
The Assessment Cycle
Outcomes
Plans for Improvement Assessment of Outcomes
8
Step 1: Mission Statement and Core Functions/Services Write a brief mission statement that describes your unit’s purpose. Identify the four to seven core functions or services that enable your unit to meet its
mission. Taken together, the mission statement and the list of core functions or services
answer the questions “What do we do and how do we support the RMUoHP mission?”
9
Some examples of mission statements and core functions
The Department of Career Services counsels students and alumni as they explore career directions and equips them with the information they need to make good decisions.
Our core functions include1. Provide resume-writing workshops, mock interviews, and related training to students as
they prepare for job searches
2. Coordinate on-campus visits by corporate recruiters and on-campus job fairs
3. Maintain a database of job postings and a library of employment-related books and DVDs for student use
4. Counsel students and alumni in best practices for job searches
The Office of the Registrar maintains and processes student academic
records accurately and efficiently.
Our core functions include1. Maintain student satisfaction with registration functions
2. Process degree audits for graduating students
3. Fill transcript requests
10
Step 2: Expected Outcomes
Write three to five expected outcomes that describe the results you expect to see in the way you perform your core functions and/or deliver your key services over the coming year.
Expected Outcomes should be related to something under the control of your unit
worded in terms of what your unit will accomplish or what your clients will think, know, or be able to do following the provision of services (“X will happen”)
measurable, meaningful, manageable
11
Some examples of expected outcomes
The Office of Institutional Research will collect and disseminate campus data for internal and external reporting purposes.
The Office of the Registrar will provide resources to enable students to use on-line registration successfully.
The Office of Career Services will engage students in development early and often through various outreach efforts.
The Office of Information Technology Support will provide its customers with prompt and effective assistance in resolving their technical problems related to systems, networks, and desktop applications.
12
Step 3: Measures
Select at least one method of measurement—preferably more—for each expected outcome that you created in Step 2.
Measures should be observable and, in most cases, yield quantitative data.
Types of measures include
counts/rates/percentages
questionnaires and surveys
minutes of meetings and focus groups
deadlines met
13
Step 4: Target Levels
For each of the measures selected in Step 3, set a specific goal for the “score” that you hope to achieve.
Other terms that are often used for target levels are indicators or criteria.
The target level that you set for each measure should appear reasonably attainable
take into account your past performance in the function or service being assessed
provide the potential for showing improvement
14
Some examples of measures and target levels
Average attendance at library orientations will be 80% of all first-year students.
90% of fifty randomly selected financial aid files will be complete and accurate when audited.
Average scores on the Student Satisfaction Survey will be 4.2 on a five-point scale.
100% of scholarship applications will be processed within sixty days of receipt.
By December 2008, faculty using TechLink will be surveyed to determine what they see as the three most critical areas of need in upgrading the service.
15
Step 5: Findings
Briefly summarize the findings of your assessment by including for each expected outcome the target level that you set
the actual level that you attained
a brief analysis of why you did or did not meet the target level
16
Some examples of findings
In the annual survey of faculty/staff satisfaction, 36% of our customers expressed frustration with the wait time for help desk requests. Our target level was no more than 15% for this measure. Analysis of survey data showed that 85% of those respondents expressing frustration with the wait time for help desk requests were located in the Office of Financial Affairs.
Average routing time for personnel action forms (PAF) was decreased by 21%. Our target level was 30% for this measure. Analysis of the routing pathways for a random sample of thirty PAFs showed that forms for administrative personnel required seven more hours in transit than those for support staff because of several additional sign-offs being necessary.
17
Step 6: Action Plans
Your action plan should be a brief description of what steps you will take to address any expected outcomes where you did not meet your target levels.
You also may wish to review the outcomes in which you met your target levels to see if other improvements or activities are needed in the future—especially if you anticipate conditions changing so as to pose a threat.
Action plans should be included as expected outcomes in your next assessment cycle.
Unless you “close the loop” by taking actions based on your findings, you’re not really doing assessment!
18
Some examples of action plans
To address the 15% decline in annual giving by alumni, a personal contact, mail, and email campaign emphasizing to new graduates the benefits of annual giving to the institution will be developed and implemented in July 2009.
The 20% increase in wait time for level-one response will be addressed by the hiring of four new employees for the help desk during fiscal 2009.
The 17% increase in the number of books not returned to the library on time will be addressed by increasing the fines for overdue books to $3.00 per day starting in fall semester 2009.
19
Potential Assessment Pitfalls
Taking measurements that are not related to expected outcomes Waiting until the plan is perfect before beginning Collecting results for so many measures that results are overwhelming Spending more time on measuring things than on doing things
20
Relationship of Assessment to Planning
21
Module Two: Overview
Tour of Assessment Folders on R Drive Template for Creating Assessment Plans Sample Assessment Plan Practice Exercises Summary Assignments: Complete by September 5
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
Template for Creating Assessment Plans
37
Template for Creating Assessment Plans
38
Template for Creating Assessment Plans
39
Sample Assessment Plan
40
Sample Assessment Plan
41
Practice Exercises: Generate Mission Statement and Core Functions or Services Divide into groups of 3 Choose 1 group member (#1) for whose unit you will write a mission statement and
four core functions or services Member #2 will ask questions, #3 will take notes on responses 5 minutes: questions and notes 5 minutes: all three members write mission statement and four core functions or
services 5 minutes: discussion of selected mission statements and core functions/services by
all workshop members
42
Practice Exercises: Generate Expected Outcomes Remaining in the same groups as in the previous exercise and retaining your focus
on the same unit for which you created a mission statement and core functions/services, use the same questioning and note-taking process again to generate three expected outcomes for the unit
5 minutes: questions and notes 5 minutes: all three members write three expected outcomes 5 minutes: discussion of selected expected outcomes by all workshop members
43
Practice Exercises: Generate Measures and Target Levels Remaining in the same groups as in the previous exercise and retaining your focus
on the same unit for which you created three expected outcomes, use the same questioning and note-taking process again to generate measures and target levels for each of the three expected outcomes
5 minutes: questions and notes 5 minutes: all three members write measures and target levels for the three expected
outcomes 5 minutes: discussion of selected measures and target levels by all workshop
members
44
Summary
Six Steps of the Assessment Cycle Mission Statement and Core Functions or Services Expected Outcomes Measures Target Levels Findings Action Plans
Why We Do Assessment To improve what we do To inform decision-making, budgeting, and planning To focus our efforts To show students, alumni, employers, and others that we’re doing a good job To satisfy the expectations of our accreditors
45
Assignments: Complete by September 5
Download a copy of the assessment template for administrative and support units from the R:\Assessment Folder
Write a first draft of your unit’s 2008-2009 assessment plan that includes Mission statement
Four core functions or services
Three to five expected outcomes
Measures and target levels for each expected outcome
Set up an appointment with the DIE to review your draft assessment plan during the period September 8 – September 19
46
Assignments: Complete by September 5
Assessment Plans Should be Created by These Individuals and Their Support Staff1.Academic Dean
2.Director of Institutional Effectiveness
3.Director of Institutional Marketing
4.Director of Research
5.Executive Vice President
6.Human Resources Manager
7.IRB Manager
8.Medical Librarian
9.Marketing Manager
10.President
11.Registrar
12.Vice President of Academic Affairs
13.Vice President of Finance
14.Vice President of Student Services