Upload
arlo
View
33
Download
0
Tags:
Embed Size (px)
DESCRIPTION
October 4, 2012. Administrative Unit Assessment: Different Purposes, Different Measures. Elaine W. Kuo , College Researcher, Foothill College Mallory Newell, College Researcher, De Anza College Robert Pacheco, Dean, MiraCosta College Gail Shirley, Planning Coordinator, MiraCosta College. - PowerPoint PPT Presentation
Citation preview
ADMINISTRATIVE UNIT ASSESSMENT:
DIFFERENT PURPOSES, DIFFERENT MEASURES
October 4, 2012
Elaine W. Kuo, College Researcher, Foothill CollegeMallory Newell, College Researcher, De Anza CollegeRobert Pacheco, Dean, MiraCosta CollegeGail Shirley, Planning Coordinator, MiraCosta College
2
Resources
Documents and Resource Linksare available at:
http://ssscauosession.wikispaces.com/
3
Today’s TopicsAdministrative Unit Assessment
Administrative Unit Objectives/Outcomes
Assessment Measures/Methods Program Review and AUOs
Best Practices Examples, Models
Designing AUOs
4
Group Dialogue – 5 Minutes Talk to someone next to you
Discuss where your colleges are in the AUO process
Share with all attendees
5
What is an Administrative Unit?
Foothill and De Anza CollegesA unit, department, program or office that
provides services to any individuals in order to directly or indirectly maximize student success.
MiraCosta CollegeA department/program providing operational services and organizational support to assist the district in advancing its Mission Statement.
6
What are Outcomes?The results, consequences, or impact of a course
of action
What are Outcomes Statements?Specific, focused, measurable written statements that
○ identify intended results, ○ specify activities undertaken, and ○ may specify target groups if applicable
What does an Outcomes Statement look like?
In its simplest form:○ [intended result] through [actions] for [target group, if
applicable]
7
AUOs?
AUOs?
AUOs?
We know about Student Learning Outcomes, but . . .
8
ACCJC and AUOs …..
ACCJC does not define what an AUO statement is, should look like, or how to assess it, yet….they are still recommending that colleges define and assess their AUOs:
“To meet the standard at the level of proficiency by 2012, the team recommends that the college accelerate the implementation of the SLO, SSLO and AUO assessment cycle at the course, program and institutional levels”
- ACCJC Recommendation, De Anza College
9
10
What is an AUO? Generally speaking . . .An Administrative Unit Outcome Statement is a
concrete, measurable statement expressing the expected achievement level (target) of a specific action.
Using the identified Assessment Method, actual achievement will be determined by comparing the resulting data to the expected achievement level (target).
Assessment results inform Program Review, during which the Administrative Unit:Analyzes the dataReviews and ReflectsDevelops a plan(s) in response to the data
11
What is the purpose of AUOs?
Create a tie to the college mission, core values, institutional goals, major planning documents, and/or division/department objectives
Specify what the unit intends to accomplish
Produce assessment data to determine if performance matches learning or service/operational expectations
Inform Program Review
12
How are colleges applying AUOs?
The answer may differ among colleges.
At Foothill and De Anza Colleges, AUOs are written statements about what clients will experience, receive understand or learn as a result of a given service. The term “Client” refers to anyone who receives the service.
At MiraCosta College, AUOs are Administrative Unit Objectives that are specific, S.M.A.R.T. benchmark statements with Outcome Statements that express the expected achievement level (target). They focus on the operational services and organization support the unit provides.
13
How we define AUOs at our campuses…….
The colleges we represent defined AUOs differently in ways that fit the framework of our institutional planning and program review processes:
1. Learning based outcomes 2. Service/operational based outcomes
However, it is important to remember that the essence of assessment remains the same: self-reflection, evaluation, plan, implementation
14
A learning based example:
Institutional Research: After working with Institutional Research staff on projects requiring at least 2 hours of discussion, Faculty and Staff at FHDA will be able to:
· Interpret and draw correct conclusions from a cross tabulation of descriptive statistics such as course success rates by ethnicity.
Foothill & De Anza
15
Examples of Assessment
A learning based example:
The Institutional Research and Planning Office administered a survey to all data users in the fall by emailing a link to an assessment survey. The results will be discussed at a department meeting for improvement purposes and changes in services provided.
http://irp.fhda.edu/cgi-bin/rws5.pl?FORM=FHDA_AUO
Foothill & De Anza
16
Additional examples of assessment
17
MiraCosta Model Institutional Mission
Institutional Goals (in
CMP)
Institutional Objectives
(in Strategic Plan)
Action Plans, based on
Inst. Objectives
Division Objectives
Division or Dept. AUOs
(obj. with outcome
stmt.)
Establish measurable assessment and timeline
for each AUO
Assess, evaluate at end of year
↓↓↓
CMP:A 10-year
Plan
Strategic Plan: 3
Plans of 3 years each, incremental
Action Plans:
Annual, incremental
Results inform
Program Review
18
HUMAN RESOURCES 2011- 2012
MCCCD Mission Statement: The MiraCosta Community College District mission is to provide educational opportunities and student-support services to a diverse population of learners with a focus on their success. MiraCosta offers associate degrees, university-transfer courses, career-and-technical education, certificate programs, basic-skills education, and lifelong-learning opportunities that strengthen the economic, cultural, social, and educational well-being of the communities it serves. (adopted by the board on 9/20/2011)
Inst. Objective
ID # (in the
Strategic Plan)
Action Plan ID # (in the
Strategic Plan)
Action Plan Target Date(s)
(in the Strategic Plan)
Responsible Party (as indicated in the
Strategic Plan)
BAS Division Objectives Related to
the Strategic Plan
Administrative Unit Objectives Related to
the Strategic Plan Action
Plans/Division Objectives
Administrative Unit Objective
Outcomes Statement
Assessment Method(s), Measure(s)
Assessment Schedule (Timeline)
Assessment Results
compared to Outcomes Statement
Status (met, unmet, in progress)
Additional Notes (if any)
III.1
III.1.1 Oct 2011 Supt/Pres
Facilitate district efforts to centralize institutional planning and accreditation.
III.1.2: Develop selection committee, recruit candidates, and recommend finalists to the supt/pres
Oct-Dec 2011 HR Director
Administrative Unit Outcome Statement: A concrete, measureable statement expressing the expected achievement level (target) of an Administrative Unit Objective.
Administrative Unit Objectives: specific, S.M.A.R.T. benchmark statements written at the department/program level that advance the Administrative Unit in achieving broader Division Objectives, Action Plan Tasks (in the Strategic Plan), and, ultimately, the Institutional Objectives and Goals that guide the District in advancing its Mission Statement.
Assessment Method (Means of Assessment): the specific tool identified and used to gather information and data that track progress toward and/or achievement of an Administrative Unit Outcome Statement. The assessment method should match the intended outcome. There are "Direct" and "Indirect" assessment methods. Direct assessment methods are quantitative (examples: timeline, usage report, error rates, cost, productivity, efficiency, tests, rubrics); Indirect assessment methods are qualitative (examples: panel interviews, questionnaires & surveys, focus groups, observations, perceived satisfaction, perceived timeliness, perceived capability).
Assessment Schedule (Timeline): specifies the timeframe in which a plan/course of action will be assessed and who will administer the assessment.
Assessment Results Compared to Outcomes Statement: gathered, documented information and data indicating the outcome of an Administrative Unit's actions are compared to the target set by the Administrative Unit's Outcome Statement. The results of this comparison indicate the status of an Administrative Unit's performance and whether or not changes need to be implemented in order to improve performance.
MiraCosta’s current format (with future plans to migrate all to SharePoint)......
19
HUMAN RESOURCES 2011- 2012
MCCCD Mission Statement: The MiraCosta Community College District mission is to provide educational opportunities and student-support services to a diverse population of learners with a focus on their success. MiraCosta offers associate degrees, university-transfer courses, career-and-technical education, certificate programs, basic-skills education, and lifelong-learning opportunities that strengthen the economic, cultural, social, and educational well-being of the communities it serves. (adopted by the board on 9/20/2011)
Inst. Objective
ID # (in the
Strategic Plan)
Action Plan ID # (in the Strategic
Plan)
Action Plan Target Date(s)
(in the Strategic
Plan)
Responsible Party (as indicated in the
Strategic Plan)
BAS Division Objectives Related to
the Strategic Plan
Administrative Unit Objectives Related
to the Strategic Plan Action
Plans/Division Objectives
Administrative Unit Objective
Outcomes Statement
Assessment Method(s), Measure(s)
Assessment Schedule (Timeline)
Assessment Results
compared to Outcomes Statement
Status (met, unmet, in progress)
Additional Notes (if any)
III.1
III.1.1 Oct 2011 Supt/Pres
Facilitate district efforts to centralize institutional planning and accreditation.
III.1.2: Develop selection committee, recruit candidates, and recommend finalists to the supt/pres
Oct-Dec 2011 HR Director
HR department staff will provide support to selection committee chair & members to ensure an efficient recruitment process.
All steps in the recruitment process are completed & finalist(s) are recommended to the superintendent president.
Timeline & Internal assessment by HR Director & chair of selection committee
1. By Oct 31, 2011 Job opening announced with application deadline by Nov 30, 2011. 2. Within 1 wk. of closing date applications released to committee. 3. Within 3 days of candidates ID'd by committee, HR Technician to schedule interview candidates. 4. Within 3 days of Finalists ID'd, HR Technician to schedule finalist interviews. 5. Within 3 days of preferred candidate ID'd HR Director to complete reference checks. 6. Within 1 day after ID'd by supt/pres HR Director to make formal job offer.
Example of a service/operational based AUO statement & assessment related to the MiraCosta Strategic Plan......
20
Example of an operational based AUO objective, statement and assessment....
BAS Division Objectives: Operational
Administrative Unit Objectives: Operational
Administrative Unit Objective Outcomes
StatementAssessment Method(s),
Measure(s) Assessment Schedule (Timeline)Assessment
Results compared to Outcomes
Statement
Status (met, unmet, in progress)
Additional Notes (if any)
Maximize the effectiveness of the operational infrastructure and processes that support the student learning environment.
Work with Director of Risk Management to Research, identify costs, and form a recommendation for acquiring an online chemical reporting and tracking program for district wide MSDS management.
An online comprehensive inventory management system is selected to collect and house Material Safety Data Sheets (MSDS) for all chemicals on district property.
2011-2012 - 1) Timeline: assist Dir. of Risk Management to research, identify costs, and select MSDS inventory management system. 2012-2013 - 1) Timeline: Seek funding. 2013-2014 - 1) Timeline: If funding is granted, target dates for purchasing and implementation. 2) Compliance with current federal, state, and local regulations.
2011-2012: By 3/1/2012 Risk Mgmt Director forms a committee to include Dir of Facilities to research available online MSDS inventory management systems. By 5/1/2012 Committee develops a final recommendation. 2012-2013: September 2012: Risk Mgm.t Director requests funding in a Program Development Plan during the annual Program Review process. If funded, Funding becomes available 7/1/2013. 2013-2014: By 8/31/2013: If funding is awarded, purchase and install online MSDS inventory management system. By 12/1/2013 MSDS for all chemicals on district property are housed in the online MSDS inventory management system.
WRITING AUOS
22
23
Writing an AUO Use the SMART model:
Specific: What is your unit’s intended outcome?Measurable: Can it be assessed effectively?Attainable: Is it reasonable?Relevant: Does it serve the needs of the
college? Does it align/support the institutional goals?
Time-bound: Is the time-frame clearly defined, with specific start and target dates?
24
WRITING EXERCISE - USING THE FOLLOWING
STEPS, WRITE: An Administrative Unit Outcome Statement
Handout
25
Step 1 Select an Administrative Unit and
identify one service it provides.
Step 2 Identify one objective or goal for the unit,
and write a statement that specifies the intended outcome (target). (hint: phrase the outcome as if it already has
occurred.)
26
Step 3 If using the learning based model, what should people
gain from this administrative unit? KnowledgeSkillsAttitudeBehavior
If using the service/organizational support model, considerEffectivenessEfficiency (cost savings measures, turn-around time,
improving processes)Level or volume of activityCompliance with external standards/regulationsClient satisfaction
27
Step 4 Use Active/Action verbs
Writing statements with active/action verbs○ Concrete○ Refer to a process○ Produce statements that are measurable
IdentifyProvide
CollectComplete
ImplementInvestigate
SelectModify
Schedule
28
Step 4 (continued) Avoid using static verbs
State-of being or condition verbsProduce statements that are difficult to measure
reliably
BelieveLike
FeelKnow
NeedThink
Want
29
WRITE
ASSESSING AUOS
31
Assessment Assessment Timeline
What to measure?○ Select AUOs to assess on a regular cycle
When to measure?○ Depending on each institution’s cycle
Who will measure?○ Assign tasks○ Resources needed
32
Assessment Methods Assessment Methods
Direct (Quantitative) – examples:○ Timeline○ Error rates○ Productivity○ Cost
Indirect (Qualitative) – examples:○ Questionnaires/surveys○ Interviews○ Focus Groups○ Observations
33
Assessment Strategies Involve all stakeholders Work with what already exists Design efforts with assessment in mind Collect data from a targeted sample Work with other units that have similar
outcome statements
34
WRITING EXERCISE:Write an Assessment Method
and anAssessment Timeline
for your AUO
Handout
EVALUATION
36
EvaluationThe process of reviewing assessment results (data) against the stated criteria (the outcome statement).
Purpose: Prepare for Program Review
PROGRAM REVIEW
38
Program ReviewTo Recap:
Assessment results inform Program Review
Purpose: to draw data-based conclusions that guide
future action
39
The End
Questions?
Documents & Resource Linksare available at:
http://ssscauosession.wikispaces.com/