Bri Hays Jill Baker

Degrees: Conducting a

Comprehensive Evaluation of Your

Integrated Planning Processes

360

San Diego Mesa College

RP ConferenceApril 11, 2014

• What is one thing you would like to learn in today’s session?

Introductions and Context

• Describe 360-degree feedback approach and applications to integrated planning

• Apply process evaluation principles to integrated planning

• Discuss research methods appropriate to a comprehensive evaluation of integrated planning processes

• Outline an evaluation plan for college integrated planning processes

• Identify strategies for facilitating the use of data for action planning and improvement

Session Learning Outcomes

• Multi-source feedback provides a more comprehensive perspective of performance• Feedback from various stakeholders at differing levels

• Foundations in the business world (HR) and I/O psychology

360 Feedback Approach

Evaluand

Information Source 1

Information Source 2

Information Source 3

Information Source 4

Information Source 5

Applying the 360 approachto process evaluation

The Impetus for Process Evaluation

Is the process working as intended?

How can we improve the process for the end-user?

Could additional efficiencies be achieved?

What is your definition of integrated planning?

“…the linking of vision, priorities, people, and the physical institution in a flexible system of evaluation, decision-making and action. It shapes and guides the entire organization as it evolves over time and within its community.” –Society for College and University Planning

Integrated Planning Is…

• Process Model• List of Key Stakeholders• Outline of Process Goals• List of Evaluation Research Questions• Process Timeline or Calendar

Integrated Planning Process Evaluation: What You Need

Step 1

Step 2

Step 3

Getting Started: What’s The Model

Step 1

Step 2

Step 3Step

1

Step 2

Step 3

Step 4

Step 5

• Program Review• Outcome Assessment• Resource Request Process• Mission and Vision• Institutional Plans (Ed Master Plan, Facilities

Plan, Tech Plan, HR plan)• Prioritization Process (Committees)• Allocation Process• Follow-Up/Evaluation

Integrated Planning Components

Sample IP Model

Integrated Planning Stakeholders

Faculty

Staff Managers

Prioritization Committees

Students

Community

• What are the overarching goals of the college’s integrated planning process?• Increase transparency in decision-making• Clearly link data, planning, and resource allocation• Inform the strategic planning priorities

• What are the operational objectives of the integrated planning process?• Increase efficiency of the integrated planning process• Eliminate redundancies• Align planning and resource allocation calendars

Identifying Integrated Planning Process Goals

• Was the process carried out as intended?• How effective and efficient was the flow of

information?• Did the correct information make its way to the

appropriate end point?• Was the information used as intended?• How do stakeholders perceive the process?• Did the process/activities work as planned?• Were resources allocated according to the plan?• How can the process be improved?

Outlining Process Evaluation Research Questions & Goals

• What are the goals of the evaluation?

• What data are needed to address the evaluation goals?

• Who are the key stakeholders in the process?

• How do the results of the evaluation need to be reported out and to whom?

• How will the information gained from the evaluation be used?

Framing the Evaluation

Program Review

Resource Allocation

• Set goals for the evaluation itself based on the goals/objectives of the process (or process redesign)

Setting Evaluation Goals

• What methods will you utilize to assess the integrated planning process?

• Examples include:• Surveys• Focus Groups• Campus-wide forums• Committee feedback or debriefing sessions• Program review and resource request content• Institutional effectiveness data/key performance

indicators

A Mixed Methods Approach

• The evaluation plan should clearly illustrate the alignment of process goals/objectives, evaluation goals, evaluation activities, and the timeframe during which activities will take place

The Evaluation Plan

Process Goal/Objective

Evaluation Plan Goal

Evaluation Activities

Timeline Person(s) Responsible

Improve transparency of resource request prioritization

Assess stakeholder perceptions of transparency

• Survey of faculty, staff, and students

• Focus group with department chairs and supervisors

• Focus group with resource allocation committees

Spring 2013 Campus Based Researcher; Dean of IE; Program Review Committee

Clarify connections between planning, data, and resource allocation

Assess the program review utilization of data and links between unit and college goals

• Content analysis of unit goal form data

• TaskStream reports on goal mapping

Summer 2013 Dean of IE; Campus Based Researcher

Sample Evaluation Plan

• What deliverables are best suited to your college?

• How can the evaluation results be turned into action items?

• What structures and processes are in place at your college to ensure the results are reported out and action is taken?

Conveying the Evaluation Results

Sample Results Summary Table

Closing the Loop

Plan

Implement

AssessAnalyze

Identify Action Items

FormativeEvaluation Cycle

• Analyze all information collected and organize it by topic or component

• Provide a conceptual framework for moving forward with specific recommendations

• Recommendations, after being vetted, become goals for the next cycle…

• …Which inform the following year’s evaluation plan

From Data to Action

Sample Evaluation Executive Summary

• Start with a baseline• Results of the baseline study inform changes to the

integrated planning process for the following year• Goals/objectives for the subsequent cycle are based

on the results of the previous cycle’s evaluation• Develop a calendar of activities to meet new goals

and objectives—list person(s) or group(s) responsible

• Track progress toward goals/objectives• Address the limitations of the previous cycle’s

evaluation

Paving the Way for Continuous Quality Improvement

• How do you integrate results into the larger assessment of the college’s institutional effectiveness?

• What process does your college have for assessing the effectiveness of resource allocation?

Next Steps

Bri HaysCampus Based ResearcherSan Diego Mesa Collegebhays@sdccd.edu

Jill BakerDean of Institutional EffectivenessSan Diego Mesa Collegejibaker@sdccd.edu

Thank You!