Integrating Higher Education Planning and Assessment: Concepts and Strategies Michael F. Middaugh Assistant Vice President for Institutional Research and

Integrating Higher Education Planning and Assessment:

Concepts and Strategies

Michael F. MiddaughAssistant Vice President for Institutional Research and Planning

University of Delaware

Commissioner and Member of Executive CommitteeMiddle States Commission on Higher Education

Andrea A. LexDean of Planning and Institutional Research

Prince George’s Community College

“The nicest thing about not planning is that failure comes as a complete surprise and is not preceded by a period of worry and depression.”

John Preston, Boston College

“A common mistake that people make when trying to design something completely foolproof is to underestimate the ingenuity of complete fools.”

Douglas Adams, Hitchhikers Guide to the Galaxy

A Brief Overview of Accreditation

What is accreditation?

• A means of providing quality assurance in the areas of– Student learning and achievement– Curriculum development and support– Faculty– Facilities and equipment– Finance, administration, and governance– Integrity and compliance

Why is it done?

• Voluntary self-regulation• Voluntary self-examination• Opportunity for peer review• Quality assurance• Public accountability• And, in the case of regional accreditation

– Ability to receive Federal financial aid dollars

What do we use?Characteristics of Excellence

• Institutional Effectiveness– Planning– Resource Allocation– Leadership– Governance– Administration– Integrity– Institutional Assessment

• Educational Effectiveness– Admissions & Retention– Support Services– Faculty– Offerings– General Education– Related Education

Activities– Assessment of Student

Learning

Institutional Mission

What have we learned?

• That the quality of an institution’s accreditation self study hinges on how it addresses the standards related to – assessing institutional effectiveness

(Standard 7)– assessing student learning outcomes

(Standard 14)– implementing a vital and comprehensive

strategic planning process (Standard 2).

Commission-Directed Follow-up(2004-2009)

• 81 percent required some sort of follow-up

• Student Learning Outcomes (S 14)

• Institutional Effectiveness (S 7)

• Strategic Planning (S 2)

• Sufficient Resources (S 3)

• Enrollment Management (S 8)

Creating and Closing the Loop

• Institutional Mission is the Foundation of & Our Reason for Being

Creating and Closing the Loop Characteristics of Excellence

Educational Effectiveness– Admissions & Retention– Support Services– Faculty– Offerings– General Education– Related Education Activities– Assessment of Student

Learning

To what extent have students achieved the educational and co-curricular goals set forth by an institution as it recruits and retains these individuals?

-- Assessment of Student Learning (Standard 14)



Standard 14: Student Learning Outcomes

• Assessment of student learning demonstrates that, at graduation, or other appropriate points, the institution’s students have knowledge, skills, and competencies consistent with institutional and appropriate higher education goals.

Selected Fundamental Elements forMSCHE Standard 14

• Articulated expectations for student learning (at institutional, degree/program, and course levels)

• Documented, organized, and sustained assessment processes (that may include a formal assessment plan)

• Evidence that student learning assessment information is shared and used to improve teaching and learning

• Documented use of student learning assessment information as part of institutional assessment



To what extent has the institution acquired and allocated human and fiscal resources in a manner that supports the achievement of its goals?

-- Assessment of Institutional Effectiveness (Standard 7)



Standard 7: Institutional Assessment

– The institution has developed and implemented an assessment process that evaluates its overall effectiveness in achieving its mission and goals and its compliance with accreditation standards.


• Documented, organized, and sustained assessment processes to evaluate the total range of programs and services, achievement of mission, and compliance with accreditation standards

• Evidence that assessment results are shared and used in institutional planning, resource allocation and renewal

• Written institutional strategic plan(s) that reflect(s) consideration of assessment results

• Evidence that institutional assessment findings are used to improve student success



• Educational Effectiveness– Admissions & Retention– Support Services– Faculty– Offerings– General Education– Related Education Activities– Assessment of Student Learning


Pla

nn

ing

Planning Planning

To what extent are an institution’s policy, decision making and resource allocation processes shaped by a systematic, evidence-based, and mission-guided planning process? --- Assessment of Institutional Planning (Standard 2)


Standard 2: Planning, Resource Allocationand Institutional Renewal – An institution conducts ongoing planning and

resource allocation based on its mission and goals, develops objectives to achieve them, and utilizes the results of its assessment activities for institutional renewal.

– Implementation and subsequent evaluation of the success of the strategic plan and resource allocation support the development and change necessary to improve and to maintain quality.


• Clearly stated goals and objectives, at both the institutional and unit levels, that reflect conclusions drawn from assessments that are used for planning and resource allocation at those levels

• Planning and improvement processes that are clearly communicated, provide for constituent participation, and incorporate the use of assessment results

• Well-defined decision-making processes and authority that facilitates planning and renewal and the assignment of responsibility for improvement and assurance of accountability

• Periodic assessment of the effectiveness of planning, resource allocation, and institutional renewal processes

Central Threads Running Through All Accreditation Requirements:

• Planning must be systematic

• Planning must be rooted in an institution’s mission

• Planning must be predicated on analytical and evaluative information

• Planning must be used for institutional decisions, including resource allocation

The Closed Loop

• Institutions must plan effectively in order to be effective.

• Where that is the case, the accreditation process is an affirmation of the evidence of that effectiveness.

Components of This Curriculum

• Best Practice in Planning

• Best Practice in Assessment

• Closing the Loop Between Planning and Assessment

• Real Life Case Studies

End Result

• Institutions must plan effectively in order to be effective.

• Where that is the case, the accreditation process is an affirmation of the evidence of that effectiveness.

Components of This Curriculum

• Best Practice in Planning

• Best Practice in Assessment

• Closing the Loop Between Planning and Assessment

• Real Life Case Studies

In thinking about what good planning is, it is useful to begin the discussion by describing what it is not:

George Keller, Academic Strategy: the Management Revolution in American Higher Education, Johns Hopkins University Press, 1983.

Strategic Planning is not:

• The production of a blueprint.

• A set of platitudes.

• The personal vision of a president or board of trustees.

• A collection of departmental plans, compiled and edited.

Strategic Planning is not:

• Done by planners.

• A substitution of numbers for important intangibles.

• A form of surrender to market conditions and trends.

• Something done on an annual retreat.

• A way of eliminating risks.

• An attempt to read tea leaves and outwit the future.

Strategic Planning IS

• Academic strategic decision-making means that a college or university and its leaders are active rather than passive about their position in history.

• Strategic planning looks outward and is focused on keeping the institution in step with the changing environment.

• Academic strategy making is competitive, recognizing that higher education is subject to economic market conditions and to incredibly strong competition.

• Strategic planning concentrates on decisions, not on documented plans, analyses, forecasts, and goals.

Strategic Planning IS

• Strategy making is a blend of rational and economic analysis, political maneuvering, and psychological interplay. It is therefore participatory and highly tolerant of controversy.

• Strategic planning concentrates on the fate of the institution above everything else.

Action- Action- Action-Oriented Oriented Oriented

Goals Goals Goals

Measurable Measurable MeasurableObjectives Objectives Objectives


Cornerstones of the Planning Process

Let’s begin at the beginning . . .

Why are we here?

Mission Statement

• A good mission statement is a carefully reasoned analysis of what an institution aspires to be, and the core values that it embraces.

• It avoids cliché language, e.g., “Students and faculty will interact in a rich intellectual environment in which each individual has the opportunity to achieve their full potential.” Noble sentiment, but says nothing about the institution’s purposes and priorities.

• Mission must speak to central institutional issues, e.g., desired balance between undergraduate and graduate education; relative emphasis on teaching, research, and service, respectively; and so on.

Mission Statement

• Mission statements are characterized by a sense of vision that, while not immutable, nonetheless represents a long-term statement of institutional values and direction around which human and fiscal resource allocation decisions can be made.

• While cognizant of the institution’s ever changing external environment, mission statements are not whimsical, morphing with each new market trend that emerges.

• The mission statement provides a clear sense of direction around which action-oriented goal statements and measurable planning objectives can be developed.

Planning Goals

• Goal statements are derived from the institutional mission, and help to define policy.

• For example, the mission statement might say that “The University affirms its historic mission of providing the highest quality education for its undergraduate students, while maintaining excellence in selected graduate programs.”

• The mission statement is underscoring the primacy of undergraduate instruction in the curriculum. The question for planners is how to provide that high quality undergraduate instruction.

How will we achieve our mission?

Planning Goals

• The “how” translates into specific, action-oriented planning goals aimed at moving the institution toward a fuller realization of its mission.

• Possible goal related to the undergraduate education mission statement: “The University will continue to attract and retain the most academically talented and diverse undergraduate students, and support their intellectual, cultural, and ethical development as citizens and scholars.”

• Action verbs such as “attract,” “retain,” and “support” elevate the goal statement to policy level. How do we know that policy is being carried out? Measurable planning objectives.

Meeting Goals

• How will we know when we get there?

• What will we look like when we get there?

Planning Objectives

Planning objectives provide empirical evidence of the extent to which planning goals are being achieved. Consider the following planning objectives as they relate to our goal to attract, retain, and support academically talented and diverse students.

• Retain a freshman admissions target of 3200 to 3400 students annually, with an admissions profile for academic year 2007 of 23,000 applications, a 40 percent admit rate, and a yield rate in excess of 35 percent.

• Improve the alignment of undergraduate enrollment distribution and instructional resource distribution across the disciplines, especially with respect to faculty.

Planning Objectives Continued

• Maintain a freshman-to-sophomore retention rate above the national average for highly selective institutions, and seek to achieve a consistent rate of 90 percent or higher.

• Maintain a graduation rate above the national average for highly selective institutions, and seek to achieve a consistent six-year rate of 75 percent or higher.

• Increase minority and international enrollment, with retention and graduation rates for those populations consistent with the university-wide averages for all students.

How are we doing?

• How close are we to getting there?

• How are we doing along the way?

Planning Objectives

• The defining characteristic for any good planning objective is that it must be measurable. In other words, assessable.

• Colleges or universities embarking on any planning process – long range, strategic, tactical – require a systematic institutional research capability.

• While smaller institutions may not have an office of institutional research, per se, they must nonetheless have the capability of quantitatively and qualitatively assessing the extent to which planning objectives are being implemented, planning goals are being achieved, and the institutions mission is being realized.

Breakout Session

Using your own Institutional Mission Statement, create at least five planning goals and associated measurable planning objectives that will help assess the extent to which that mission is being realized.

Linking Planning and Assessment

Two Case Studies at

The Same Institution

The First Cut: Creating a Culture of Planning

University of DelawareStrategic Planning: 1990 - 2000

University of Delaware Mission Statement

“The University reaffirms its historic mission to provide the highest quality education for its undergraduate students, while maintaining excellence in selected graduate programs. The faculty are responsible for helping students learn to reason critically and independently, gain knowledge of the diverse culture and environment in which they live, and develop into well-informed citizens and leaders. To accomplish these goals, the University provides a learning setting enriched by undergraduate student research, experiential learning, and study-abroad programs. The University places high priority on the education of qualified Delaware residents and provides opportunity for a diverse group of citizens to participate in postsecondary education. Since the University is located in a state with a small population, providing programs of quality and diversity requires a community of student-scholars that reaches beyond the boundaries of the state, one that reflects the nation’s racial and cultural diversity.

“The University of Delaware also aspires to excellence in graduate education, the heart of which is scholarship and research. The creation, application, and communication of knowledge is a primary goal of the institution and of every faculty member, providing the substance for creative, informed teaching. Research is typically based on cooperation between faculty and students, whereby faculty mentors teach students to conduct independent research and to master problem-solving techniques. Through involvement of undergraduates in faculty research, the University creates a special bond between its undergraduate and graduate programs.

“The University is also committed to providing service to society, especially in Delaware and the neighboring region. Public service is a responsibility of every academic unit. In addition, each faculty member is responsible for service to the University community and to his or her profession. The University emphasizes practical research, provides extension services, and works to solve problems confronting the community.

(University of Delaware, Middle States Self-Study Report, April 2001)

Case Study – University of Delaware

• In 1987, the University embarked on a comprehensive, long-range planning process, termed Project Vision. Over a period of 18 months, the campus developed a planning document with a broad spectrum of planning goals and measurable objectives embracing all aspects of University operations.

• In Fall of 1988, the President who initiated Project Vision suddenly resigned. At the same time, the Delaware economy along with that of the entire mid-Atlantic region was plunging into deep recession


• Rather than let 18 months of planning activity go for naught – even though resources would be scarce for the foreseeable future – a panel of distinguished senior faculty was assembled to review the Project Vision planning document and to cull out those goals and objectives that were clearly consistent with, and essential to furthering the University mission.

• The resulting document, Focused Vision, was economical when compared with its progenitor, both in terms of words and resource requirements. However, the economy would still clearly preclude anything even remotely approaching implementation.

• In 1990, the University hired its 25th President, David P. Roselle.


• In order to maintain planning momentum, the President consulted with his senior staff, the faculty, and appropriate constituencies across campus to determine those areas that required immediate attention.

• From these consultations, the President articulated four strategic initiatives that would constitute the focus of decision-making and resource allocation activity in the immediate future. Those initiatives were competitive compensation for faculty and staff; enhanced access to the University for undergraduates through expanded availability of financial aid; a more student-centered campus environment; and renovation and rehabilitation of campus facilities.


These priorities were not a “wish list.” They grew out of a careful examination of empirical data provided by the University’s Office of Institutional Research and Planning and other data sources. Consider the following:

• When compared with the 24 Category Doctoral I universities in the states contiguous to Delaware, and the District of Columbia, in 1991 the average salary for all three major faculty ranks at the University of Delaware ranked near the bottom of the list.

• The Student College Selection Survey indicated that students were receiving offers of more aid from admissions competitors, and that the aid packages had more grants and fewer loans than University aid packages. Not surprising, the University was at a competitive disadvantage for academically talented students.


• University scores on the ACT Student Opinion Survey suggested that the institution had considerable room for improvement with respect to student satisfaction with programs and services, and with a number of areas in student life.

• The University was looking at in excess of $200 million in deferred maintenance to its buildings and grounds.


• A critical factor in moving forward with these initiatives was getting the campus to understand that the economy was in recession and that there would be no immediate or massive infusions of new resources.

• Colleges and universities have multiple revenue streams – tuition, state appropriation in the case of public institutions, contracts and grants, gifts, etc. While growing revenue streams is an important strategic initiative, so too is the commitment to not balance budgets on the backs of students through inordinately large tuition increases.

• Resource reallocation would be the primary source of funding the four strategic initiatives, and it was critical that the campus understand from where funds were reallocated, and why.

The University went on public record in 1991:

• Average total compensation for faculty at each academic rank would be at or above the median within five years for the 24 Category I Doctoral Universities identified as salary peers.

• Total undergraduate financial aid from all sources would increase by 100 percent within five years.

• Student satisfaction with programs and services at the University, as measured through the ACT Student Opinion Survey would demonstrate significant gains within five years.

• The University would commit itself to a policy of annually setting aside at least 2 percent of the replacement value of the physical plant, to be used for facilities renovation and rehabilitation.

Results - SalariesTotal Compensation, 1989-90 Compared with 1999-2000

FULL PROFESSOR

Institution 1989-90 1999-2000

Princeton University 91,800 University of Pennsylvania 151,900

University of Pennsylvania 91,500 Princeton University 134,100

Georgetown University 88,700 Georgetown University 126,100

Carnegie-Mellon University 85,800 Carnegie-Mellon University 123,200

University of Virginia 85,100 University of Virginia 122,900

George Mason University 84,400 UNIVERSITY OF DELAWARE 116,200

J ohns Hopkins University 83,800 Temple University 114,300

New J ersey Institute of Technology 83,100 George Mason University 113,900

Rutgers University - New Brunswick 82,000 College of William and Mary 113,400

Lehigh University 80,800 Lehigh University 111,300

University of Maryland - College Park 80,500 American University 111,100

Virginia Polytechnic Inst. & State University 76,700 J ohns Hopkins University 110,800

-------------------------------------------------------------------------------------------------------------------------------------------------------

University of Maryland - Baltimore County 76,700 George Washington University 110,800

-------------------------------------------------------------------------------------------------------------------------------------------------------

University of Pittsburgh - Main Campus 76,500 New J ersey Institute of Technology 109,200

George Washington University 76,500 Rutgers State University-N.Brswck 109,000

American University 76,300 Pennsylvania State University 108,600

Pennsylvania State University 75,300 University of Maryland - College Park 106,800

UNIVERSITY OF DELAWARE 74,700 University of Pittsburgh - Main Campus 105,600

College of William and Mary 74,400 Virginia Polytechnic Inst. & State University 103,800

Virginia Commonwealth University 72,600 Drexel University 103,700

Drexel University 71,000 Virginia Commonwealth University 103,300

Old Dominion University 70,400 University of Maryland - Baltimore County 96,200

Temple University 68,400 Howard University 93,300

Catholic University of America 64,800 Catholic University of America 86,800

Howard University 64,000 Old Dominion University No Report

Results – Financial Aid

Growth In Undergraduate Financial Aid at the University of Delaware, FY 1990 Compared with FY 2000

FY 1990 FY 2000 % Increase

University Administered Funds $4,458,640 $28,036,660 528.8

State Grant Funds $3,869,000 $6,643,500 71.7

Other Fund Sources $2,169,602 $4,394,180 102.5

TOTAL $10,497,242 $39,074,340 272.2

Results – Student Satisfaction

Comparison of 1995 University of Delaware Scores on ACT Student Opinion Survey with 1990 Scores and with National Norms

1990 Scores 1995 Scores 1995 ScoresCompared with Compared With Compared withNational Norms 1990 Scores National Norms

University is Ahead 21 44 39

University is Tied 14 11 15

University is Behind 27 7 8

Results - Facilities

• By 2000, the University had renovated every classroom in its entire building inventory, retrofitting most with state-of-the-art teaching technology.

• An aggressive program of fundraising enabled not only the aforementioned renovation and rehabilitation, but also the construction of several new classroom and student services buildings.

• The University is now on a cycle of planned maintenance, as opposed to deferred maintenance.

Results – From an Accreditation Perspective

“ The University of Delaware has every reason to take enormous pride in what it has accomplished over the past 10 years. A decade ago, it was coming out of a period of considerable turmoil. Today, the University is seen as a national model for the integration of information technology in every aspect of university life: teaching and learning, research and service, academic support, and campus administration. It has created a physical plant that has few, if any, peers among public universities and would be the envy of most private colleges. These substantial achievements could not have happened without extraordinary leadership from the senior administration.”

“ Better than almost any university we are familiar with, Delaware has a clear sense of what it wants to be, namely, a university that offers high quality undergraduate education with targeted areas of excellence in graduate education and research.”

" The review team was enormously impressed by the high level of morale

that pervades the faculty, staff, and students. Almost without exception, the people we spoke to take great pride in being part of the University.”

Middle States Evaluation Team, 2001

• The University submitted its Periodic Review Report to the Commission on Higher Education in May 2006, outlining its strategies for incorporating systematic assessment of student learning outcomes into all facets of campus planning.

• The Commission thanked the University for its Report, commended the University on its content, and reaffirmed accreditation.

What Was Not Apparent to MSCHE…

• Despite an exemplary self study and PRR process, planning was not yet institutionalized into a formal, iterative process at the institution.

• The four strategic initiatives articulated in 1990 continued to be the benchmarks against which the University measured success well into the 2000 decade.

• The University needed to move forward with a new strategic planning effort that would more substantively incorporate planning into the organizational fiber of the institution.

The Second Cut: Institutionalizing Planning

University of DelawareStrategic Planning: 2007 - ???

The Path to Prominence

• On July 1, 2007, Patrick T. Harker succeeded David Roselle as the 26th President of the University.

• As former Dean of the Wharton School at the University of Pennsylvania, President Harker was intimately familiar with the concept of strategic planning.

• He immediately appointed a Strategic Planning Committee led by two distinguished senior faculty and comprised largely of individuals from the academic as opposed to administrative side of the institution.

The Path to Prominence

• During the 2007-08 academic year, the Strategic Planning Committee met with over 100 constituent groups from within, and external to the University. These information-gathering sessions were part of a comprehensive SWOT analysis.

• In April 2008, the Report of the Strategic Planning Committee was delivered to the President.

• In July, 2008, the President presented the Path to Prominence, the strategic planning document that would guide institutional activity for the next several years.

The Path to Prominence: The Strategic Initiatives

• A Diverse and Stimulating Undergraduate Academic Environment

• A Premier Research and Graduate University

• Excellence in Professional Education

• The Initiative for the Planet

• The Global Initiative

• An Engaged University


• Each of the six initiatives was operationalized with measurable goals and planning objectives.

• The Path to Prominence document and supporting materials were broadly circulated via the University website: http://www.udel.edu/prominence/ for maximum transparency and accountability with respect to what was being accomplished.

• In Fall 2008, the U.S. and Delaware economies began to unravel, placing severe financial constraints on full implementation of the strategic initiatives.

http://www.udel.edu/prominence/


• While each of the six initiatives remain viable and will receive attention regardless of economic circumstances, hard decisions needed to be made with respect to where resources would be allocated.

• It was determined that three initiatives would be the primary (but not exclusive) focus of resource allocation:

- A Diverse and Stimulating Undergraduate Academic Environment

- A Premier Research and Graduate University

- Excellence in Professional Education

The Path to Prominence: Moving Forward

• The University decided to take advantage of the MSCHE decennial accreditation reaffirmation process, and utilize the “Special Topics Model” of self study as a means of documenting progress in implementing these three key strategic initiatives.

• The scheduled 2011 site visit by an MSCHE Evaluation Team affords the University not only the opportunity to demonstrate compliance with the 14 standards in Characteristics of Excellence; it provides us with a team of expert consultants in the topical areas on which we have elected to focus. Their feedback on the content of the Special Topics Self Study will prove invaluable in moving the institution forward.

The Path to Prominence: Tactical Issues

• In any strategic planning process, there needs to be a clear linkage between human and fiscal resource allocation and the initiatives outlined in the strategic plan.

• The University of Delaware has adopted a responsibility centered budget management model that is designed to financially reward activity that supports full implementation of key strategic initiatives.

• The University has also designated an Associate Provost for Institutional Effectiveness, tasked with overseeing assessment activity that measures the effective implementation of The Path to Prominence.

The Path to Prominence: Tactical Issues

• The Steering Committee for the 2011 MSCHE has tasked itself with responsibility for design of a formal, iterative institutional strategic planning process that will, on an ongoing basis, continuously evaluate the implementation of existing strategic initiatives, assess the continuing relevance of initiatives over time, and develop new initiatives, as appropriate, in the face of a changing external environment and/or advent of specific strategic opportunities.

• The intent is to institutionalize the strategic planning process so that it is truly an organizational function rather than the creation or vision of any particular individual.

Considerations When Creating a

Strategic Planning Process

“Planning to Plan:”

Organizing Your Institutionfor the Planning Process

Organizational Structure

• The Planning Committee: Who Serves? How Representative? What is the Scope of Responsibilities:

• If the Planning Committee is to be effective, it must be “blessed” by the Chief Executive Officer and the Chief Academic Officer. The President and Provost need not be members of the Committee, but ideally will attend the initial meeting, and will formally receive the Committee Report.

Organizational Structure

• Who staffs the Planning Committee?

Institutional Research

Budget Office

OthersFacultyStaff

Planning Tools

• Surveys: Students– Admitted Student Questionnaire– Entering Student Needs Assessment Survey– Student Satisfaction Survey– Post-Graduation Placement Survey– Alumni Survey

• Surveys: Employees– Employee Satisfaction Survey– Faculty Activity Analysis– Campus Climate Survey

Planning Tools

• Institutional Studies:

– Admissions Monitoring Analyses

– Faculty Workload Analyses

– Budget Support Data

– Persistence/Graduation Analyses

– Compensation Studies: Market/Equity

– Financial Ratio Analyses

– Benchmarking Studies

– Economic Impact Studies

Planning Tools

• Academic Program Review– Programmatic Self-Examination– Internal and External Review Teams– Focus on Several Levels of Quality

1. Programs With National/International Distinction

2. Programs of Regional Distinction (Meet Local Needs)

3. Programs That, With an Infusion of Resources, Could Achieve Distinction

4. Programs Targeted for Retrenchment or Elimination

Planning Tools

• Environmental Scan:

a. Demographic issues

b. Market issues

c. Political issues

d. Economic issues

e. Technological issues

f. Natural disasters

g. Other

Planning Tools – SWOT Analysis

• Take an honest look at your institution within the context of a SWOT analysis (Strengths; Weaknesses; Opportunities; Threats)

• Leveraging Situation - Opportunity + Strength

Constraining Situation - Opportunity + Weakness

Vulnerable Situation – Threat + Strength

Problematic Situation – Threat + Weakness

In the Final Analysis, Planning is Directed at Answering Four Basic Questions:

1. Who are the markets we are trying to serve?

2. What services must be in place to fully serve those markets?

3. What is the institutional “branding” that will enable our college or university to appeal to those markets?

4. How will we know if we are successful in serving those markets?

Largely Dictated by Mission

• Baccalaureate Colleges: Primarily undergraduate students (what kinds?)

• Master’s Institutions: Undergraduate students, selected graduate students, targeted state/community partnerships

• Doctoral Universities: Undergraduate students, graduate students, research/public service contractors/grantors, broad range of regional, national, international partnerships

• Community Colleges: Matriculated undergraduate students, occasional students with specific training needs, general interest students, local businesses, county government

Who are the markets we are trying to serve?

What services must be in place to fully serve those markets?

Dictated by Markets

• Academic Support Services (Library, computing, advising, tutoring, etc.)

• Student Support Services (Residence life, counseling, health services, student center, recreation services, etc.)

• Institutional Support Services (General administrative support, sponsored research, extension office, alumni/governmental relations, etc.)

What is the institutional “branding” that will enable our college or university to appeal to those markets?

Market Specific

• University of Delaware: “A Teaching University”

• Trenton State College : “The College of New Jersey”

• University of Phoenix: “The University for Working Adults”

How will we know if we are successful in serving those markets?

Middle States Assessment Standards

• Standard 7: The institution has developed and implemented an assessment process that evaluates its overall effectiveness in achieving its mission and goals and its compliance with accreditation standards.

• Standard14: Assessment of student learning demonstrates that, at graduation, or other appropriate points, the institution’s students have knowledge, skills, and competencies consistent with institutional and appropriate higher education goals.

Breakout Session

How would you respond to the four key planning questions within the context of your institution’s mission?

In the Final Analysis, Planning is Directed at Answering Four Basic Questions:

1. Who are the markets we are trying to serve?

2. What services must be in place to fully serve those markets?

3. What is the institutional “branding” that will enable our college or university to appeal to those markets?

4. How will we know if we are successful in serving those markets?

What Should Be Assessed Under

Standard 7?

• You do not have to measure everything.

• PRIORITIZE what is to be measured within the context of your institution’s culture and needs.

Students

• Admitted• Entering• Continuing• Non-Returning• Graduating• Alumni

Environmental Issues

• Student and Faculty Engagement

• Student and Staff Satisfaction

• Employee Productivity

• Compensation

- Market

- Equity

• Campus Climate

• Economic Impact

Some Basic Measures

A Typical Admissions Monitoring Report

Campus Summary: First-Time Freshman Applicants, Their SAT Scores and Predicted Grade Index, by Admission Status and by Residency Status for the Entering Classes in theFall of 2005 as of 09/15/05; Fall 2006 as of 09/18/06; and Fall 2007 as of 09/13/07.

2005 2006 2007 2005 2006 2007 2005 2006 2007 2005 2006 2007 2005 2006 2007 2005 2006 2007Counts - Resident 2,340 2,332 2,088 109 148 172 1,940 1,877 7,147 1,362 1,255 1,174 0.83 0.80 3.42 0.70 0.67 0.61 - Nonresident 18,984 19,209 20,133 7,506 5,871 5,838 7,295 8,101 8,489 2,078 2,201 2,348 0.38 0.42 0.42 0.28 0.27 0.28 - Total 21,324 21,541 22,221 7,615 6,019 6,010 9,235 9,978 15,636 3,440 3,456 3,522 0.43 0.46 0.70 0.37 0.35 0.23

SAT Verbal - Resident 557 562 569 456 444 445 567 576 584 559 564 575 - Nonresident 572 579 584 541 535 536 609 615 620 592 600 604 - Total 571 577 582 540 532 533 600 607 614 579 587 594

SAT Math - Resident 563 567 574 454 446 452 547 582 589 566 570 580 - Nonresident 596 598 605 561 551 554 636 635 643 619 620 627 - Total 592 595 602 560 549 551 623 625 634 598 602 611

Predicted GradeIndex - Resident 2.67 2.75 2.83 1.66 1.60 1.71 2.75 2.78 2.85 2.75 2.78 2.85 - Nonresident 2.90 2.94 2.94 2.40 2.36 2.41 2.98 3.04 3.05 2.98 3.05 3.04 - Total 2.82 2.88 2.88 2.22 2.30 2.37 2.90 2.95 2.98 2.89 2.95 2.98

AllApplicants

AdmissionDenied

Eastern Seaboard State UniversityWeekly Admissions Monitoring Report

Ratio of Offersto Total Applications

Ratio of Acceptsto Offers (Yield)

AdmissionOffered

AdmissionAccepted

Drilling Down

• Why do some students to whom we extend an offer of admission choose to attend our institution?

• Why do other students to whom we extend an offer of admission choose to attend a different school?

• How is our institution perceived by prospective students within the admissions marketplace?

• What sources of information do students draw upon in shaping those perceptions?

• What is the role of financial aid in shaping the college selection decision?

Continuing/Returning Students• Student Satisfaction Research

- ACT Survey of Student Opinions - Noel-Levitz Student Satisfaction Inventory

• ACT Survey of Student Opinions

Student use of, and satisfaction with 21 programs and services typically found at a college or university (e.g. academic advising, library, computing, residence life, food services, etc.)

Student satisfaction with 43 dimensions of campus environment (e.g., out-of-classroom availability of faculty, availability of required courses, quality of advisement information, facilities, admissions and registration procedures, etc.)

Self-estimated intellectual, personal, and social growth; Overall impressions of the college experience

NOTE: Survey is available in four-year and two-year college versions.

What About Non-Returning Student Research?

TABLE 1: ENROLLMENT, DROPOUT RATES AND GRADUATION RATESFOR FIRST-TIME FRESHMEN ON THE NEWARK CAMPUS (Total)

Enrollment and Dropout Rates Graduation Rates

Entering 1st 2nd 3rd 4th 5th 6th within within withinFall Term Fall Fall Fall Fall Fall Fall 3 yrs 4 yrs 5 yrs Total

1995 N 3154 2673 2439 2355 599 113 21 1721 2219 2344% enrollment 100.0% 84.7% 77.3% 75.3% 19.0% 3.6% 0.7% 54.6% 70.4% 74.3%% dropout 0.0% 15.3% 22.7% 24.7% 26.4% 26.1%





2000 N 3128 2738 2524 2453 496 85 24 1884 2297 --% enrollment 100.0% 87.5% 80.7% 79.2% 15.9% 2.7% 0.8% 60.2% 73.4%% dropout 0.0% 12.5% 19.3% 20.8% 23.9% 23.8%

2001 N 3358 2976 2746 2674 472 0 31 2138 -- --% enrollment 100.0% 88.6% 81.8% 80.6% 14.1% 0.0% 0.9% 63.7%% dropout 0.0% 11.4% 18.2% 19.4% 22.3% 0.0%

What About Non-Returning Student Research?Drilling Deeper…..

• Commercial instruments exist, but response rates tend to be low, and reported reasons for leaving politically correct – personal or financial reasons.

• For the last several years, we have administered the Survey of Student Opinions during the Spring term to a robust sample of students across freshman, sophomore, junior, and senior classes.

• The following Fall, the respondent pool is disaggregated into those who took the Survey and returned in the Fall, and those who took the Survey, did not return in the Fall, and did not graduate.

• Test for statistically significant differences in response patterns between the two groups.

Student Engagement

• College Student Expectations Questionnaire (CSXQ)

• College Student Experiences Questionnaire (CSEQ)

• National Survey of Student Engagement (NSSE)

Graduating Students

Pursuing Still

Number Number of Full-Time Part-Time Further Seeking

Curriculum Group in Respondents¹ Employment Employment Education Employment

Class N % % % % %

Agriculture & Natural

Resources 175 63 36.0 58.7 12.7 19.0 7.9

Arts & Sciences

Humanities 417 145 34.8 67.6 10.3 13.1 9.0

Social Sciences 904 295 32.6 71.2 4.4 14.2 8.5

Life & Health Sciences 158 63 39.9 73.0 -- 25.4 --

Physical Sciences 166 61 36.7 72.1 -- 16.4 9.8

Business & Economics 571 227 39.8 89.9 3.1 2.2 4.4

Engineering 189 103 54.5 85.4 -- 11.7 1.9

Health Sciences 390 144 36.9 72.9 2.8 21.5 2.8

Human Services,

Educ. & Public Policy 557 213 38.2 85.9 2.8 4.2 5.6

2005 University Total23,527 1,314 37.3 77.2 4.0 11.9 5.9

NOTE: Percentages may not total to 100.0 because of rounding

EMPLOYMENT AND EDUCATIONAL STATUS OFBACCALAUREATES BY CURRICULUM GROUP

CLASS OF 2005

Alumni Research

• Commercially prepared instruments exist. Decide if they meet your needs or if you have to develop your own.

• Decide early on why you are doing this research: Are you assessing the continuing relevance of the college experience? Are you cultivating prospects for the Development Office? Both? Be up front if fund raising is a component.

• Decide the which classes you need to survey; don’t go after every living alumnus unless you are a very young institution.

Non-Student Measures of Institutional Effectiveness:

Teaching Productivity, Instructional Costs,and Externally Funded Scholarship

Transparency in Assessment

www.udel.edu/ir

http://www.udel.edu/ir

What About Administrative Productivity???

10 Key Strategic Indicators forInstitutional Health

1. Revenue Structure

2. Expenditure Structure

3. Excess (Deficit) of Current Fund Revenues Over Current Fund Expenditures

4. Percentage of Freshman Applicants Accepted and Percentage of Applicants Who Enroll

5. Ratio of Full Time Students to Full Time Faculty

10 Key Strategic Indicators forInstitutional Health

6. Institutional Scholarship and Fellowship Expenditures as a Percent of Total Tuition and Fees Income

7. Tenure Status of Full Time Faculty

8. Percent of Total Full Time Equivalent Employees Who Are Faculty

9. Estimated Maintenance Backlog as a Percentage of the Replacement Value of Plant

10. Percent of Living Alumni Who Have Given At Any Time During The Past Five Years

Think of the Budget as Buckets of Money

Two Kinds of Buckets:

• Revenues

• Expenditures

Major Revenue Streams

• Tuition and Fees• Governmental Appropriations• Contracts and Grants• Endowment Income• Sales and Services of Educational Activities• Auxiliary Operations• Other

Major Expenditure Categories

• Instruction• Research• Public service• Academic Support• Student Services• Institutional Support• Operation and Maintenance of Plant• Student Aid• Mandatory Transfers• Non-Mandatory Transfers• Auxiliary Enterprises

Note: The term “Education and General Expenditures” refers to the core costs of doing business related to the institution’s mission of teaching, research, and service. It does not include non-mission related activities which typically fall under the umbrella of auxiliary enterprises.

Ratio Analysis:

Revenue Contribution Ratios

Expenditure Demand Ratios

Tuition Dependency Ratio

Net Tuition

Education and General Expenditures

Other Important Revenue Contributors

• State Appropriation

• Gifts

• Endowment Income

Expenditure Demand Ratios

Instructional Demand Ratio

Instructional Expenditures

Total Operating Income

Institutional Support Demand Ratio

Institutional Support Expenditures

Total Operating Income

Useful Resource

IPEDS Peer Analysis System

http://nces.ed.gov/ipedspas/mainScreen.asp

Institutional “Dashboards” that report on key success indicators can be an succinct means of reporting on basic measures of institutional effectiveness.

www.udel.edu/ir/UDashboard

http://www.udel.edu/ir/UDashboard

Claims of institutional effectiveness are stronger when focal institution’s measures on important measures are compared with those of peer institutions.

Choosing Peer Groupings

• Scientific – Cluster Analysis or Other Multivariate Tool

• Pragmatic - e.g. Compensation Peers

• “Whatever!” - e.g. Admissions Peers

We are extending the Dashboard concept to include key variables related to the University’s new Strategic Plan that enable us to compare our position vis-à-vis actual peers and aspirational peers.

www.udel.edu/ir/UDashboard_peers

http://www.udel.edu/ir/UDashboard_peers

Dashboard Variables Being Collected

STUDENT CHARACTERISTICS

Percent of Accepted Applicants Matriculated

Total Minority Percentage in Undergraduate Student Body

Freshman to Sophomore Retention Rate

Four Year Graduation Rate

Six Year Graduation Rate

RESEARCH ACTIVITY

Total R&D Expenditures per Full Time Faculty

Total Service Expenditures per Full Time Faculty

FINANCE

University Endowment as of June 30

Alumni Annual Giving Rate

INSTRUCTION

Full Time Students per Full Time Faculty

Total Degrees Granted

Total Doctoral Degrees Granted

Percent of Faculty With Tenure

Percent of Faculty That Are Full Time

Percent of Women Among Full Time Faculty

Percent of Minorities Among Full Time Faculty

Percent of Minorities Among Full Time Staff

Total Faculty Who Are National Academy Members

What Should Be Assessed Under

Standard 14?

Assessing Student Learning Outcomes

• I’ll provide only a brief overview, as there are others (Linda Suskie, Trudy Banta, Jeff Seybert) who are far better versed than I am.

• That said, understand that assessment of student learning is at the core of demonstrating overall institutional effectiveness.

• Assessment of student learning is a direct response to the inadequacy of student grades for describing general student learning outcomes.

According to Paul Dressel of Michigan State University (1983), Grades Are:

“ An inadequate report of an inaccurate judgment by a biased and variable judge of the extent to which a student has attained an undefined level of mastery of an unknown proportion of an indefinite material. ”

There is no “one size fits all” approach to assessment of learning across the

disciplinesNone of these should be applied to evaluation of individual student performance for purposes of grading and completion/graduation status.

1. Standardized Tests• General Education or Discipline Specific• State, Regional, or National Licensure Exams

2. Locally Produced Tests/Items• “Stand Alone” or Imbedded

3. Portfolios/Student Artifacts• Collections of Students’ Work• Can Be Time Consuming, Labor Intensive, and Expensive

4. Final Projects• Demonstrate Mastery of Discipline and/or General Education

5. Capstone Experiences/Courses• Entire Course, Portion of a Course, or a Related Experience

(Internship, Work Placement, etc.)

Tying It All Together……

Helpful Hints for Successfully Integrating Planning, Assessment, and Other Processes

• Start from institutional Mission: The Mission Statement should define the institution and its core values, communicate what business(es) it will engage in, and convey a sense of its unique character.

• Understand the Market and create a “Brand”: Market and branding decisions are important in today’s competitive environment, and should naturally flow from the mission. Avoid rushing into new markets without careful research/planning.

• Long Range and Strategic Planning: these types of plans should set the stage, articulating directions chosen to help the institution achieve its mission. Focus here is on what to do.


• Tactical and operational Planning: These plans have a shorter time horizon and focus on how to achieve the goals and purposes that have been set.

• Integrate Academic, Resource, and Facilities Elements: Academic decisions should be the driving force, but good planning is holistic in approach.

• Set Priorities and Establish a Timetable: Decisions about priorities and timelines should recognize the “art of the possible” for a unique institutional environment.


• Plans as Guidelines, not Monuments: The process is much more important than the plan itself. Change is constant.

• Identify and Implement Appropriate Linkages: Results from assessments should flow to budget decisions and into new rounds of planning. Department/division work should link to institutional work.

• Leadership is Imperative: none of this will happen without direct support of campus executives.


• This is About Change Management: Respect the institution’s culture and history; be realistic in expectations; work toward the sustainable.

• Encourage Broad Participation and Wide Communication: These characteristics provide a solid foundation for a collegial process, and improve prospects for a successful implementation.

• Student Learning Outcomes Assessments and assessment of Institutional Effectiveness: Both types of assessment are important in determining if the institution is achieving its goals.


• Develop a Capacity for Institutional Research: Internal research and external benchmarking is essential in assessment, but the function can be institutionally organized in many ways.

• Judge Progress and Re-Calibrate the Plan: Consider how the institution is doing in reaching its goals and purposes, and whether processes are all they should be.

• Do Not Re-Invent the Wheel: Use external resources such as the Society for College and University Planning and other topical higher education organizations and data bases.

Documents

Integrating Higher Education Planning and Assessment: Concepts and Strategies Michael F. Middaugh Assistant Vice President for Institutional Research and