Qual QuantDOI 10.1007/s11135-012-9706-3

Developing measures for performance excellence:is the Baldrige criteria sufficient for performanceexcellence in higher education?

Muhammad Asif · Abdul Raouf · Cory Searcy

Received: 12 January 2012 / Accepted: 10 April 2012© Springer Science+Business Media B.V. 2012

Abstract The implementation of the education criteria for performance excellence(ECPE)—the education version of the Baldrige criteria—is driven by the growing needfor performance excellence in higher education. Ironically the criteria meant for perfor-mance excellence have no theoretical underpinning and are based on normative performanceimprovement requirements. The lack of content and internal validity is reflected in somecritical weaknesses in the criteria. Based on a critical review, this paper highlights potentialimprovements in the areas of aligning processes with the organisational mission, operationsfocus, customer focus, workforce focus, and performance measurement. To help enhancethe ECPE, the paper develops performance measures for systematic implementation of thecriteria. This is discussed in terms of defining desired outcomes, developing key measures,identifying specific indicators to track performance, and choosing methods to assess perfor-mance. The discussions provide a basis for future revisions to the ECPE.

Keywords Baldrige criteria · Education criteria for performance excellence (ECPE) ·Performance excellence · Higher education · Performance measurement

M. Asif (B)Prince Sultan University, KSA, Riyadh, Saudi Arabiae-mail: Muhammad.assif@gmail.com

A. RaoufUniversity of Management and Technology, Lahore, Pakistane-mail: abdulraouf@umt.edu.pk

C. SearcyRyerson University, Toronto, ON, Canadae-mail: cory.searcy@ryerson.ca

123

M. Asif et al.

1 Introduction

The Baldrige criteria for performance excellence (BCPE) consists of three versions: busi-ness, healthcare, and education. The education criteria for performance excellence (ECPE)are used by educational institutes worldwide to impact bottom-line improvements in per-formance including operational, financial, and market outcomes. The criteria require anorganisation to assess its improvement efforts, diagnose its overall performance manage-ment systems and identify strengths and opportunities for improvement (NIST 2010). TheBaldrige criteria is considered a comprehensive means for achieving performance excellencein education. Since the criteria share some common elements with TQM (Alonso-Almeidaand Fuentes-Frías 2012; Seetharaman et al. 2006), it is also considered a means for TQMimplementation (Bou-Llusar et al. 2009; Black and Porter 1996; Dean and Bowen 1994;Flynn and Saladin 2006).

This paper challenges the basic structure of the ECPE and identifies a number of areas forimprovement in the ECPE. The main focus of the ECPE is on describing processes that arenecessary for achieving excellence. In doing so, the ECPE provides organisations flexibilityin terms of achieving the desired targets. However, the ECPE does not guide the systematicimplementation and performance measurement of the criteria. This paper has two main con-tributions. First, it identifies a number of areas for improvement that should be addressedin future revisions of the ECPE. Second, the paper develops performance measures for keyaspects of the ECPE. The purpose is to elaborate the process of development of such measuresso that measures could be developed for the rest of the processes in the ECPE.

The rest of the paper is structured as follows. The next section provides an overview ofthe structure and contents of the ECPE. This is followed by a critical review of the criteriaand areas for improvements are identified. The paper then discusses the need to developperformance measures. This is further explained through examples of academic processes.The paper ends with conclusions.

2 ECPE

2.1 The basic structure of the ECPE

The ECPE claims to interpret generic principles of performance excellence in the particularcontext of education (ECPE 2011; Houston 2008). The criteria consist of 11 core valueswhich are embedded in seven categories. Six of these categories are labelled as ‘systematicprocesses’, which include ‘leadership’, ‘strategic planning’, ‘operations focus’, ‘workforcefocus’, ‘customer focus’, and ‘measurement, analysis and knowledge management’. A focuson these systematic processes yields performance results, including ‘leadership and gover-nance outcomes’, ‘budgeting’, ‘financial and marketing outcomes’, ‘workforce-focused out-comes’, ‘customer-focused outcomes’, and ‘student learning and process outcomes’ (Fig. 1).

Unlike many of the standardised management systems—such as ISO 9001 and ISO 14001which are prescriptive in nature—the ECPE is descriptive in nature. This means that the ECPEdoes not pose compulsory requirements for organisations but, rather, provides flexibility inachieving the desired ends. This approach is advantageous because it allows organisationsflexibility in achieving desired targets and organisations can design their processes accordingto their own unique context. However, this approach also leaves practitioners with limitedspecific guidance in performance improvement, particularly with respect to questions suchas how to achieve targets and what techniques and methods to be employed.

123

Developing measures for performance excellence

Core values • Visionary leadership • Focus on the future • Managing for

innovation • Agility • Organizational and

personal learning • Valuing workforce

members and partners • Student-centered

excellence • Societal responsibility • Management by fact • Focus on results and

creating value • Systems perspective

Systematic Processes 1.Leadership2.Strategic planning3.Customer focus 4.Measurement, analysis,

and knowledge management

5.Workforce focus 6.Operations focus

7. Performance Results Student learning and process outcomes Customer-Focused outcomes Workforce-Focused outcomes Leadership and governance outcomesBudgeting, financial and market outcomes

Embedded in Yielding

Fig. 1 The basic structure of the ECPE—adapted from the ECPE (2011)

2.2 A snapshot of the literature on the ECPE

The underlying theoretical framework of the Baldrige Criteria is of critical importance, sinceit conveys a message about the route to competitiveness (Flynn and Saladin 2006). However,the ECPE has no theoretical underpinning and is based on normative practices and processimprovement ideas of quality experts and practitioners. While the research on the validationof BCPE—the business version of the criteria—has expanded in the last few years; this hasnot been the case with the ECPE. As Badri et al. (2006, p. 1119) noted, “there are only a fewstudies that fully address Baldrige in the area of education”.

The existing literature can be classified into two categories. The first type of literatureexplores the impact of the Baldrige criteria on the performance of educational institutes.Some representative examples in this type include Belohlav et al. (2004); Evans (1997), andWeinstein et al. (1998). Belohlav et al. (2004), for instance, found that implementation ofthe Baldrige criteria led to a higher level of student engagement in the learning process,as evidenced by more abundant and higher-quality feedback to the instructors. The overallmessage of the first type of research is that implementation of the Baldrige criteria leads toperformance improvements. The second type of research focuses on the contents/categoriesand validity of the criteria. However, while there is ample research on the validity of the busi-ness version of the criteria (i.e., BCPE), little is known about the validity of the ECPE. Thefew exceptions include Badri et al. (2006) and Winn and Cameron (1998). Badri et al. (2006)found that all categories of the criteria are significantly linked with organizational outcomesin terms of organizational performance results, and student, stakeholder and market focus.While that study focused on determining the relationship between the criteria constructs andperformance, Winn and Cameron (1998) examined the validity of the proposed relationshipsamong the Baldrige dimensions using data from higher education. The findings suggestedthat modifications to the criteria were required and that questions remain as to whether theECPE constitutes a valid model. This concern is also raised by a number of authors. Forinstance, Badri et al. (2006, p. 1119) noted that “while developing a curriculum based uponBaldrige principles has received noteworthy attention, what is not readily evident within theliterature is the actual application of the MBNQA concepts as part of the educational deliv-

123

M. Asif et al.

Table 1 Key areas needing improvement in the ECPE

Key areas needingimprovement inthe criteria

Explanation

Alignment of theprocesses withthe mission

The criteria provide an overarching emphasis on the alignment of organisa-tional processes with the mission of the HEI. The underlying assumptionis that organisations are always able to develop a well-formulated mission.Inability to develop an appropriate mission—due to incomplete under-standing of the environment, stakeholders and academic excellence—could result in inefficient processes.

Operations focus The supply chain in higher education is fundamentally different from thetraditional supply chain in manufacturing. The conventional concepts ofsupply chain management discussed in the ECPE do not apply readily inhigher education. Further ‘supplier selection’, ‘supplier performance eval-uation’, and ‘dealing with poorly performing suppliers’, as discussed inthe ECPE, reflect a traditional manufacturing approach to supplier man-agement.

Customer focus The customer–supplier relationship in higher education is complex and fun-damentally different from other sectors of the economy. Relying on theinput of only one type of stakeholder—such as students—could risk theinterests of other stakeholders and the overall integrity of the system.Management needs to balance the requirements of various stakeholdersto design best-in-class academic programs. The ECPE does not addressthis aspect.

Workforce focus This category is too generic and does not discuss workforce managementfrom an academic perspective.

Performance measurement The ECPE has two main weaknesses regarding performance measurement:first, performance measures currently mentioned in the criteria—such as‘productivity’, ‘cycle time’, and ‘efficiency’—are rather vague and theirapplication in an academic setting needs to be discussed further. Second,the ECPE does not discuss systematic performance measurement. The cri-teria needs to have a solid approach towards performance measurement.

ery process”. On a similar note Arif and Smiley (2004) noted that while the Baldrige Awardin education has captured the attention of decision makers, there has been little empiricalresearch examining the usefulness of the award criteria to guide the actions of organizationsthat seek to improve performance.

The gap in existing research is clear. The research needs to advance beyond the ‘rela-tionships between the ECPE’s elements and performance’ to question the existence and(in)adequacy of the existing elements. As the well-known axiom says ‘you cannot solve aproblem at the same level of consciousness at which it was created’. Future research shouldestablish the validity of the criteria. As a first step of the needed research, this paper providesa critical analysis of the ECPE. The next section discusses some of the weaknesses of thecriteria.

3 Weaknesses of the ECPE

The weaknesses of the ECPE are summarised in Table 1. A detailed discussion of the weak-nesses highlighted is provided in the sections below.

123

Developing measures for performance excellence

3.1 Alignment of organisational processes with the mission

As a fundamental starting point, the ECPE requires educational institutes to develop a missionand then align organisational processes with that mission. The criteria requires:

What are your stated purpose, vision, values, and mission? What are your organiza-tion’s core competencies and their relationship to your mission?...What are the keyelements that engage them [workforce] in accomplishing your mission and vision?(ECPE 2011, p. 4)

How do you select, collect, align, and integrate data and information for tracking dailyoperations and overall organizational performance, including progress relative to stra-tegic objectives and action plans? (ECPE 2011, p. 16)

The criteria allow educational institutes considerable flexibility in their approach to per-formance excellence. Performance excellence, however, is not always about alignment oforganisational processes with the mission. If an organisation fails to develop the right mis-sion, the alignment of the processes with the mission would be of little use. Performanceexcellence is also about using the existing knowledge of process management and total qualitymanagement to achieve excellence. For instance, the mission of the institute may require theorganisation to improve satisfaction of stakeholders. The mission, however, does not guidethe institute on how to achieve customer satisfaction. Simply designing academic processesto align with the mission without using the existing knowledge of quality management maynot be very fruitful. An example is of program design. The mission of the organisation mayrequire developing an effective syllabus, curriculum, or a program. The existing knowledgeof performance management shows that quality improvement methods, such as quality func-tion deployment (QFD), can be used to systemize program design and to facilitate inclusionof the voice of the customer. However, the criteria do not draw on the existing knowledgebase. This is despite the fact that applying tools such as QFD in an education context findswide support in the literature (see, for example, Aytac and Deniz 2005, Lo Franco and LaRosa 2012, Martins and Aspinwall 2001, and Sirvanci 2004).

To summarise, future revisions of the criteria should focus on two key issues. First, highereducation institutes (HEIs) should carefully scan the broader external and internal environ-ment and understand their stakeholders and meanings and means of academic excellencebefore formulating the mission. This is because only a properly formulated mission andsubsequent alignment of organisational processes with the mission is likely to give rise to per-formance excellence. In other words, the ECPE should go beyond mere ’alignment’. Second,the ECPE should place an explicit focus on using the advanced knowledge of performancemanagement during program design and the development of other processes. ‘Program designaligned with the mission’ is not sufficient on its own for developing best-in-class programsand, thus, performance excellence.

3.2 Operations focus

This category in the ECPE consists of two clauses: (1) “Work systems: how do you design,manage, and improve your work systems?” and (2) “Work processes: how do you design,manage, and improve your key work processes?” The criteria requires:

Supply-Chain Management: How do you manage your supply chain? How do youensure that suppliers you select are qualified and positioned to enhance your per-

123

M. Asif et al.

Table 2 Customer–supplier relationship in higher education-based on Asif and Raouf (2012) and Raouf(2004)

Customer Supplier Services by supplier

Student Faculty Curriculum design, teaching,professional and intellectualdevelopment

Administrators Support services, material andequipment, provision of requiredresources

Students Learning in group and team work

Faculty members Faculty members Learning from peers and seniorfaculty members

Students Feedback from students aboutteaching methods, courseevaluation, teacher evaluation, andthe overall program design

Industry andparents ofstudents

Higher Educationinstitutes

Knowledge, wisdom, know-how,know-what, know-why,professional and intellectualdevelopment

Higher Education Institutes Funders Financial resources

Community License to operate, provision ofrequired resources

Professional and accreditation bodies Regular program assessment,provision of professionalrequirements and guidelines

formance and student and stakeholder satisfaction? How do you evaluate supplierperformance? How do you deal with poorly performing suppliers? (ECPE 2011,p. 22)

This category is too generic and does not discuss operations from an academic perspective.The supply chain concept mentioned in the ECPE needs to be explained in terms of curricu-lum development, program design, program contents, delivery methods, student assessment,and faculty management. The ECPE, however, portrays a picture of traditional supply chainby requiring how do you ensure that your suppliers are qualified?; how do you evaluatetheir performance?; and how do you deal with poor performance of suppliers?. This typeof customer–supplier relationship works well in a business setting where two parties aredependent on each other for financial gains. However, in academics, conventional supplierssuch as the government, private funders, and the community work independently and manytimes voluntarily provide resources to the educational institutes. The customer-institute mayhave little or no influence over the mentioned suppliers. This clearly highlights that the con-ventional supply chain terms of business parlance need to be used carefully in academics dueto the unique nature of its processes. Such conventional supply chain concepts can be used,at their best, in non-academic functions of the educational institutes such as administrationand support functions. However, such concepts are difficult to apply in academics which isthe core area of any educational institute.

Further, the traditional concepts of ‘customer’, and ‘supplier’ are also different in edu-cation. This is shown in Table 2, which is based on Asif and Raouf (2012), Raouf (2004),and Tribus (1995). The Table makes it clear that different stakeholders may assume mul-tiple roles at a given time. ‘Students’, ‘faculty’ and ‘HEIs’ assume the role of both cus-

123

Developing measures for performance excellence

tomer and supplier. As an example, consider the ‘student’ in a role of supplier of knowl-edge for other students and faculty. The ECPE requirements ‘ensuring that suppliers arequalified’, ‘evaluating their performance’, and ‘dealing with poor performance’ are a misfitin an academic setting. This category, thus, needs to be revised in future versions of thecriteria.

3.3 Customer focus

This category of the ECPE examines how an organisation engages with students and otherstakeholders for long term market success. An essential outcome of ‘customer focus’ is theincorporation of the voice of the customer and other stakeholders into program design. Thishelps bring legitimacy. The stakeholders in an academic program could include students, fac-ulty, HEIs, government, regulatory agencies, professional and accreditation bodies, industryor future employers, funders, community, etc.

Some stakeholders may have requirements not aligned with the requirements of otherstakeholders. One example is the dual role of students as ‘customers of higher education’and ‘grade seekers’ (Meirovich and Romar 2006; Asif and Raouf 2012). Further students maybe unaware of the knowledge and skills required by their future employers. The problem ofconflicting stakeholder-requirements is discussed in the literature by Asif and Raouf (2012),Chen (2010), Cruickshank (2003), Meirovich and Romar (2006), and Sirvanci (2004). Sucha scenario highlights the need for wider stakeholder consultation and balancing the stake-holder requirements for optimal results. Omitting the latter step could risk the incorporationof only powerful entities and ignoring less powerful stakeholders. This, in turn, could distortthe whole purpose of stakeholder consultation. The ECPE, however, is mute on this criticalaspect. The ECPE requires:

Listening to Current Students and Stakeholders: How do you listen to students andstakeholders to obtain actionable information? How do your listening methods varyfor different student groups, stakeholder groups, or market segments? How do youuse social media and Web-based technologies to listen to students and stakeholders,as appropriate? How do your listening methods vary across the stages of students’ andstakeholders’ relationships with you?

Given that defining the customer in higher education has been problematic in the past(Sirvanci 2004; Meirovich and Romar 2006) and HEIs have struggled to balance stake-holder-requirements (Houston 2007), this is a critical issue. Some authors have discussedthis issue from the perspective of the systems approach (Houston 2007; Galbraith 1999).In any case, this critical aspect needs to be discussed in the future revisions of thecriteria.

3.4 Workforce focus

This category in the ECPE consists of two clauses: (1) Workforce environment: how do youbuild an effective and supportive workforce environment and (2) Workforce engagement: howdo you engage your workforce to achieve organisational and personal success? Academicprocesses are the core processes in a HEI (cf. administrative processes meant to supportthe core processes). The workforce for academic processes mainly consists of faculty andresearchers. The faculty members, in an academic setting, are usually required to do multipletasks including research, teaching, student counselling, administrative tasks, services to pro-

123

M. Asif et al.

fession, and services to community, etc. To achieve high performance, the workload of thefaculty needs to be balanced for various activities. This would also require commensuratedevelopment of supporting structures such as setting criteria for workload allocation, assess-ment, and rewards systems. However, the ‘workforce’ category of the ECPE is too genericand the mentioned issues are not addressed in the criteria.

3.5 Performance measures

The ‘results’ category of the ECPE requires determining operational process effectiveness inaddition to other measurements. The operational process effectiveness could be determinedin terms of program design, admissions, faculty management, learning and development,etc. The ECPE, however, does not explicitly address the mentioned issues. Rather, it makesvague linkages to ‘productivity’, ‘cycle time’, and ‘other measures of process effectiveness,efficiency, and innovation’. The criteria states:

Operational effectiveness: what are your current level and trends in key measures ofindicator of the operational performance of your key work systems and processes,including productivity, cycle time, and other appropriate measures of process effec-tiveness, efficiency, and innovation?(ECPE 2011, p. 23)

The concepts of productivity, cycle time, and efficiency are more common in manufac-turing which is characterised by tangible inputs and outputs. While it is easy to measureproductivity and efficiency and their various types (Krajewski et al. 1999; Slack et al. 2010)in the manufacturing sector, these concepts do not readily apply in education because inputand output are often intangible and it is difficult to make measurements and comparisons. Thefuture revisions of the criteria need to eliminate vague measures and offer in-depth insightsinto performance measurement.

On a similar note, the ‘measurement, analysis, and knowledge management’ category ofthe ECPE, requires organisations to use performance measures for various organisationalprocesses. The use of performance measures is in line with one of the core values of theECPE, i.e., fact-based decision making to align organisational direction and resources withkey processes. The ECPE requires:

Performance Measures: […] What are your key organizational performance measures,including key short-term and longer-term budgetary and financial measures? How fre-quently do you track these measures? How do you use these data and information tosupport organizational decision making and innovation? (ECPE 2011, p. 16)

The general approach of the ECPE is that it requires academic institutes to use perfor-mance measures. However, it does not explain how to develop such performance measures.In other words it discusses ‘what to do’ but not ‘how to do it’. This leaves practitioners withdifficulty in implementation. The question that remains unanswered is how to systematicallydevelop performance measures for various academic processes such as admission, facultyrecruitment and development, teaching, and learning and development.

To summarise, currently there are two issues with ‘performance measurement’ in theECPE. First is the existence of vague performance measurement metrics including produc-tivity, cycle time, and efficiency; and second is the lack of clear guidance on developmentof performance measures. Without the latter, performance measurement, improvement, andexcellence will remain elusive.

123

Developing measures for performance excellence

Fig. 2 Systematic approach todevelopment of performancemeasures

Desired outcomes

Key measures

Indicators

Methods

to be measured in terms of

through development of specific

using

4 Developing performance measures for academic processes

As noted earlier, the ECPE is based on normative performance management practices, andin doing so, it requires the use of performance measures for various processes. Performanceexcellence, however, requires a more systematic approach characterised by upfront planningof clear outcomes, developing measures and indicators for such outcomes, and methods totrack performance (Veleva and Ellenbecker 2001; Chen 2012). This systematic approach isshown in Fig. 2. The Figure shows that systematic performance measurement requires devel-opment of key measures. The ‘key measures’ need to be assessed through the developmentof specific indicators and appropriate methods (Ball and Wilkinson 1994). While the keymeasures are generic and represent a higher level of abstraction; indicators are more specific,measurable and represent the operational level. For example, a HEI interested in entrepre-neurial leadership would measure the performance of its initiatives in terms of new businessventures by its graduates. ‘New business ventures’ is, thus, the key measure for entrepreneur-ial leadership. The ‘key measure’ must be measured through development of more specificindicators (Veleva and Ellenbecker 2001), which in this case could be ‘number of graduatesstarting new business’. Finally, this measurement needs to be carried out using appropriatemethods. This focuses on assembling the required supporting information through ‘data col-lection’. The data collection could be carried out using different methods including surveysof graduates or developing a database of graduates initiating new businesses. To summarise,while ‘entrepreneurial leadership’ is the intended outcome; ‘new business ventures’ is a pos-sible key measure; ‘number of graduates starting new businesses’ is a possible indicator; andthe ‘use of surveys or database’ to get relevant data is a potential method. The development ofkey measures, indicators, and methods makes performance measurement more systematic,helps in making policy decisions, and better allocation of resources (Kells 1992). The ECPE,however, does not address such issues.

While the use of indicators to measure performance is common (Kells 1992); the use of astructured approach to develop performance indicators aligned with the mission of the insti-tute requires further attention by HEIs (Ball and Wilkinson 1994). Processes in an academicsetting would undoubtedly vary from one setting to another. However, there are several gen-eral processes that are common to any academic setting. These processes include: developinga mission and vision for the institute, program design, admissions, faculty recruitment anddevelopment, learning and development, support services, and financial aspects. These pro-

123

M. Asif et al.

Table 3 The key processes in an academic setting

Area Description Reason for inclusion

Mission andvisiondevelopment

Setting the direction and defining thefocus of the HEI.

A strategic essential that definesdesired outcomes and determinesallocation of resources, direction ofefforts, and execution of processes.HEIs could lose their focus anddirection without setting a visionand mission. Mission and visiondevelopment is also a requirementof the ECPE.

Program design A structured approach to definecontent, delivery methods, andassessment to achieve desiredlearning outcomes.

Academic programs are the mainproduct of HEIs. A focus on thiskey area is required forperformance excellence andsatisfaction of a broad range ofstakeholders.

Admissions Admissions provide input to HEIs inthe form of students for thedeveloped programs.

‘Admissions’ is a critical process thataffects the rest of the processes anddetermines the performanceoutcomes of the HEI. Like anyperformance improvementprogram, HEIs need to control thiscritical process.

Facultyrecruitment anddevelopment

How the faculty is recruited,developed, and managed in theoverall academic processes toachieve performance excellence.

The ‘faculty’ is an important type ofstakeholder who have a critical rolein any academic program includingprogram design, delivery,assessment, research, and otherfunctions of the HEI. Effectivemanagement of the faculty entailsbetter performance.

Learning and development How the HEI ensures learning anddevelopment of students andfaculty, and develops it fororganisational excellence.

Being the seats of learning, HEIsbear the responsibility to ensurelearning and development ofstudents and faculty. Organisationscan capitalise on this feature toachieve performance excellence.

Support services Such services include security,medical, maintenance, housing,sports, and logistics.

Since the support services are meantto support the main processes, theyneed to be designed and managedto support academic excellence.

Financial aspects Financial aspects concern the overallmanagement of the financialportfolio of an organisation.

Financial viability is an importantcontributor to performanceexcellence. Financial outcomes isalso a construct of the ‘performanceresults’ category of the ECPE.

cesses are central to any academic setting and also represent the key processes at strategic,tactical, and operational levels. A description of the processes and a summary of the rationalefor their inclusion is provided in Table 3.

Performance excellence hinges on performance measurement and subsequent improve-ment. As the well-known axiom says, ‘what gets measured, gets managed’. However, trackingthe overall performance of the HEI requires a structured approach (Ligarski 2009) character-ised by the development of suitable measures, indicators, and methods (Ball and Wilkinson

123

Developing measures for performance excellence

1994). A structured approach to performance measurement, thus, lies at the core of perfor-mance excellence. The ECPE does not provide any guidance on this critical area of perfor-mance excellence. Addressing this area in future versions of the criteria could make perfor-mance excellence more systematic. Through a number of representative examples, Table 4illustrates the development of performance measures for the above mentioned academic pro-cesses. The performance measures are discussed in terms of desired outcomes, key measures,indicators, and methods.

One example of a structured approach to performance measurement discussed in Table 4 isdevelopment of the vision and mission of the HEI. A HEI may have a mission which empha-sizes achievement of world-class teaching, entrepreneurial leadership, innovative researchthat impacts social and technological change, active role in the development of society, or adesire to develop a stimulating campus, or a multi-cultural university. A structured approachto the realisation of these outcomes consists of developing key measures for each outcome.The key measures for world-class teaching could include quality of teaching, student suc-cess, program completion, and student’s knowledge, skills and values. However, these keymeasures are not sufficient on their own and organisations need to develop more specificindicators to ensure the realisation of the mentioned outcomes. The indicators in this casecould include number of students completing the program, dropout rate, median score ofstudents, % of students with a particular GPA, satisfaction rate of students, graduates, andalumni for academic and support activities, course rating, and graduate’s employment rate.The measurement of these indicators could be carried out through data collection, surveys,and formal and informal feedback from the relevant stakeholders (Ball and Wilkinson 1994).Reliability and validity must be key considerations throughout the process. In particular,care must be taken to ensure that the indicators measure what they are intended to measure.Further examples are provided in Table 4.

Performance indicators

Assessment

Key measures

Benchmarking

Carried out by independent

party

Carried out by process owner

Comparison against industry best-practices

Self-assessment Audit

Hybrid approach to performance assessment through combined use of audits, self-assessment and benchmarking

The areas missing in the ECPE

Fig. 3 The modes of assessment approaches to track performance along key measures and indicators

123

M. Asif et al.

Tabl

e4

Perf

orm

ance

mea

sure

sfo

rva

riou

sac

adem

icpr

oces

ses

Proc

ess

sdohtem laitnetoP

srotacidnI serusae

m yeK

semoctu

O

Prog

ram

com

plet

ion

rate

•

Num

ber o

f stu

dent

s com

plet

ing

the

prog

ram

•

Dro

pout

rate

•

Dat

a ab

out n

umbe

r of p

asse

d an

d dr

oppe

d st

uden

ts

Stud

ent s

ucce

ss ra

te

•M

edia

n sc

ore

of st

uden

ts

•%

of s

tude

nts w

ith a

par

ticul

ar G

PA

•St

uden

ts’ G

PA d

ata

colle

ctio

n

•Su

rvey

invo

lvin

g st

uden

ts, a

lum

ni, a

nd e

mpl

oyer

s Q

ualit

y of

teac

hing

•

Satis

fact

ion

rate

of s

tude

nts,

grad

uate

s, an

d al

umni

for

acad

emic

and

supp

ort s

ervi

ces

•C

ours

e ra

ting

– m

edia

n ev

alua

tion

of th

e co

urse

by

stud

ents

•

Cou

rse

feed

back

from

stud

ents

Wor

ld-c

lass

teac

hing

Stud

ents

’ and

gra

duat

es’

know

ledg

e, sk

ills,

and

valu

es

•St

uden

ts’ m

edia

n sc

ore

•G

radu

ates

em

ploy

men

t rat

e

•St

uden

ts’ G

PA d

ata

colle

ctio

n

•Su

rvey

of g

radu

ates

em

ploy

men

t rat

e •

Surv

ey o

f gra

duat

es e

mpl

oyed

at h

igh

posi

tions

En

trepr

eneu

rial

lead

ersh

ip

New

bus

ines

s ven

ture

s •

Num

ber o

f gra

duat

es st

artin

g ne

w b

usin

esse

s •

Surv

ey /

data

base

of g

radu

ates

initi

atin

g ne

w

busi

ness

es

Pate

nts

•N

umbe

r of p

aten

ts

•N

umbe

r of s

pin-

offs

•

Num

ber o

f pat

ents

add

ress

ing

loca

l nee

ds

Inno

vativ

e re

sear

ch

that

impa

cts s

ocia

l, ec

onom

ic a

nd

tech

nolo

gica

l cha

nge

Idea

s and

mod

els o

f soc

ial,

econ

omic

, and

tech

nolo

gica

l re

leva

nce

•N

umbe

r of j

ourn

al p

aper

s add

ress

ing

loca

l nee

ds

•N

umbe

r of r

esea

rch

proj

ects

•

Num

ber o

f tec

hnol

ogy

proj

ects

•

Num

ber o

f res

earc

h pr

ojec

ts a

ddre

ssin

g lo

cal n

eeds

•D

ata

colle

ctio

n ab

out e

ach

indi

cato

r suc

h as

dat

a on

nu

mbe

r of s

pin-

offs

, pat

ents

, jou

rnal

pap

ers,

rese

arch

pr

ojec

ts.

•Ex

pert

eval

uatio

n of

pat

ents

, res

earc

h pr

ojec

ts, a

nd

jour

nal p

aper

s abo

ut th

eir r

elev

ance

to th

e so

ciet

al

need

s

Stud

ents

’ sat

isfa

ctio

n fo

r ca

mpu

s lay

out a

nd o

utlo

ok

•Sa

tisfa

ctio

n ra

te o

f stu

dent

s for

des

ign,

layo

ut, a

nd v

isua

l ap

peal

of t

he c

ampu

s •

Cam

pus d

esig

n-fo

r-pu

rpos

e

•N

umbe

r of s

ocia

l act

iviti

es p

er y

ear

Ath

letic

eve

nts

•N

umbe

r of a

thle

tic e

vent

s per

yea

r •

Ove

rall

spor

ts p

rofil

e of

the

inst

itute

Stud

ent a

ctiv

ities

•

Num

ber o

f stu

dent

act

iviti

es p

er y

ear

•%

of s

tude

nt m

embe

rs o

f var

ious

clu

bs p

er y

ear

Stim

ulat

ing

cam

pus

Even

ts in

volv

ing

loca

l co

mm

unity

•

Num

ber o

f eve

nts i

nvol

ving

loca

l com

mun

ity

•D

ata

colle

ctio

n fo

r eac

h ac

tivity

•

Stud

ents

and

oth

er st

akeh

olde

rs’ f

eedb

ack

thro

ugh

form

al a

nd in

form

al c

onve

rsat

ions

•

Rep

orts

from

new

spap

ers,

mag

azin

es, a

nd o

ther

in

depe

nden

t sou

rces

.

Mis

sion

and

vi

sion

deve

lopm

ent

Mul

ticul

tura

lun

iver

sity

N

atio

nalit

ies o

f stu

dent

s, pa

st g

radu

ates

, and

facu

lty

•N

umbe

r of n

atio

nalit

ies o

f cur

rent

and

pas

t stu

dent

s •

Num

ber o

f nat

iona

litie

s of f

acul

ty

•D

ata

colle

ctio

n fo

r nat

iona

litie

s of s

tude

nts a

nd fa

culty

123

Developing measures for performance excellence

Tabl

e4

cont

inue

d

Proc

ess

sdoh temla itnet oP

srotacidnIserusae

mye

Kse

moctuO

Aca

dem

icpr

oces

ses

Com

para

bilit

y w

ith th

e be

st-

clas

s pro

gram

s off

ered

in

mar

ket

•N

umbe

r of p

rogr

ams,

cour

ses,

and

cour

se c

onte

nts c

ompa

rabl

e w

ith

the

best

-cla

ss p

rogr

ams i

n th

e m

arke

t •

Dat

a co

llect

ion

abou

t rel

evan

t ind

icat

ors

Mee

ting

dem

ands

of t

he

indu

stry

, pro

fess

iona

l bo

dies

, and

oth

er

stak

ehol

ders

•A

ppro

val o

f pro

gram

from

com

pete

nt a

utho

ritie

s – su

ch a

s pr

ofes

sion

al b

odie

s (ye

s/no

) •

Tota

l num

ber o

f app

rove

d pr

ogra

ms o

ffer

ed b

y th

e in

stitu

te

•St

akeh

olde

rs’ s

atis

fact

ion

for e

xist

ing

prog

ram

s •

Empl

oyer

satis

fact

ion

for a

cade

mic

pro

gram

s and

gra

duat

es

•D

ata

colle

ctio

n of

num

ber o

f app

rove

d pr

ogra

ms

•B

ench

mar

king

aga

inst

bes

t per

form

ers

•Fe

edba

ck fr

om st

akeh

olde

rs a

bout

rele

vant

in

dica

tors

•R

epor

ts fr

om n

ewsp

aper

s, m

agaz

ines

, and

ot

her i

ndep

ende

nt so

urce

s.

Prog

ram

des

ign

Bes

t-cla

ss a

cade

mic

pr

ogra

m

Prof

essi

onal

de

velo

pmen

t and

de

velo

pmen

t of c

ritic

al

thin

king

and

ana

lytic

al sk

ills

•St

uden

ts m

edia

n sc

ore

•G

radu

ates

em

ploy

men

t rat

e •

Tim

e to

firs

t em

ploy

men

t afte

r gra

duat

ion

•D

ata

colle

ctio

n ab

out s

tude

nts m

edia

n sc

ore

•Fo

rmal

and

info

rmal

feed

back

from

stud

ents

, al

umni

, em

ploy

er, a

nd o

ther

stak

ehol

ders

•

Ben

chm

arki

ng a

gain

st b

est p

erfo

rmer

s

•M

edia

n hi

gh sc

hool

/ co

llege

gra

des /

per

cent

age

•En

tranc

e ex

ams

•H

igh

scho

ol re

sults

•

Med

ian

qual

ifyin

g te

sts s

core

(suc

h as

GM

AT,

GR

E, T

OEF

L, a

nd

IELT

S)•

Qua

lific

atio

n te

st re

sults

Q

ualit

y of

inco

min

g st

uden

ts

•Pr

ep-y

ear p

rogr

am c

ompl

etio

n ra

te

•Pr

ep-y

ear r

esul

ts

Adm

issio

n A

mpl

e en

rolm

ent o

f hi

gh q

ualit

y st

uden

ts

Num

ber o

f enr

olm

ents

•

Num

ber o

f stu

dent

s enr

olle

d •

Dat

aof

stud

ents

’enr

olm

ent

Teac

hing

per

form

ance

•

Perc

enta

ge o

f fac

ulty

with

exc

elle

nt te

achi

ng ra

ting

•D

ata

colle

ctio

n ab

out a

vera

ge c

ours

e ev

alua

tion

and

teac

hers

eva

luat

ion

Facu

ltym

anag

emen

tC

ompe

tent

facu

lty

Res

earc

h pr

ofile

•N

umbe

r of p

ublic

atio

ns

•N

umbe

r of s

ecur

ed a

cade

mic

gra

nts

•%

of f

acul

ty w

inni

ng a

cade

mic

gra

nts

•N

umbe

r of r

esea

rch

proj

ects

secu

red

/ com

plet

ed

•Pe

rcen

tage

of f

acul

ty o

n ed

itoria

l boa

rd /

revi

ewer

s pan

els o

f jou

rnal

s

•D

ata

abou

t res

earc

h pr

ojec

t, pu

blic

atio

ns,

and

pres

ence

on

the

edito

rial b

oard

s of

jour

nals

Scho

larly

act

iviti

es

incl

udin

g co

nfer

ence

s, se

min

ars,

and

rese

arch

pr

ojec

ts

•%

of f

acul

ty a

ttend

ing

conf

eren

ces a

nd se

min

ars

•N

umbe

r of c

onfe

renc

es, s

emin

ars,

wor

ksho

ps,

gran

ts, e

xcha

nge

prog

ram

s per

yea

r •

Num

ber o

f on-

goin

g / c

ompl

eted

rese

arch

pro

ject

s

•D

ata

colle

ctio

n re

gard

ing

facu

lty

parti

cipa

tion

in sc

hola

rly a

ctiv

ities

and

nu

mbe

r of s

chol

arly

eve

nts

Lea

rnin

g an

d de

velo

pmen

t

Prof

essi

onal

de

velo

pmen

t; an

d

Stat

e-of

-the-

art

scho

larly

act

iviti

es

Exch

ange

pro

gram

s •

% o

f fac

ulty

usin

g sa

bbat

ical

leav

es

•N

umbe

r of e

xcha

nge

prog

ram

s ava

ilabl

e an

d av

aile

d by

stud

ents

and

re

sear

cher

s

•D

ata

colle

ctio

n ab

out n

umbe

r of f

acul

ty

usin

g sa

bbat

ical

s •

Dat

a ab

out n

umbe

r of s

tude

nts a

vaili

ng

exch

ange

pro

gram

s

123

M. Asif et al.

Tabl

e4

cont

inue

d

Proc

ess

sdohtem laitnetoP

sr otacidnIserusae

mye

Kse

m octuO

Num

ber o

f peo

ple

avai

ling

the

serv

ices

Num

ber o

f hou

rs o

f ser

vice

ava

ilabi

lity

•D

ata

colle

ctio

n ab

out n

umbe

r of s

ervi

ce

user

s and

serv

ice

avai

labi

lity

Supp

ort

serv

ices

Bes

t-cla

ss su

ppor

t se

rvic

es in

clud

ing

secu

rity,

med

ical

, m

aint

enan

ce,

hous

ing,

spor

ts, a

nd

logi

stic

s

Satis

fact

ion

of st

uden

ts fr

om

serv

ices

Sa

tisfa

ctio

n ra

te o

f ser

vice

use

rs

•Su

rvey

abo

ut sa

tisfa

ctio

n of

use

rs fr

om th

e pr

ovid

ed se

rvic

es

•Fo

rmal

and

info

rmal

feed

back

from

use

rs

Fina

ncia

l por

tfolio

Expe

nditu

res (

Res

earc

h,

teac

hing

, and

supp

ort

serv

ices

)

•To

tal b

udge

t or %

of b

udge

t allo

cate

d to

rese

arch

, tea

chin

g, a

nd

supp

orts

ervi

ces

•To

tal t

each

ing

and

rese

arch

cos

ts

Rev

enue

s •

Inco

me

gene

rate

d fr

om re

sear

ch p

roje

cts

•In

com

e ge

nera

ted

from

con

sulta

ncie

s •

Inco

me

gene

rate

d fr

om sp

in-o

ffs /

pat

ents

Fina

ncia

lFi

nanc

ial v

iabi

lity

and

Prof

itabi

lity

Spon

sors

hip

/ end

owm

ents

fr

om v

ario

us so

urce

s •

Am

ount

gen

erat

ed fr

om v

ario

us sp

onso

rshi

ps

Dat

a co

llect

ion

abou

t eac

h of

the

men

tione

d in

dica

tors

123

Developing measures for performance excellence

One precaution that needs to be exercised in the use of indicators is to supplement themwith peer review (Ball and Wilkinson 1994). While the performance indicators provideobjective evaluation, peer reviews, although subject to bias, could offer deeper insights toperformance improvement. The measures and indicators noted in the Table provide a com-pass for performance measurement. HEIs could use and benchmark these key measures andindicators to develop a comprehensive list of performance measures for their own uniquecontexts. While a structured approach to the realisation of vision and mission is critical forperformance excellence (Ball and Wilkinson 1994); the ECPE is mute on this critical aspect.

Once such measures and indicators are developed and used by the institute then the nextstep is performance assessment. This could be carried out using self-assessment, audit, andbenchmarking, or a combination of these approaches. Self-assessment is a dominant mode ofassessment in performance excellence. The main advantage of self-assessment is that it pro-motes greater involvement of the process owner as compared to an audit. Audits are a popularmeans of regular assessment. Their main advantage is that they are usually carried out againstcertain criteria—such as clauses of a standard—and are carried out by trained personnel.Owing to these two characteristics audits provide a more systematic means of performanceassessment. For a detailed comparison of self-assessment and audits, see, Karapetrovic andWillborn (2001) and Ni and Karapetrovic (2003). HEIs could also use a hybrid approachwhereby the two approaches are used together to overcome the limitations and benefit fromthe advantages of each. For example, audits could be used for the follow up of a previousself-assessment—for instance, to see the extent to which points noted in self-assessments areaddressed. Benchmarking is the comparison of organisational processes against best prac-tices in the market. Its purpose is to calibrate performance against best performers and alsoto glean new insights. This is shown in Fig. 3.

Essentially organisations could use the mentioned modes in a synergistic manner. Thiscould enhance the effectiveness of performance evaluation and could provide more com-prehensive performance assessment. However, the ECPE is mute on the synergistic use ofthe mentioned assessment methods. Future revisions of the criteria should address this point(Fig. 3).

5 Conclusions

The ECPE has evolved as a powerful means for performance excellence in education. How-ever, the ECPE do not have theoretical underpinnings. This paper identifies a number ofareas for improvements in the ECPE which need to be addressed in future revisions of themodel. The most important is that, as noted throughout this paper, the descriptive approachemployed by the criteria is too generic and does not provide specific guidance for perfor-mance excellence. For example, the ECPE requires effective management of the supplychain. The conventional concepts of supply chain, however, are different in academia thanin manufacturing and, thus, pose difficulty in understanding and application in an academiccontext.

The ECPE requires organisations to develop a mission and then align organisational pro-cesses with the mission. The underlying assumption is that the mission is appropriate andalignment would serve the required purpose. However, if an educational institute fails todevelop the right mission—in terms of understanding and addressing stakeholder needs anddeveloping best-class program—then alignment of academic processes with the mission maylead to inefficient processes. The ECPE does not address this point. The descriptive natureof the ECPE requires the organisations to consider a number of processes for performance

123

M. Asif et al.

excellence. However, it does not provide guidance on how to do that. The use of the existingknowledge of performance excellence is not a requirement. As a result organisations mayend up designing inefficient and ineffective processes even though they may be aligned withthe mission. The weaknesses of the ECPE are also discussed in terms of ‘workforce man-agement’, ‘supply chain management’, ‘customer focus, and ‘performance measurement’.The key point is that the criteria are too generic and do not discuss the requirements from anacademic perspective. This in turn could pose difficulties in their implementation.

This paper highlights the need for systematic development of performance measures tofacilitate performance excellence implementation. The systematic development of perfor-mance measures consists of defining upfront the desired outcomes, determining key mea-sures, and developing indicators and methods for measurement. The paper develops per-formance measures for a number of academic processes including developing mission andvision, program design, admission, faculty recruitment and management, learning and devel-opment, support processes, and financial portfolio. The implementation approach discussedin this paper is metrics-oriented and its starting point is defining clear outcomes, performancemeasures, and measurable indicators. Further, the assessment of performance measures isdiscussed in terms of synergistic use of self-assessment, audits, and benchmarking. Whileself-assessment remains a regular means of assessment; audits can be used to track progressof non-conforming processes identified in self-assessment. Benchmarking helps calibrateorganisational performance against the best practices in industry. Synergistic use of the threemeans of performance measurement is currently missing in the ECPE and needs to be con-sidered in future revisions of the model.

References

Alonso-Almeida, M.M., Fuentes-Frías, V.G.: International quality awards and excellence quality modelsaround the world. A multidimensional analysis. Qual. Quant. 46(2), 599–626

Arif, M., Smiley, F.M.: Baldrige theory into practice: a working model. Int. J. Educ. Manage. 18(5),324–328 (2004)

Asif, M., Raouf, A.: Setting the course for quality assurance in higher education. Qual. Quant. (2012). doi:10.1007/s11135-011-9639-2

Aytac, A., Deniz, V.: Quality function deployment in education: a curriculum review. Qual. Quant. 39(4),507–514 (2005)

Badri, M.A., Selim, H., Alshare, K., Grandon, E.E., Younis, H., Abdulla, M.: The Baldrige educationcriteria for performance excellence framework: empirical test and validation. Int. J. Qual. Reliab.Manage. 23(9), 1118–1157 (2006)

Ball, R., Wilkinson, R.: The use and abuse of performance indicators in UK higher education. High.Educ. 27(4), 417–427 (1994)

Belohlav, J.A., Cook, L.S., Heiser, D.R.: Using the Malcolm Baldrige national quality award in teaching: onecriteria, several perspectives. Decis. Sci. J. Innov. Educ. 2(2), 153–176 (2004)

Black, S.A., Porter, L.J.: Identification of the critical factors of TQM. Decis. Sci. 27(1), 1–21 (1996)Bou-Llusar, J.C., Escrig-Tena, A.B., Roca-Puig, V., Beltran-Martın, I.: An empirical assessment of the EFQM

excellence model: evaluation as a TQM framework relative to the MBNQA Model. J. Oper. Man-age. 27(01), 1–22 (2009)

Chen, S.H.: A performance matrix for strategies to improve satisfaction among faculty members in highereducation. Qual. Quant. 45(1), 75–89 (2010)

Chen S.H.: The establishment of a quality management system for the higher education industry. Qual. Quant.46(4), 1279–1296 (2012)

Cruickshank, M.: Total quality management in the higher education sector: a literature review from an inter-national and Australian perspective. Total Qual. Manage. Busi. Excell. 14(10), 1159–1167 (2003)

Dean, J.W., Bowen, D.E.: Management theory and total quality: improving research and practice throughtheory development. Acad. Manage. Rev. 19(3), 392–418 (1994)

123

Developing measures for performance excellence

ECPE (Education Criteria for Performance Excellence) 2011–2012. National Institute of Standards and Tech-nology (NIST), Gaithersburg (2011)

Evans, J.R.: Critical linkages in the Baldrige award criteria: research models and educational challenges. Qual.Manage. J. 5(1), 13–30 (1997)

Flynn, B., Saladin, B.: Relevance of Baldrige constructs in an international context: a study of national cul-ture. J. Oper. Manage. 24(5), 583–603 (2006)

Galbraith, P.L.: Systems thinking: a missing component in higher education planning?. High. Educ.Policy 12(2), 141–157 (1999)

Houston, D.: TQM and higher education: a critical systems perspective on fitness for purpose. Qual. High.Educ. 13(1), 3–17 (2007)

Houston, D.: Rethinking quality and improvement in higher education. Qual. Assur. Educ. 16(1), 61–79 (2008)Karapetrovic, S., Willborn, W.: Audit and self-assessment in quality management: comparison and compati-

bility. Manage. Audit. J. 16(06), 366–377 (2001)Kells, H.R.: Performance indicators for higher education: a critical review with policy recommendations.

Education and Employment Division, Population and Human Resources Department, The World Bank,(1992)

Krajewski, L.J., Ritzman, L.P., Malhotra, M.K.: Operations management. Pearson Prentice Hall, Upper SalleRiver (1999)

Ligarski, M.J.: Appraisal of lecturers for the assessment of the quality of post-graduate studies. Qual.Quant. 43(1), 137–144 (2009)

Lo Franco, E., La Rosa, S.: Quality as an autonomous body of knowledge. An in-depth survey on Italian highereducation. Qual. Quant. 46(3), 751–776 (2012)

Martins, A., Aspinwall, E.M.: Quality function deployment: an empirical study in the UK. Total Qual.Manage. 12(05), 575–588 (2001)

Meirovich, G., Romar, E.J.: The difficulty in implementing TQM in higher education instuction—the dualtiyof instuctor/student roles. Qual. Assur. Educ. 14(4), 324–337 (2006)

Ni, Z., Karapetrovic, S.: Perennial self-audit: model and applications. Manage. Audit. J. 18(5), 363–373 (2003)NIST: Baldrige National Quality Program. http://www.nist.gov/baldrige/ (2010). Accessed 05 May 2010Raouf, A.: Quality in higher education. Proc. Pak. Acad. Sci. 41(2), 165–174 (2004)Seetharaman, A., Sreenivasan, J., Boon, L.P.: Critical success factors of total quality management. Qual.

Quant. 40(5), 675–695 (2006)Sirvanci, M.B.: Critical issues for TQM implementation in higher education. TQM Mag. 16(6), 382–386 (2004)Slack, N., Chambers, S., Johnston, R.: Operations management with myomlab, 6th ed. Financial Times,

Prentice Hall (2010)Tribus, M.: Total quality in schools of business and engineering. In: Roberts, H.V. (ed.) Academic initiatives

in total quality for higher education., ASQ quality press, Milwaukee (1995)Veleva, V., Ellenbecker, M.: Indicators of sustainable production: framework and methodology. J. Clean.

Prod. 9(6), 519–549 (2001)Weinstein, L., Petrick, J., Saunders, P.: What higher education should be teaching about quality—but is not.

Qual. Prog. 31(4), 91–95 (1998)Winn, B.A., Cameron, K.S.: Organizational quality: an examination of the Malcolm Baldrige National quality

framework. Res. High. Educ. 39(5), 491–512 (1998)

123