Badri_Awards of excellence in institutions of HE.pdf

7/28/2019 Badri_Awards of excellence in institutions of HE.pdf

http://slidepdf.com/reader/full/badriawards-of-excellence-in-institutions-of-hepdf 1/19

Awards of excellence ininstitutions of highereducation: an AHP

approach Masood A. Badri and

Mohammed H. Abdulla

The authors

Masood A. Badri is Professor of Operations Management andMohammed H. Abdulla is an Associate Professor, both at theUnited Arab Emirates University, College of Business andEconomics, Department of Business Administration, Al-Ain,

United Arab Emirates.

Keywords

Analytical hierarchy process, Centres of excellence,United Arab Emirates, Higher education

Abstract

This paper examines how institutions of higher education mightoperationalize faculty performance evaluation in terms of research, teaching, and university and community service. An

analytic hierarchy process model is developed and presented,allowing decision makers to couple performance evaluation andacademic reward/awards and recognitions offered by institutions

of higher education, and provides an objective way to comparefaculty members. Weights are provided for each of the criteria inthe evaluation process for a more objective outcome. Reward/

award systems might include promotion decisions, merit pay,tenure, long-term contracts, and annual reward/awards of excellence in research, teaching or service. The model might be

used to make judgment on the qualification of candidates forsuch systems, and could be used on the department level,college level, or university-wide level. In addition, the model

could rank faculty members within each discipline or major. Anillustrative example is provided of the model at the United Arab

Emirates University.

Electronic access

The Emerald Research Register for this journal isavailable at

www.emeraldinsight.com/researchregister

The current issue and full text archive of this journal isavailable atwww.emeraldinsight.com/0951-354X.htm

Introduction

The subject of faculty academic reward/awards

and recognition at colleges and universities is a

challenging and controversial one. Faculty

members believe that the process of linking

academic reward/awards and recognition, and

performance is still less than satisfactory

(Braskamp, 1983; Centra, 1972; Millman, 1981).

Reward/award and recognition systems have two

major components. First, the policies and

procedures that an institution of higher education

must adopt in relation to faculty member reward/

award must be identified; second, the criteria that

are relevant and important in evaluating the

suitability of a particular individual faculty person

for a certain academic reward/award must be

designed. However, many researchers have

pointed out that there is an aura of mystery about

the decision-making process involved (Sowell,

1997; Nevison, 1980, p. 153; Zoffer, 1978, p.903).The need for system reappraisal seems to be a

major issue confronting college and university

administrators in many schools (Barnett, 1996;

McFerron et al ., 1996; Stark and Miller, 1976).

In brief, the lack of more explicit criteria

adopted in academic reward/award systems and

formal decision-making methodologies, or

models, is perhaps, serious. There is a need to

make the criteria adopted in academic reward/

award system decisions more explicit, and how

these criteria are actually used to reach final

decisions. In addition, the absence of a simple

methodology to combine the diverse factorseffectively in arriving at a consistent decision is also

a source of conflict and controversy.

In this paper we propose a model that identifies

essential criteria that are relevant and important in

evaluating the suitability of a particular individual

faculty member for academic reward/awards in

institutions of higher education. The model will

cover the three major dimensions of research,

teaching and service. It will highlight the specific

elements to be included in each dimension. Once

the model is established, we will propose a

prioritization methodology based on the AHP,

which is firmly grounded in mathematical theory.A real life example will be discussed showing the

implementation of the proposed model and

methodology in selecting academic award of

excellence winners at the University of the United

Arab Emirates.

In addition, the paper will recommend a process

for the application of the model at four specific

levels, within each discipline, department, college,

or university-wide level. Recognizing the fact that

research quantity, teaching methods and styles of

presentation, and variety and level of involvement

International Journal of Educational Management

Volume 18 · Number 4 · 2004 · pp. 224-242

q Emerald Group Publishing Limited · ISSN 0951-354X

DOI 10.1108/09513540410538813

224



in university and community services, might differ

from one discipline to another, the model could be

used as a stand-alone method for specific purposes

or more general objectives. The model offers easy-

to-use, versatile, and objective methodology to

prioritize faculty members inside a discipline or

major for excellence awards, merit pays, tenure

decisions, or other recognition schemes. On auniversity-wide level, the model could be used to

identify winners of academic excellence awards

and recognitions or other benefit providing

programs that are based on faculty performance.

In developing the model, many relevant

previous literatures are reviewed. From these

studies, a list of criteria is designed for use in the

analytic hierarchy process model. In addition to

reviewing literature, we visited many Web sites of

higher education institutions to understand and

list some of the Web-published criteria of academic

excellence. Many universities offer Awards of

Excellence, and their process and outcomes tothese awards are posted online on the Internet. At

the time of our search, examples of such

universities include Pacific Lutheran University,

University of North Carolina, University of Texas

at Austin, University of Kansas, University of

Tasmania, Laurentian University, and Stephen

F. Austin State University.

Criteria for academic reward/awards

There appears to be a good deal of consensus in

the literature as to what criteria are frequently used

by responsible committees in assessing the

suitability or fitness of candidates for certain

academic reward/awards such as tenure and other

recognition schemes (Jolson, 1974; Luthans,

1967; Remus, 1977; Stark and Miller, 1976;

Zoffer, 1978). In addition to tenure systems, these

reward/award and recognition schemes might

include long-term contracts (McFerron et al .,

1996; Helms et al ., 2001), salary increments,

promotions along with annual awards for

excellence associated with research, teaching and

service have received less attention (Ehie and

Karathanos, 1994; McKenna et al ., 1995; Alpert,

1985; Henninger, 1998).

Faculty members at most universities areevaluated on the basis of their performance in

three major areas of teaching, research, and service

(Helms et al ., 2001). Basing faculty reward/awards

and recognition on productivity rather than

longevity or rank is supported in many studies

(Leatherman, 2000). In this section we will review

literature on common criteria used to assess faculty

members for the various schemes of reward/award

and recognition. Specifically, we will attempt to

highlight these criteria associated with research,

teaching and service.

Criteria for academic reward/awards in

research

Many studies suggest strong relationships between

faculty research productivity and academic

reward/award systems offered by universities

(Hoyt, 1974; Salthouse et al ., 1978; Mesak and

Jauch, 1991). Research, in particular, has gained a

great deal of attention in recent years. One studyfound that a sample of college-level faculty felt

compelled to do research and publish in order to

continue in the university and advance. Seldin

(1984), in a survey of 616 academic deans, shows

that, when it comes to reward/awards and

recognition, the research component could be

considered to be more important relative to

teaching and service. Hence, most institutions of

higher education assign the highest weight to

research and lesser weights to teaching and service.

In most cases, service gets smallest weights. Seldin

(1984) shows that evaluators will display

increasing concern for both the quality of research,as well as the quantity. He adds that a tendency

towards reliance on third-party evaluations of

scholastic competence to reduce local professional

biases and political considerations is expected.

Mesak and Jauch (1991) recall that evaluators

need to consider relative differences in individual

contributions of research co-authoring a certain

form of research output.

For business schools, the AACSB assumes that

faculty scholarship can affect faculty reward/award

systems (Henninger, 1998). Usually, the AACSB

has great impact on the type of faculty hired by

schools and how they are rewarded/awarded.Business schools often select and reward/award

faculty for their credentials and ability to conduct

empirical research and publish in refereed journals

(Cotton et al ., 1993; Ehie and Karathanos, 1994;

McKenna et al ., 1995).

At the UAE University, research is considered

to be an important criterion in recruiting faculty

members. It is also considered an essential element

in the academic promotion process (in promoting

a faculty member to associate professor or full

professor, the weights are 40 per cent, 40 per cent,

and 20 per cent for research, teaching and

university and community service). It is

understandable that when assessing candidates for

overall academic excellence, the weights would be

different from those used for academic promotion

purposes.


teaching

The University of North Carolina at Chapel Hill is

taking aggressive steps to foster quality and

excellence in teaching and is including awards to

recognize teaching excellence and creating special

Awards of excellence in institutions of higher education

Masood A. Badri and Mohammed H. Abdulla


Volume 18 · Number 4 · 2004 · 224-242

225



activities to support and strengthen instruction

(Helms et al ., 2001). They recommend institutions

provide tangible incentives and encouragement for

faculty and teaching assistants to take advantage of

professional opportunities that are offered.

The relationship between teaching and

academic reward/awards has been inconclusive

with positive results in some studies (Katz, 1973;Siegfried and White, 1973) and insignificant (or

with negative correlation) in others (Marsh and

Dillon, 1980; Tuckman, 1979). Seldin (1984)

expects that teaching should be scrutinized more

closely, not only by student evaluations, but also by

examining course materials. He adds that the

component of teaching should also include

developing materials and evaluating learning.

Kiesler and Sproull (1987) call for teaching that

leads to the increase in the use of microcomputers,

intelligent or smart classrooms, and other products

of electronic miniaturization to be encouraged

more. Ludesh (1987) also calls for the use of technology to support teaching, because it

encourages students’ learning independent of a

classroom environment. Mesak and Jauch (1991)

also suggest the inclusion of expertise (advanced

training and education) and instructional delivery

skills. Srinivasan and Basu (1989) suggest the

consideration of instructional design skills as a

major factor in teaching evaluation.

Previous studies and recent literature reviews

noted that student evaluation of teachers in

institutions of higher education can improve

teacher effectiveness (Brightman et al ., 1993;

Cohen, 1980; Cohen and Herr, 1982; Tiberiuset al ., 1989). Many researchers suggest that

students are the natural party to evaluate the

dimension of teaching; these studies show that

student ratings are reasonably reliable and stable

(Costin, 1968; Centra, 1972; Feldman, 1977).

Many studies provide evidence that systematic

training in teaching such as teaching workshops

and seminars do appear to improve teachers’

performance in the classroom. Some of these

studies where general in nature (Bray and Howard,

1980; Brightman, 1987; Caroll, 1980; Levinson-

Rose and Menges, 1981), while other studies

focused on students in specific majors. These

specific majors include chemistry (Murphy, 1973;

Yaghlian, 1973), psychology (Crites, 1965),

nursing (Lewis and Orvis, 1973), mathematics

(Tubb, 1975; Daniels, 1970).


service

University and community service was found to

have little impact on reward/awards (Dornbusch,

1979; McCarthy, 1980), except for those involved

in internal administrative service as a departmental

head, dean, or director of research (Tuckman et al .,

1977; Tuckman and Hagemann, 1976).

Seldin (1984) shows that service includes

campus committee, community and professional

services, time at institution, personal attributes,

and student advising. Mesak and Jauch (1991)

suggest several elements of service to include

university service (serving on departmental,college and university committees, securing

contracts and grants, and supervision of doctoral

students), professional services (reviewing books

and papers, membership of professional

organization committees, consulting activities)

and community service (serving on community

boards and activities, speeches to local community

groups).

Models of faculty evaluation and reward/

award systems

Research suggests that top-tier schools report thatthree elements are most important in the

evaluation of a faculty member. They include

teaching, research and service (Earls, 1997;

Collison, 1990; Clement and Stevens, 1989; Farh

et al ., 1988). There is widespread agreement that

the universities need to emphasize exemplary

teaching, research and service to complement one

another and provide a creative atmosphere for

discovery, learning and development for faculty

and students (Gage, 1993). Faculty evaluation

basically entails the comparison of the actual

achievements of a faculty member, materialized

over a certain period of time, with predetermined

standards. Mesak and Jauch (1991) provided a

deterministic multi-criteria evaluation model.

Their model considered several constraints

imposed by an institution of higher education such

as time, mission, and environment. Time

constraint reflected the fact that many colleges and

universities insist that faculty members should

satisfy a particular period of time at the institution

before they become eligible for various reward/

awards. Each institution carries out a certain

mission related to research, teaching and service.

Accordingly, faculty members should satisfy

certain minimum achievements along the different

dimensions of research, teaching, and service,considered separately as well as collectively, before

being considered for any of the three decisions.

Environment constraints are concerned with

external market forces related to competitiveness.

The paper of Mesak and Jauch (1991) provides

a mechanism for evaluation without providing

details and explanations on suggesting or

developing the values that should be present in

each of the three dimensions of research, teaching

and service. In addition, the paper does not

provide suggestions with regard to how priorities




Volume 18 · Number 4 · 2004 · 224-242

226



or preferences for each element in the three

dimensions should be accounted for. Moreover, a

real world example was not provided to

understand the mechanism better. In summary,

the authors did not determine the parameters

associated with the “mission” constraints together

with other considerations dealing with measuring

the three dimensions of performance.Liberatore et al . (1992) suggested using an AHP

model to aid in the process of selecting papers to

win academic awards in the College of Commerce

and Finance at Villanova University. The objective

of the AHP system was to develop an improved

evaluation system that is similar in concept and

application to the scoring model employed by

many universities. However, their research was

limited to criteria used to evaluate a certain

research paper. These criteria included research

objectives, justification, design, execution-

implementation, and recommendations-

implications. Saaty and Ramanujam (1983)employed the AHP as an evaluation approach for

the selection of candidates for promotion and/or

tenure. However, the paper was mainly focused on

explaining the AHP procedure and its

mathematical basis using hypothetical data.

Many colleges of higher education sponsor

annual awards in research (Dagani, 1993;

Liberatore et al ., 1992; Mesak and Jauch, 1991;

Lischwe et al ., 1987), in teaching (Dagani, 1993;

Mesak and Jauch, 1991; Gage, 1993), and in

community service (Hamilton, 1995; Dagani,

1993; Mesak and Jauch, 1991), to encourage and

reward/award faculty efforts in these areas.However, the criteria used to evaluate each of the

performance dimensions are not published

explicitly or in detail. In this section we will

propose and introduce the design of a

comprehensive criteria model for the award

system(s).

As we mentioned, there were several methods or

procedures we used to develop the model(s) in

addition to the review of literature. First, many

schools were contacted to request their criteria for

evaluating faculty members’ performance on the

three dimensions. Second, for universities that

make such criteria available for the public, several

university Web pages were visited.

The analytic hierarchy process

Thomas Saaty developed the analytic hierarchy

process (AHP) approach to hierarchy development

and validation at the Wharton School of Business.

The AHP is a method of breaking down a

complex, unstructured situation into its

component parts; arranging those parts, or

variables, into a hierarchic order. The method is

based on assigning numerical values to subjective

judgments on the relative importance of each

variable; and synthesizing the judgments to

determine which variables have the highest priority

(Saaty, 1994).

The AHP is built on three principles: the

principle of constructing hierarchies, the principle

of establishing priorities, and the principle of

logical consistency. The task of setting prioritiesrequires that the criteria and sub-criteria be

layered in the hierarchy so that the elements of

each level are comparable among themselves in

relation to the elements in the next higher level.

A weighting process is used to obtain overall

priorities. This is done by moving down the

hierarchical structure of information and

weighting the priorities measured in a hierarchy

level with respect to the criterion in the next higher

level with the weight of that criterion. For the

weighting process, the AHP uses pairwise

comparison to create derived scale for each set of

criteria in the hierarchy. The pairwise comparisonasks the individual to specify levels of intensity or

preference.

Understanding of hierarchies

Learning theorists have determined that many

learning goals can be thought of as hierarchical in

nature. The hierarchical structure of knowledge

appears in a wide variety of subject areas. There is

even evidence suggesting that the arrangement of

information in a hierarchical fashion is what

separates novices from experts. The hierarchy

provides a visual means of defining and assessing a

target objective.

Understanding inconsistencies

It is a fact that we are very seldom perfectly

consistent in making comparative judgments,

particularly when we deal with intangibles that

have no scales of measurement; and, we should not

expect to be totally consistent. The real world

often lacks consistency, and we must be able to

reflect that in our models. For example, Team A

can beat Team B, and Team B can beat Team C,

yet Team C might then beat Team A. When a

decision maker uses subjective measurements,

such as the ones in the assessment of faculty

excellent performance, consistency of judgment is

not guaranteed. One of the greatest advantages of

the AHP is that it has built-in systematic checks on

consistency of judgments.

We use a test statistic, the inconsistency index

(CI), to measure consistency in decision makers’

comparison of performance criteria (or elements).

The CI is then compared to values of the same

index for a randomly generated matrix, RI. The

ratio of CI to the average RI for the same order

matrix is called the consistency ratio, CR.




Volume 18 · Number 4 · 2004 · 224-242

227



A consistency ratio of zero means that the

comparisons are perfectly consistent; higher ratios

indicate lower consistencies. Expert Choice 2000,

the software used, provides a measure of logical

rationality, called the inconsistency ratio, but does

not force the user to be consistent.

The inconsistency ratio is calculated for each set

of judgments. It is important to emphasize that theobjective is to make “good” decisions, not to

minimize the inconsistency ratio. Good decisions

are most often based on consistent judgments, but

the reverse is not necessarily true. It is easy to make

perfectly consistent judgments that are nonsensical

and result in terrible decisions. When the

inconsistency ratio is zero we have complete

consistency; when it is greater than zero there is

some inconsistency. The larger the value of the

inconsistency ratio the more inconsistent the

judgments. If it is 0.10 or less the inconsistency is

generally considered tolerable. The degree of

inconsistency that indicates a “significant”problem depends, of course, on the specific

situation where the model is applied. The number

0.10 is given as a general guideline.

As long as there is enough consistency to

maintain coherence among the objects of our

experience, the consistency need not be perfect. It

is useful to remember that most new ideas that

affect our lives tend to cause us to rearrange some

of our preferences, thus making us inconsistent

with our previous commitments. If we were to

program ourselves never to change our minds, we

would be afraid to accept new ideas.

To measure the inconsistency of all thejudgments made in the decision hierarchy, we take

the inconsistency value of each set of comparisons

and multiply it by the priority of the element with

respect to which of these comparisons are made,

and add for all the elements. This gives a single

overall weighted number. To decide how

acceptable this number is, we form a ratio with a

similar number obtained by multiplying the

corresponding random inconsistency value for an

equal number of comparisons by the priority of the

elements, and again add over each attribute. As

mentioned, the resulting ratio should be 0.10 or

less.

AHP steps for developing the academic

award model

For the excellence in performance in higher

education problem, seven steps are followed, as

outlined by Saaty. These steps will be utilized in

the development of the hierarchy of performance

assessment in institutions of higher education. The

steps are:

(1) Define the problem (assessment of

performance excellence in UAE University).

(2) Structure the hierarchy (the main category of

performance (research, teaching, and

university and community service) and

detailed elements of each category).

(3) Construct a pairwise comparison matrix of the

relevant contribution or impact of each

element on each governing criterion in the

next higher level.(4) Obtain judgments required to develop the set

of matrices in step 3.

(5) Having collected all the pairwise comparison

data, the priorities are obtained and the

consistency is tested.

(6) Perform steps 3, 4, and 5 for all levels and

clusters in the hierarchy.

(7) Use hierarchical composition to weight the

vectors of priority by the weights of the

criteria, and take the sum over all weighted

priority entries corresponding to those in the

next lower levels and so on.

Study objectives

The study attempts to first develop a valid and

reliable instrument to assess faculty performance

with regard to research, teaching, and service; and

then, using the AHP, to provide an objective

assessment of the candidates. The instrument will

provide an objective framework to measure

performance.

To conform to the AHP methodology, the study

was broken into several phases to facilitate the

management of the research. The four phases

consisted of hierarchy development, hierarchy

validation, assessment instrument development,

and instrument reliability testing. The hierarchy

development process involved a review of various

assessment lists and approaches used by the UAE

University and other universities, and the review of

assessment variables in literature. After eliminating

certain variables based on selection criteria, the

remaining items were clustered and arranged into a

hierarchy. Stringent scientific methodologies were

used in the other phases. The final outcome of this

research is to be applied to assessment and

selection of “excellent performance” with regard

to research, teaching, and service in the UAE

University.

Stated explicitly, the objective of this study is todevelop an AHP model for performance

assessment, and explore its implementation in

UAE University. Using the AHP, an instrument

will be designed, analyzed, tested and validated.

Full field experimentation, decision makers’

involvement, and implementation would ensure

model validity and acceptance. More specifically,

the objectives are the following:. through extensive field research, identify the

dimensions and criteria of faculty

performance assessment in higher education;




Volume 18 · Number 4 · 2004 · 224-242

228



. using the AHP, provide a model to increase

the objectivity of the decision-making process

regarding excellent performance assessment;

and hence, provide a simple and

understandable methodology for the

development and design of the AHP-based

model;. design an objective model based on the AHP

to determine the weighing of the criteria for

evaluation and assessment and provide simple

procedures and directions for the easy

implementation of the model in UAE

University; and. solicit feedback on user satisfaction with the

model developed and with the AHP as a

methodology for excellent faculty

performance.

This paper will continue to broaden the causal

base for faculty performance by considering the

design and development of a system that is based

on the AHP for objectivity. Even though the focus

of the study will be on performance excellence in

UAE University, the developed model and the

method of analysis would be comprehensive

enough to be used, or modified, and implemented

in other institutions of higher education for the

same purpose or for other purposes.

The developed AHP models for awardsystems

To understand better the steps performed toimplement the AHP model, we did the following:

Pre-hierarchy development

It involved the review of current practices of

reward/awards in the UAE University, other

universities in the region, universities that

published some sort of information on reward/

awards on the Web (mostly North American

universities), and the review of literature dealing

with the subject. We acquired the current form

used by the UAE University to start with. The

form was modified and sent to the colleges for

feedback. Since the College of Business andEconomics and the College of Engineering are

both accredited (AACSB and ABET respectively),

a review of accreditation criteria was also

conducted.

Setting up the decision hierarchy

We decomposed the main task into primary

elements; then, we grouped these elements into

several main categories with different levels

forming a chain or hierarchy. In other words, the

lower hierarchy was a decomposition of the main

category in the higher (or parent) hierarchy. The

final items were clustered and arranged into a

hierarchy based on Saaty’s hierarchy development

process.

Hierarchy validation process

The process involves the validation and weighing

of the hierarchy. To ensure involvement and

participation, the final nomination team included

seven faculty members, one from each college at

the university. The team determined the relative

importance of each item in the hierarchy while

simultaneously validating the hierarchy. Extensive

interviews with each member were conducted

during this step. The main purpose of this step was

to collect input data to measure the relative

importance of any element over other elements.

The decision makers follow a pairwise comparison.

The process resulted in a set of comparison

matrixes. An assessment instrument was

developed based on the weight of each item in thehierarchy (a detailed presentation of the results of

this step will follow later in this paper).

Estimating relative weights of elements

Since there were several methods to compute the

priorities/importance of elements in each matrix,

we took as input, pairwise comparisons prepared

in the previous steps, and produced as output

relative weights of elements at each level.

Implementation of the AWARD model at the

UAE University

The weights of elements for various levels,

computed in the previous phases, were aggregated

to produce a vector of composite weights that

serves as a rating of decision variables or selection

choices. In order to obtain a precise measurement

scale, a nine-point combination scale was assigned

to each item within the hierarchy (Saaty, 1994).

Each item on the scale consisted of nominal,

descriptive and ratio value.

The main objective of this study is to develop a

comprehensive model that establishes definitions

of criteria for evaluating faculty member

candidates on the three components of research,

teaching and service. This model could be used for

a variety of objectives as will be explained later. As

shown in Figure 1, the model in its first level

attempts to identify the overall winner (i.e.

“Excellent Research Award”, “Excellent Teaching

Award”, “Merit Award Nomination Winner(s)”,

“Excellent Performance in the School of

Business”, etc.).

The second level identifies the main areas where

the winners are supposed to perform with

excellence. In our model, these three areas are

research, teaching, and university and community




Volume 18 · Number 4 · 2004 · 224-242

229



service. It is clear that other criteria could be easily

added, if required. If the objective is to select “Best

Research Performance”, then we might ignore the

other two components of teaching and service. In

this case, we will have two levels of decision makingonly, selecting the winner(s), and the criteria of

evaluation.

The AHP “research” criteria model

At the UAE University, the Chancellor’s Decree

number 139 for 2002 specifies the main elements

of research. These elements or criteria include the

following:. scientific research published or accepted for

publication;

. participation in scientific meetings;

. publication of texts and reference books;

. innovative activities;

. scientific findings; and

. other distinguished scientific achievements.

Based on the by-laws of the university and search

of literature, with regard to research, we identified

and classified four main categories (level three):

research production, patents and awards,

refereeing and memberships, and other research

related contributions. The main subcategories for

each are not shown in Figure 1. However, Table I,

which provides the outcome of the model, also

provides an explanation of each of the levels for

research.

Figure 1 Basic model




Volume 18 · Number 4 · 2004 · 224-242

230



For the research component, we are going as far

as five levels. The research production component

is further broken down into seven criteria (level

four) of research published in rank 1 journals,

research published in rank 2 journals, research

published in rank 3 journals, research published in

international conference proceedings, papers,

roundtables, or demonstrations presented at

academic or professional meetings, paper reviews,

and published books. Only the criteria of

published books are further broken down into six

elements (level five) of published text books (for

teaching), published reference books (not for

teaching), published translated books, published

chapters in books, edited works in books or

textbooks, and monographs.

Patents and awards are based on four criteria of

patents and innovative works, international awards

for research, regional awards for research, and

local awards for research. Refereeing and

memberships are based on five criteria of memberships of editorial boards of international

journals, membership of editorial boards of local

journals, membership of professional international

societies, refereeing scientific articles in

international journals, refereeing Doctoral or

Master’s theses, and dissertations. Finally, other

related contributions include efforts and

contributions in establishing labs, research units,

etc., research grants from international

institutions, research grants from local institutions,

and research grants from the university.

We need to recommend at this point that otheruniversities could attempt to establish their own

criteria for assessing excellence performance as

they see suitable.

The AHP “teaching” criteria model



of teaching. These elements or criteria include the

following (Table II):. teaching loads and efforts, and the variety of

courses taught by a faculty member;. participation in the development of scientific

materials for a course;. documentation of teaching methods and

materials;. the use of new technology teaching methods

and participation in developing them;

Table I Priority scores and inconsistencies for the research part of the AHP model

R1. Research published 0.643

R11. Research published in journals 0.731

R111. Rank 1 journals 0.642



R114. International conference proceedings 0.056 (0.07 )

R12. Published books and similar activities 0.188R121. Published text books 0.222

R122. Published reference books 0.376

R123. Published translated books 0.152

R124. Published chapters in books 0.111

R125. Edited works in books or textbooks 0.085

R126. Monographs 0.054 (0.03 )

R13. Other research paper contributions 0.081 (0.06 )

R131. Papers presented in international meetings 0.750

R132. Paper reviews 0.250 (0.00 )

R2. Patents and awards 0.183

R21. Patents and innovative works 0.247

R22. International awards for research 0.545

R23. Regional awards for research 0.141

R24. Local awards for research 0.067 (0.03 )

R3. Refereeing and memberships 0.100

R31. Memberships of editorial boards of international journals 0.555

R32. Membership of editorial boards of local journals 0.132

R33. Membership of professional international societies 0.054

R34. Refereeing scientific articles in international journals 0.045

R35. Refereeing Doctoral or Master’s theses and dissertations 0.215 (0.04 )

R4. Other research contributions 0.074 (0.0

R41. Contributions in establishing labs, research units, etc. 0.143

R42. Research grants from international institutions 0.599

R43. Research grants from regional and local institutions 0.180

R44. Research grants from the university 0.079 (0.05 )




Volume 18 · Number 4 · 2004 · 224-242

231





“self-assessment”, “books and reference books

used”, “educational resources used in teaching”,

and “assessment methods and student

relationships”. Again, we could use all these

components in the AHP model if we want to.

The category of course files and exams includes

three criteria of the completeness of the course

files, the design and organization of courseoutlines, and quality of design and preparation of

exams. Efforts in development of courses and

methods of teaching are based on two criteria:

quality of design of the teaching portfolio design,

and documentation of the teaching portfolio.

Finally, contributions in conferences and

workshops related to teaching methods is based on

three criteria of organizing conferences and

workshops related to teaching, participation in

international conferences and workshops related

to teaching, and participation in regional or local

meetings related to teaching.

The AHP “service” criteria model



of the university and community service

component. These elements or criteria include the

following (Table III):. participation in committee works,

participation in the development or

management of university units or

curriculum;. participation in planning or execution of

faculty members’ professional development

activities;. participation in international academic

accreditation endeavors;. participation in assessment and evaluation of

faculty peer reviews;. participation in student academic advising;. participation in extra-curricular activities;. participation in efforts of academic exchange

with other universities and societies; and. participation in recruiting highly qualified

faculty members.

As shown in Table III, the service criteria model

has two main categories: university service and

community service. University service has two

components: membership of committees and

contribution to conferences and seminars.

Another level is added to the two components. The

contribution to conferences and seminars

component is based on the six criteria of lecturing

in seminars at the university, contribution to

professional workshops at the university level,

official works performed as requested by faculty or

university, representing the university in regional

administrative meetings, extra-curricular activities

performed, and administration positions held at

the university (university, college, or department

level).

The community service category includes six

components or criteria: contribution to

conferences locally, community presentations and

articles, consultations and training, applied studies

in collaboration with other institutions, chairing

scientific societies, and memberships in scientificsocieties. Another level is added to community

shows and articles to be based on local and

regional newspaper articles, local and regional

magazine articles, radio and television productions

(or participation), artistic performances and shows

(including artistic exhibitions). In addition,

consultations and training is based on the three

criteria: consultations provided to governmental

agencies, consultations provided to private firms,

and organizing specialized training for local firms.

Detailed illustration of the “Awards of Excellence” at UAE University

There are several awards of academic excellence

offered by the United Arab Emirates University.

On the basis of prestige, these awards include

“Overall University Excellence Award”,

“University Excellence Award in Research”,

“University Excellence Award in Teaching”,

“University Excellence Award in Service”,

“College Excellence Award in Research”, “College

Excellence Award in Research”, “College

Excellence Award in Service”, and “College Best

Performance Award”. The Chancellor presents

the “Excellence” awards annually. Only three

winners are announced every year, one in each

area of research, teaching and service. In addition

to recognition, they receive financial incentives.

Other nominations that were on the list of

candidates but were not nominated for

“Excellence Awards” could be considered for

“College Excellence Awards”. The Chancellor

also presents these awards. However, these awards

carry less monetary incentive compared to

“Excellence Awards”. Excellence Awards,

university and college levels, have to be approved

by a committee formed by the Deputy Vice

Chancellor for Academic Affairs (or Provost). The

final award, “College Best Awards” do not carrymonetary incentives, and are presented by the

Dean of each college. Figure 2 provides a better

view of the process involved in selecting academic

excellence winners at the United Arab Emirates

University.

An online questionnaire was designed according

to the decision hierarchy developed in Figure 1.

Related elements were grouped together to form a

matrix for pairwise comparison. A committee in

the Provost’s office was formed to screen the

nominations, perform an initial comparison,




Volume 18 · Number 4 · 2004 · 224-242

233



identify the final candidates, and perform the

assessment for the selection of winners. The AHP

model described here is concerned with the final

assessment for the selection process. The

committee consisted of five members: the

Assistant Deputy Vice Chancellor, three College

Deans, and the Advisor to the Deputy Vice

Chancellor.

The committee members were invited to

complete the online questionnaire to record

priority weights for each category, research,

teaching, and university and community services.

In doing so, they also assigned weights to each

specific element within each category. The results

of this phase were the development of pairwise

comparison matrices, or the relative weights of the

elements on each level in each hierarchy were

computed. Priority scores and inconsistencies for

the “research” part of the hierarchy are provided in

Table I. Similar scores for “teaching” and

“university and community service” are provided

in Tables II and III respectively.

We recall that we termed as “Validation-team”

also identified priority scores when they made their

pairwise comparisons of the categories and sub-

categories. For further validation purposes,

we formed two columns of numbers consisting of

priority weights of all categories and subcategories

assigned by the “Validation-team” and the Provost

team. The Pearson’s Correlation Coefficient is

Table III Priority scores and inconsistencies for the service part of the AHP model

S1. University service 0.333

S11. Committee memberships 0.750

S111. Participation in university-level

committees 0.655

S112. Participation in college-level

committees 0.250

S113. Participation in department-level

committees 0.095 (0.02 )

S12. Contributions to conferences/seminars 0.250 (0.00 )

S121. Lecturing in seminars

at the university 0.080

S122. Contributions in organizing

workshops at the university 0.174

S123. Official work performed

as requested by college 0.243

S124. Representing the university

in regional meetings 0.138

S125. Extra-curricular activities performed 0.109

S126. Administrative positions held

at the university 0.256 (0.03 )

S2. Community service 0.667 (0.00 )

S21. Contributions in local

conferences 0.079

S22. Community shows/articles 0.056

S221. Regional/local newspaper articles 0.261

S222. Regional/local magazine articles 0.169

S223. Radio/TV productions and

shows appearances 0.451

S224. Artistic performances and

shows 0.119 (0.03 )

S23. Consultations/trainings 0.239

S231. Consultations provided to

government 0.311

S232. Consultations provided to

private firms 0.493

S233. Organizing special trainings

for local firms 0.196 (0.05 )

S24. Empirical research with

other institutions 0.372

S25. Chairing scientific societies 0.155

S26. Memberships in scientific

societies 0.099 (0.04 )




Volume 18 · Number 4 · 2004 · 224-242

234



0.936, which indicates a high association between

the two groups.

The computed overall priority scores for thethree main categories of research, teaching and

university and community service are (0.682),

(0.216) and (0.103) respectively. As we

mentioned, the weights given to the three

categories for academic promotion purposes are

(0.40), (0.40) and (0.20) respectively. It is clear

that the committee members assign weights that

are not consistent with the same ones assigned

when promotion decisions are taken. The

committee feels that awards have other objectives,

processes, and category-elements. Hence, the

differences in weights reflected by the goal of

“awards of excellence” and “academicpromotions” are justifiable and understandable.

As shown in Table I, with regard to research,

publications are considered to be the most

important element with a weight of (0.643). The

Table also shows that publications have three

subcategories of research published in journals,

published books or similar activities, and other

research paper contributions. Out of these three

subcategories, research published in journals

receives the highest scores with a weight of (0.731).

The Table also provides inconsistency scores for

each of the comparisons that was performed (the

italicized numbers in the Table). As explainedearlier, scores below 0.10 are desired. All

inconsistency scores meet the desired criteria.

Inconsistency scores range between (0.00) and

(0.07).

Table II shows priority and inconsistency scores

for the teaching category. The highest priority is

given to student evaluation of teaching (0.374),

with diversification in teaching being second with a

score of (0.191). Inconsistency scores range

between (0.00 and 0.05). Priority and

inconsistency scores for the university and

community services are provided in Table III.

Community services receive an importance weight

of (0.667), while university services get (0.333).Inconsistency scores range between (0.00) and

(0.05).

The next phase involved computing the ideal

scores for each of the candidates on the three main

categories and subcategories. After assigning

weights for each of the main categories and

subcategories, the committee members had to use

the same pairwise comparison procedure but this

time to assess the candidates with regard to those

categories. With regard to research, and as shown

in Table IV, the first candidate, MG1 receives a

clear cut priority (an ideal score of 0.338,

compared to the ideal score of 0.253 received byENG). The table also shows the ideal scores with

regard to each subcategory along with the

inconsistency scores. With regard to the main four

subcategories of research, candidate MG1

outperformed all other candidates on only one of

the subcategories (research published). The

candidate ENG outperformed all other candidates

with respect to the other three main subcategories

of patents and awards, refereeing and

memberships, and other research contributions. It

is now obvious how crucial were the results in

Table I, where this main subcategory received an

importance weight of 0.643 compared to 0.183,0.100, and 0.074 received by the other remaining

subcategories respectively.

As shown in Table V and Table VI, candidate

AGR performed best on teaching (a score 0.417

compared to 0.192, 0.145, and 0.246). Candidate

MG1 and AGR were equal but performed better

than the other candidates with regard to university

and community services (a score of 0.329

compared to 0.112, and 0.231). All three tables

provide desirable inconsistency scores, which are

below the upper limit of (0.10).

Figure 2 The process of Awards for Excellence




Volume 18 · Number 4 · 2004 · 224-242

235



According to the AHP methodology, the overall

winner (University Excellence Award) is MG1.

The University Excellence Award in Research was

awarded to ENG. The University Excellence

Award in Teaching was awarded to AGR. The

University Excellence Award in University and

Community Service was not awarded to any of the

candidates. All other candidates that were

nominated by their colleges but were not selected

by the Provost’s committee were awarded their

respective colleges’ awards for best performance.

Another important aspect of the AHP is theconcept of sensitivity analysis or “what if?” The

sensitivity feature allows the model builder (or

committee members) to alter the weights assigned

to the objectives and observes their overall effects.

The sensitivity feature provides a mechanism for

helping decision makers define the range of

possibilities that the organization will face. In other

words, there is a possibility of changes taking place

in the decision-making process, criteria,

alternatives, priorities, or scores that may render

the judgments in this model requiring

modifications. Nevertheless, the proposed

hierarchy offers a robust framework that could

model different scenarios and changes that might

occur. It is an adaptive methodology for

prioritization and changes, and can be considered

as a modeling representation of a knowledge base.

Sensitivity analysis examines the sensitivity of

the results and changes in the priorities of the

criteria. This is a particularly important aspect of

an AHP problem analysis, since results are based

on subjective expert assessments. Sensitivity

analysis can be performed from any level in thehierarchy; the software displays in a graphic form,

the sensitivity of alternatives to priority changes of

the criteria immediately below a user-selected

node. This flexibility is very useful for fine-tuning

the sensitivity analysis.

The Expert Choice software incorporates a

methodology that allows use of the original

judgments to facilitate any changes. It allows the

user to vary the priorities of the alternatives, sub-

criteria and criteria. Any variations will affect the

priorities of all the other elements in the AHP

Table IV Ideal mode scores for candidates with respect to the research component and inconsistencies

Candidates MG1 MG2 ENG AGR Inconsisten

Final Scores (RESEARCH) 0.338 0.133 0.277 0.253

R1. Research published 0.390 0.131 0.295 0.183

R11. Research published in journals 0.455 0.145 0.222 0.178

R111. Rank 1 journals 0.553 0.101 0.212 0.135 0.04

R112. Rank 2 journals 0.495 0.117 0.194 0.194 0.02

R113. Rank 3 journals 0.391 0.138 0.276 0.195 0.05

R114. International conference proceedings 0.696 0.052 0.151 0.101 0.05

R12. Published books and similar activities 0.199 0.270 0.265 0.266

R121. Published text books 0.111 0.222 0.222 0.444 0.00

R122. Published reference books 0.200 0.400 0.200 0.200 0.00

R123. Published translated books 0.250 0.250 0.250 0.250 0.00

R124. Published chapters in books 0.271 0.120 0.418 0.191 0.03

R125. Edited works in books or textbooks 0.191 0.120 0.418 0.271 0.03

R126. Monographs 0.161 0.144 0.425 0.270 0.02

R13. Other research paper contributions 0.681 0.063 0.160 0.096

R131. Papers presented in international meetings 0.685 0.058 0.164 0.093 0.02

R132. Paper reviews 0.669 0.076 0.151 0.104 0.05

R2. Patents and awards 0.259 0.097 0.447 0.198

R21. Patents and innovative works 0.143 0.086 0.507 0.264 0.01R22. International awards for research 0.272 0.088 0.483 0.157 0.01

R23. Regional awards for research 0.281 0.140 0.340 0.239 0.02

R24. Local awards for research 0.499 0.074 0.257 0.169 0.04

R3. Refereeing and memberships 0.370 0.084 0.400 0.146

R31. Memberships of editorial boards of international journals 0.308 0.064 0.509 0.199 0.01

R32. Membership of editorial boards of local journals 0.567 0.101 0.166 0.166 0.02

R33. Membership of professional international societies 0.478 0.138 0.256 0.128 0.00

R34. Refereeing scient ific artic les in international journals 0.480 0.108 0.216 0.196 0.01

R35. Refereeing doctoral or master theses and dissertations 0.370 0.100 0.345 0.185 0.00

R4. Other research contributions 0.245 0.167 0.356 0.232

R41. Contributions in establishing labs, research units, etc . 0.138 0.126 0.449 0.288 0.02

R42. Research grants from international institutions 0.200 0.200 0.400 0.200 0.00

R43. Research grants from regional and local institutions 0.377 0.114 0.188 0.321 0.02

R44. Research grants from the university 0.472 0.108 0.256 0.164 0.02




Volume 18 · Number 4 · 2004 · 224-242

236



model. The evaluation and choice module

provides five different graphical modes for

performing sensitivity analysis, namely:

performance, dynamic, gradient, two-dimensional

plot, and differences. Each of these graphical

modes provides a different viewpoint to a

sensitivity analysis. Under any of these five modes,

the user can easily manipulate criterion priorities

and immediately see the impact of the changes (as

reflected in the ranking of alternatives). A decision

maker can easily use the mouse to change any of

the weights of the criteria and observe the

corresponding changes in the weights of the

alternatives and their graphical display. Thisfeature makes it possible and easy to perform

several “what-if” analyses once the model is

created and tested.

Since they are all interrelated, the resultant

changes can be observed in these elements.

Figure 3 and Figure 4 are graphical

representations of this analysis. There are several

sensitivity analysis procedures available.

The “dynamic sensitivity” is used to

dynamically change the priorities of the objectives

to determine how these changes affect the

priorities of the alternative choices. By dragging

the objective’s priorities back and forth in the left

column, the priorities of the alternatives will

change in the right column. If a decision maker

thinks an objective might be more or less

important than originally indicated, the decision

maker can drag that objective’s bar to the right or

left to increase or decrease the objective’s priority

and see the impact on alternatives. They provide a

visual representation of the percentage of

importance of each criterion in each alternative.

This kind of analysis could be carried out on any

node in the hierarchy. The figure represents the

sensitivity analysis for the three main categories of research, teaching, and service. For example, if the

priority of teaching is increased, the most preferred

candidate becomes AGR, and so on.

The “performance sensitivity” analysis shows

how the alternatives were prioritized relative to

other alternatives with respect to each objective as

well as overall. The “gradient sensitivity” analysis

shows the alternatives’ priorities with respect to

one objective at a time. The “head-to-head

sensitivity” analysis shows how two alternatives

compared to one another against the objectives in a

Table V .Ideal mode scores for candidates with respect to the teaching component and inconsistencies


Final Scores (TEACHING) 0.192 0.145 0.246 0.417

T1. Diversification in teaching 0.282 0.207 0.255 0.255

T11. Number of courses (credit hours) taught 0.286 0.143 0.286 0.286 0.00

T12. Average number of preparations per semester 0.250 0.250 0.250 0.250 0.00

T13. Number of sections of the same course taught 0.395 0.140 0.232 0.232 0.02

T131. Average number of students per semester 0.395 0.140 0.232 0.232 0.02

T2. Efforts in developing courses

and methods of teaching 0.131 0.130 0.320 0.419

T21. Theoretical courses developed 0.163 0.163 0.395 0.278 0.02

T22. Lab courses developed 0.075 0.075 0.517 0.333 0.01

T23. Books translated for the purpose of teaching 0.200 0.200 0.200 0.400 0.00

T24. Web-based teaching material developed 0.051 0.50 0.308 0.591 0.04

T25. Other media productions for teaching 0.114 0.110 0.430 0.347 0.02

T251. Software program development for teaching 0.116 0.100 0.549 0.235 0.02

T252. CD ROM development for teaching 0.109 0.109 0.209 0.572 0.00

T253. Developing other materials for teaching 0.108 0.164 0.256 0.472 0.02

T3. Student evaluation of teaching 0.115 0.076 0.193 0.616 0.02

T4. Course files and exams 0.221 0.115 0.238 0.427

T41. Completeness of the course files 0.232 0.140 0.232 0.395 0.02T42. Design/organization of course files 0.189 0.109 0.351 0.351 0.00

T43. Quality of design/preparation of exams 0.230 0.097 0.179 0.493 0.04

T5. Teaching portfolio 0.125 0.076 0.249 0.551

T51. Quality/design of the teaching portfolio 0.127 0.075 0.249 0.549 0.01

T52. Documentation of the teaching portfolio 0.110 0.083 0.249 0.558 0.04

T6. Contribution in conferences/

workshops related to teaching 0.245 0.245 0.255 0.255

T61. Organizing conferences/workshops related to teaching 0.250 0.250 0.250 0.250 0.00

T62. Participation in international conferences/workshops

Related to teaching 0.250 0.250 0.250 0.250 0.00

T63. Participation in regional and local conferences/workshops

Related to teaching 0.167 0.167 0.333 0.333 0.00




Volume 18 · Number 4 · 2004 · 224-242

237



decision. The “two-dimensional (2D plot)

sensitivity” analysis shows the alternatives’

priorities with respect to two objectives at a time.

Members of the Provost committee requested

looking at a large number of changes in the priority

scores (weights) for each category (and later

several subcategories). After performing these

detailed sensitivity analyses, the committee wrote

their final recommendation to the Provost. In their

report, they included their observations with

regard to effect of weight changes on the

performance of each candidate.

Conclusion

The structured approach offered by the AHPallows different individuals to participate equally

in the decision-making process. The analytical

process can provide a critical link of developing

trust and true group participation. The AHP

allows diverse viewpoints to be considered and

integrated. The important thing is that all

participants have input to, and ownership of, the

final evaluation.

Offering awards in institutions of higher

education, no matter what the nature of the awards

is, can be a complex, multi-faceted, judgmental

process, and requires the participation of

committee members in most cases. It is important

that the decision-making process is rational,

consistent, and defensible. The AHP may offer an

opportunity to integrate all these issues to provide

a true mechanism that offers objectivity and

understanding.

The AHP incorporates qualitative factors and

academic judgments in rating candidates for

awards. Thus, criteria affected by issues that

cannot be quantified in a point scoring system will

be ranked more suitably by the AHP. When

nominating a candidate for a certain award, some

criteria are quantitative (such as the number of

publications in a journal), and others are

qualitative (such as the documentation of the

teaching portfolio). In AHP, the criteria are

specified in the decision hierarchy and are notrestricted in any way. One criterion can also be

divided into sub-criteria. The requirement to

construct a decision hierarchy provides the

additional advantage of diminishing the chance

that an important criterion will be forgotten.

In the AHP, the number of new scores required

can be minimized if the decision maker imposes

“complete consistency” on the judgments. This

feature of AHP, calculating inconsistency ratios, is

an important feature that assures that the decision-

making process offers results that cannot be

Table VI Ideal mode scores for candidates with respect to the service component and inconsistencies


Final scores (SERVICES) 0.329 0.112 0.231 0.329

S1. University service 0.384 0.169 0.226 0.222

S11. Committee memberships 0.388 0.192 0.212 0.208

S111. Participation in university-level committees 0.606 0.068 0.167 0.159 0.03

S112. Participation in college-level committees 0.140 0.395 0.232 0.232 0.02

S113. Participation in department-level committees 0.098 0.209 0.346 0.346 0.02

S12. Contributions to conferences/seminars 0.371 0.102 0.266 0.261

S121. Lecturing in seminars at the university 0.395 0.140 0.232 0.232 0.02

S122. Contributions in organizing workshops at the university 0.134 0.106 0.487 0.273 0.01

S123. Official work performed as requested by college 0.451 0.119 0.261 0.169 0.03

S124. Representing the university in regional meetings 0.286 0.097 0.182 0.435 0.03

S125. Extra-curricular activities performed 0.097 0.092 0.324 0.488 0.03

S126. Administrative positions held at the university 0.730 0.061 0.104 0.104 0.02

S2. Community service 0.304 0.085 0.233 0.378

S21. Contributions in local conferences 0.151 0.046 0.270 0.533 0.04

S22. Community shows/articles 0.328 0.170 0.188 0.314

S221. Regional/local newspaper articles 0.451 0.119 0.169 0.261 0.03

S222. Regional/local magazine articles 0.391 0.276 0.138 0.195 0.05

S223. Radio/TV productions and shows appearances 0.271 0.120 0.191 0.418 0.03S224. Artistic performances and shows 0.250 0.250 0.250 0.250 0.00

S23. Consultations/trainings 0.313 0.139 0.141 0.407

S231. Consultations provided to government 0.340 0.048 0.115 0.496 0.03

S232. Consultations provided to private firms 0.318 0.064 0.145 0.474 0.03

S233. Organizing special trainings for local firms 0.261 0.451 0.169 0.119 0.03

S24. Empirical research with other institutions 0.321 0.058 0.181 0.440 0.03

S25. Chairing scientific societies 0.184 0.047 0.507 0.263 0.04

S26. Memberships in scientific societies 0.483 0.088 0.272 0.157 0.01




Volume 18 · Number 4 · 2004 · 224-242

238



disputed by others interested in the outcome. The

AHP gives some structure to this award process.

The decision maker should consider the

defensibility of his or her scores as part of the

process. In other words, the framework is robust

enough to easily facilitate issues such as

inconsistencies in judgments and conflict

resolution, if any, within and among groups.

An ancillary result of using the AHP in support

of this award decision is that it exposes members of

the committee to a structured approach to decision

making. The AHP provides a realistic description

of the award problem. The award problems are

characterized by multiple goals, sub-goals, and

priority weights. AHP provides the opportunity to

deal with all of these aspects by incorporating them

in the decision hierarchy. AHP assumes that all

criteria are independent, which precludes

interactions between the criteria. AHP assumes

that a decision maker can compare two candidates

Figure 3 AHP and sensitivity analysis




Volume 18 · Number 4 · 2004 · 224-242

239



on a specific criterion without considering theother criteria.

Structuring can refer to the construction of the

model as well as to the use of this model. AHP

clearly structures the process of establishing

priorities. Although AHP presumes that the phases

are executed sequentially, the decision maker can

return to a previous phase in order to make some

changes. Because of the flexibility of the method it

is not necessary to repeat all judgments when a

change is made. Changes in the model, such as the

addition of an alternative or criterion, have only a

limited impact on other parts of the model.

Selecting candidates for excellence awardsinvolves many factors; therefore, decision makers

should be able to state differences in the relative

importance of the criteria. AHP includes criteria at

one or more levels in the decision hierarchy. All

elements in a certain level have to be compared

pairwise in order to calculate the importance of the

criteria. In this way, it is very easy to assign

different values to the importance of different

criteria.

A method for award decision making should

provide the selecting committee with the

opportunity to perform a number of analyses.Sensitivity analyses and what-if analyses show the

consequences of changes in, for example, the

importance of factors. Confidence in the outcome

of the analysis will increase if small changes in the

relative importance of factors do not have much

impact on the overall priority rating. When the

AHP analysis has been completed it is rather easy

to determine the consequences of changes in the

judgments on the overall priorities.

A method is only a useful support tool if it is

easy to understand. Selecting committee members

have to understand how the method derives the

overall priorities. The comprehensibility of AHP isincreased by both the construction of the decision

hierarchy and the subsequent pairwise

comparisons. The understandability is, however,

somewhat limited by the complex calculations

based on the eigenvectors and eigenvalues of the

pairwise comparison matrices.

The AHP method is easy to use without

elaborate training. As explained, AHP analysis can

be divided into distinct phases. These phases

require decision makers to think in a structured

way. They have to define the problem in terms of

Figure 4 AHP and sensitivity analysis




Volume 18 · Number 4 · 2004 · 224-242

240



goals, criteria (sub-criteria) and alternatives. Next,

AHP requires them to make every possible

pairwise comparison on a certain level. This

enables the computation of the consistency ratio.

Finally, using an AHP model is quite easy provided

that an AHP software package is available.

References

Alpert, D. (1985), “Performance and paralysis: the organizationalcontext of the American research university”, Journal of Higher Education , Vol. 56 No. 3, pp. 241-81.

Barnett, L. (1996), “Are teaching evaluation questionnairesvalid? Assessing the evidence”, Journal of Collective Negotiations , Vol. 25 No. 4, pp. 335-49.

Braskamp, L. (1983), Guidebook for Evaluating Teaching ,Measurement and Research Division, Office of Instructional Resources, University of Illinois, Urbana, IL.

Bray, J. and Howard, G. (1980), “Methodological considerationin the evaluation of a teacher-training program”, Journal of Educational Psychology , Vol. 72 No. 1, pp. 62-70.

Brightman, H. (1987), “Towards teaching excellence in thedecision sciences”, Decision Sciences , Vol. 18 No. 4,pp. 646-61.

Brightman, H., Elliot, M. and Bhada, Y. (1993), “Increasing theeffectiveness of student evaluation of instructor datathrough a factor score comparative report”, Decision Sciences , Vol. 24 No. 1, pp. 192-9.

Caroll, J. (1980), “Effects of training programs for universityteaching assistants”, Journal of Higher Education , Vol. 51,pp. 167-83.

Centra, J. (1972), The Utility of Student Ratings for Instructional Improvement , Project Report 72-16, Educational TestingServices, Princeton, NJ.

Clement, R. and Stevens, G. (1989), “Performance appraisal in

higher education: comparing departments of managementwith other business units”, Public Personnel Management ,Vol. 18 No. 3, pp. 263-78.

Cohen, P. (1980), “Effectiveness of student rating feedback forimproving college instruction: a meta-analysis of findings”, Research in Higher Education , Vol. 13,pp. 321-41.

Cohen, P. and Herr, G. (1982), “Using an interactive feedbackprocedure to improve college teaching”, Teaching of Psychology , Vol. 9 No. 3, pp. 138-40.

Collison, F. (1990), “Transportation and logistics programs:criteria for faculty hiring and retention”, Transportation Journal , Fall, pp. 44-53.

Costin, F. (1968), “A graduate course in the teaching of psychology: description and evaluation”, Journal of

Teacher Education , Vol. 19 No. 4, pp. 425-32.Cotton, C., McKenna, J., Van Auken, S. and Yeider, R. (1993),

“Mission orientations and deans’ perceptions:implications for the new AACSB accreditation standards”,Journal of Organizational Change , Vol. 6 No. 1, pp. 17-27.

Crites, J. (1965), “Test review-Watson-Glaser critical thinkingappraisal”, Journal of Counseling Psychology , Vol. 12No. 3, pp. 328-30.

Dagani, R. (1993), “University of Utah’s Robert W. Parry wins1993 Priestly Medal”, Chemical & Engineering News ,Vol. 70 No. 21, pp. 21-3.

Daniels, J. (1970), “Effects of interaction analysis upon teachingassistants and student achievement in introductory collegemathematics”, Doctoral dissertation, Dissertation Abstract

International , Vol. 31, pp. 2768-9, Indiana University,Bloomington, IN.

Dornbusch, S. (1979), “Perspectives from sociology:organizational evaluation of faculty performance”, inLewis, D.R. and Becker, W.E. Jr (Eds), Academic Reward/ Awards in Higher Education , Ballinger, Cambridge, MA.

Earls, A. (1997), “The top 25 Techno-MBAs”, Computerworld ,Vol. 31 No. 20, pp. 84-7.

Ehie, I. and Karathanos, D. (1994), “Business facultyperformance evaluation based on the new AACSBaccreditation standards”, Journal of Education for Business , Vol. 67 No. 1, pp. 257-62.

Farh, J., Werbel, J. and Bedeian, A. (1988), “An empiricalinvestigation of self-appraisal-based performanceevaluation”, Personnel Psychology , Vol. 41, pp. 141-56.

Feldman, K. (1977), “Consistency and variability among collegestudents in rating their teachers and courses: a review andanalysis”, Research in Higher Education , Vol. 6,pp. 233-47.

Gage, D. (1993), “The winds of change: improving and assessingteaching quality”, SAM Advanced Management Journal ,Vol. 58 No. 2, pp. 4-10.

Hamilton, A. (1995), “Special: Cummings Award lecture 1948 –40 years in the poisonous trade”, American Industrial Hygiene Association Journal , Vol. 56 No. 5, pp. 423-31.

Helms, M., Williams, A. and Nixon, J. (2001), “TQM principlesand their relevance to higher education: the question of tenure and post-tenure review”, The International Journal of Education Management , Vol. 15 No. 7, pp. 322-31.

Henninger, E. (1998), “Perceptions of the impact of the newAACSB standards on faculty qualifications”, Journal of Organizational Change Management , Vol. 11 No. 5,pp. 407-24.

Hoyt, D. (1974), “Interrelationships among instructionaleffectiveness, publication record and monetary reward/award”, Research in Higher Education , Vol. 2, pp. 81-9.

Jolson, M. (1974), “Criteria for promotion and tenure: a faculty

view: research note”, Academy of Management Journal ,Vol. 17, pp. 149-54.

Katz, D. (1973), “Faculty salaries, promotions, and productivityat a large university”, American Economic Review , Vol. 63No. 3, pp. 469-77.

Kiesler, S. and Sproull, L. (1987), Computing and Change on Campus , Cambridge University Press, Cambridge.

Leatherman, C. (2000), “Despite their gripes, professors aregenerally pleased with careers, poll finds”, The Chronicle of Higher Education , 3 March, p. A19.

Levinson-Rose, J. and Menges, R. (1981), “Improving collegeteaching: a critical review of research”, Review of Educational Research , Vol. 51, pp. 403-34.

Lewis, D. and Orvis, C. (1973), “A training system for graduatestudent instructors of introductory economics at theUniversity of Minnesota”, Journal of Economic Education ,Vol. 5, pp. 38-46.

Liberatore, M., Nydick, R. and Sanchez, P. (1992), “The

evaluation of research papers (or how to get an academiccommittee to agree on something)”, Interfaces , Vol. 22No. 2, pp. 92-100.

Lischwe, S., Manning, O. and Williman, J. (1987), “Encouragingthe research process: SIUE’s experience grantsdevelopment seminar for faculty”, Journal of the Society of Research Administrators , Vol. 18 No. 3, pp. 49-53.

Ludesh, S. (1987), “Microcomputer use in higher education:summary of a survey”, Education Bulletin , Vol. 22,pp. 13-17.




Volume 18 · Number 4 · 2004 · 224-242

241



Luthans, F. (1967), The Faculty Promotion Process , Bureau of Business and Economic Research, University of Iowa, IowaCity, IA.

McCarthy, M. (1980), “Continuing education service as acomponent of faculty evaluation”, Lifelong Learning: The Adult Years , 3 May, pp. 8-11.

McFerron, R., Camp, C., Lynch, M. and Woods, L. (1996), “Tenurestandards within state-wide systems of higher education:

the collective bargaining milieu”, Journal of Collective Negotiations inthe Public Sector , Vol. 25 No. 4, pp.365-75.

McKenna, J., Cotton, C. and Van Auken, S. (1995), “Businessschool emphasis on teaching, research and service toindustry: does where you sit determine where youstand?”, Vol. 8 No. 2, pp. 3-16.

Marsh, H. and Dillon, K. (1980), “Academic productivity andfaculty supplemental income”, Journal of Higher Education , Vol. 51, pp. 546-55.

Mesak, H. and Jauch, L. (1991), “Faculty performanceevaluation: modeling to improve personnel decisions”,Decision Sciences , Vol. 22 No. 5, pp. 1142-57.

Millman, J. (1981), Handbook of Teacher Evaluation , Sage,Beverly Hills, CA.

Murphy, M. (1973), “The development and assessment of an

experimental teacher-training program for beginninggraduate assistants in chemistry”, Dissertation Abstract International , Vol. 33, p. 4223, (Doctoral dissertation, OhioState University, Columbia, OH, 1972).

Nevison, C. (1980), “Effects of tenure and retirement policies onthe college faculty”, Journal of Higher Education , Vol. 51No. 2, pp. 150-66.

Remus, W. (1977), “Strategies for the publish or perish world; orwhy journals are unreadable”, Interfaces , Vol. 8 No. 1,pp. 64-9.

Saaty, T. (1994), “How to make a decision: the analytic hierarchyprocess”, Interfaces , Vol. 24 No. 6, p. 9-26.

Saaty, T. and Ramanujam, V. (1983), “An objective approach tofaculty promotion and tenure by the analytic hierarchyprocess”, Research in Higher Education , Vol. 18 No. 3,pp. 311-31.

Salthouse, T., McKeachie, W. and Lin, Y. (1978), “Anexperimental investigation of factors affecting universitypromotion decisions”, Journal of Higher Education , Vol. 49No. 2, pp. 177-83.

Seldin, P. (1984), Changing Practices in Faculty Evaluation: ACritical Assessment and Recommendations for Improvement , Jossey-Bass, San Francisco, CA.

Siegfried, J. and White, K. (1973), “Teaching and publishing asdeterminants of academic salaries”, Journal of Economic Education , Vol. 4, pp. 90-8.

Sowell, T. (1997), “Glimpses of academe”, Forbes , Vol. 159No. 4, pp. 59-60.

Srinivasan, V. and Basu, A. (1989), “The metric quality of orderedcategorical data”, Marketing Science , Vol. 8, pp. 205-30.

Stark, B. and Miller, T. (1976), “Selected personnel practices

relating to research and publication among managementfaculty: research note”, Academy of Management Journal ,Vol. 19 No. 3, pp. 502-5.

Tiberius, R., Sackin, H., Slingerland, J., Tubas, K., Bell, M. andMatlow, A. (1989), “The influence of student evaluativefeedback on the improvement of clinical teaching”,Journal of Higher Education , Vol. 60 No. 6, pp. 665-81.

Tubb, G. (1975), “Heuristic questioning and problem-solvingstrategies in mathematics teaching assistants and their

students”, Dissertation Abstract International , Vol. 36,pp. 235-6A, (Doctoral dissertation, Texas A&M University,College Station, TX, 1974).

Tuckman, H. (1979), “The academic reward/award structure inAmerican higher education”, in Lewis, D.R. and Becker,W.E. Jr (Eds), Academic Reward/Awards in Higher Education , Ballinger, Cambridge, MA.

Tuckman, H. and Hagemann, R. (1976), “An analysis of the

reward/award structure in two disciplines”, Journal of Higher Education , Vol. 47 July/August, pp. 447-64.

Tuckman, H., Gapinski, J. and Hagemann, R. (1977), “Facultyskills and the salary structure in academe: a marketperspective”, American Economic Review , Vol. 67,pp. 692-702.

Yaghlian, N. (1973), “University teaching: the impact of an in-service program for teaching fellows in chemistry”,Dissertation Abstract International , Vol. 33, p. 6192A,(Doctoral dissertation, University of Michigan, Ann Arbor,MI, 1972).

Zoffer, H. (1978), “The consummate faculty person”, Academy of Management Review , pp. 901-6.

Further reading

Crowe, T., Nobles, J. and Machimada, J. (1998), “Multi-attributeanalysis of ISO 9000 registration using AHP”, International Journal of Quality & Reliability Management , Vol. 15 No. 2,pp. 205-22.

Doost, R. (1997), “Faculty evaluation: an unresolved dilemma?”,Managerial Auditing Journal , Vol. 12 No. 2, pp. 98-104.

Jauch, L. and Glueck, W. (1975), “Evaluation of universityprofessors’ research performance”, Management Science ,Vol. 22 No. 1, pp. 66-75.

Karpetrovic, S. and Willborn, W. (1999), “Holonic model for

quality system in academia”, International Journal of Quality & Reliability Management , Vol. 16 No. 5,pp. 457-84.

Kasten, K. (1984), “Tenure and merit pay as reward/awards forresearch, teaching, and service at a research university”,Journal of Higher Education , Vol. 55 No. 4, pp. 500-14.

Lam, K. and Zhao, X. (1998), “An application of quality functiondeployment to improve the quality of teaching”,International Journal of Quality & Reliability Management ,Vol. 15 No. 4, pp. 389-413.

McIntosh, T. and Van Kovering, T. (1986), “Six-year case study of faculty peer reviews, merit ratings, and pay awards in amultidisciplinary department”, The Journal , Spring,pp. 5-13.

Magnusen, K. (1987), “Faculty evaluation, performance, and

pay”, Journal of Higher Education , Vol. 58 No. 5,pp. 516-29.

Tang, T. (1997), “Teaching evaluation at a public institution of higher education: factors related to the overall teachingeffectiveness”, Public Personnel Management , Vol. 26No. 3, pp. 379-89.

Webster, C. (1990), “Evaluation of marketing professors: acomparison of students, peer, and self-evaluation”,Journal of Marketing Education , Spring, pp. 11-17.




Volume 18 · Number 4 · 2004 · 224-242

Documents

Badri_Awards of excellence in institutions of HE.pdf