The Determinants and Performance Effects of Supervisor Bias

Jasmijn C. Bol

University of Illinois at Urbana-Champaign

bol@uiuc.edu

January 2008

Abstract

This paper examines the determinants and performance effects of leniency and centrality bias.

An empirical analysis of a compensation plan for low-level employees with both objective and

subjective performance measures leads to two key results. First, the causes of supervisor bias

include: employee performance, the difference between the organizational level of the supervisor

and the employee, the financial position of the firm, the duration of the employee-supervisor

relationship, and supervisor characteristics. This indicates that supervisors take their own

personal preferences into account when appraising employee performance. Second, supervisor

bias affects future employee incentives. Contrary to previous assumptions, the results show that

biased performance ratings can have both positive and negative effects on incentives. Leniency

bias positively affects performance improvement, while centrality bias has a negative effect on

performance.

Acknowledgements: I want to thank Stan Baiman, Antonio Dávila, Henri Dekker, Chris Ittner,

Frank Moers, Fernando Peñalva, Joan Enric Ricart, Stan Veuger and seminar participants at the

2006 Global Management Accounting Research Symposium in Copenhagen, the 2006 AAA

Annual Meeting in Washington D.C., the University of Pennsylvania, the London School of

Economics, the University of Colorado at Boulder, the University of Illinois at Urbana-

Champaign, the London Business School, the Harvard Business School and the University of

Southern California for their comments and suggestions.

1

I. INTRODUCTION

This paper provides empirical evidence on the determinants and performance effects of

supervisor bias. Knowledge of supervisor bias is important for managers in charge of incentive

design. Understanding the causes of supervisor bias, and the consequences of supervisor bias on

future performance, can help compensation system designers make better-informed decisions

about how much supervisor discretion to allow for in performance-based compensation plans,

thereby improving incentive contracting.

In this paper, I focus on two well-known forms of supervisor bias: leniency bias and

centrality bias (Prendergast, 1999). Leniency bias is the tendency to provide employees with

inflated subjective performance ratings (Bretz, Milkovich, & Read, 1992), while centrality bias is

the tendency to compress performance ratings, creating less variance in performance ratings than

in actual performance. Although there is considerable empirical evidence indicating that the use

of subjective performance measures leads to lenient and compressed ratings (Landy & Farr,

1980; Murphy & Cleveland, 1991; Bretz et al., 1992; Prendergast & Topel, 1993; Jawahar &

Williams, 1997; Prendergast, 1999; Moers, 2005), empirical evidence on how supervisor bias

actually influences employee behavior is almost entirely absent. This paper extends the current

literature by focusing both on what causes a supervisor to bias performance ratings and on how

biased ratings affect the incentive effects of performance-based compensation contracts.

I study a compensation plan for low-level employees of a financial service provider. The

incentive system includes both objective and subjective performance measures and gives

supervisors some freedom to make ex post modifications to financial rewards. I use contract

design and performance data from five branch offices for 2003 and 2004. I begin my analysis by

establishing that the performance ratings on the subjective dimension are subject to leniency and

2

centrality bias. I then examine why the performance ratings of some employees are subject to

more leniency and centrality bias than those of others. Finally, I examine how these biases affect

performance change between 2003 and 2004.

The results indicate that supervisors do not behave as pure contract-driven robots when

appraising performance. I find that differences in organizational level between employee and

supervisor, and the financial performance of the company, are determinants of leniency and

centrality bias. This is consistent with supervisors taking information-gathering costs and

attitudes towards rating accuracy into consideration when rating behavior. Supervisor

characteristics, the duration of the employee-supervisor relationship, and employee performance

influence the extent of leniency bias, suggesting that supervisors consider the cost of

communicating evaluations and how employee ratings reflect on their own management

capabilities.

Finally, performance ratings influence incentive provision. Leniency bias has a positive

effect on performance improvement, while centrality bias has a negative effect, for both above-

average and below-average performers.

The study makes several important contributions to the performance evaluation and

compensation literature. This is the first paper that empirically investigates how biased

performance ratings influence the effectiveness of a performance-based compensation contract.

Prior studies have established the existence of leniency and centrality bias, but not the

consequences of these biases on future performance. Moreover, this paper does not only present

evidence indicating that the use of subjective performance measures leads to biased performance

ratings; it also provides insights into the determinants of supervisor bias. Finally, this paper

offers a detailed description of the salient features of an incentive system for low-level

3

employees and of the supervisors’ and employees’ reactions to this system. This gives us insight

into the compensation practice for low-level employees, an area in which detailed information is

scarce (Indjejikian, 1999).

This paper consists of five sections. Section II reviews research on supervisor bias in

compensation contracts and develops the hypotheses. The research design is presented in section

III and in section IV the results are analyzed. The final section summarizes the results and

discusses future research possibilities.

II. HYPOTHESES

Most papers dealing with subjectivity in compensation contracts focus on the

determinants of subjectivity. These studies examine the role of subjectivity in incentive systems

and indicate how introducing subjectivity improves contracts by mitigating incentive distortions

or reducing risk (Baker, Gibbons, & Murphy, 1994; Baiman & Rajan, 1995; Hayes & Schaefer,

2000; Ittner, Larcker, & Meyer, 2003; Gibbs, Merchant, Van der Stede, & Vargus, 2004).

However, the fact that performance evaluation is subject to supervisor discretion can also give

rise to a number of problems, the most prominent of which is rating inaccuracy.1

Since the correctness of subjective performance evaluations cannot be assessed by

outside parties, supervisors can take their own preferences into consideration when rating

employee performance. One documented consequence of allowing supervisors to apply

discretion is biased performance ratings (Landy & Farr, 1980). A long line of research, mainly in

human resources management, investigates performance rating accuracy in subjective

1 Inaccurate performance ratings are clearly not the only potential negative consequence of introducing subjectivity

into explicit compensation contracts. Other concerns, such as unclear measurements criteria, are also likely to

influence the effectiveness of the compensation contract. In this paper, I limit the discussion to examining the effects

of biased performance ratings.

4

performance appraisal and presents considerable evidence on biased performance ratings (e.g.,

Murphy & Cleveland, 1991; Bretz et al., 1992; Jawahar & Williams, 1997; Moers, 2005). These

papers find that supervisors tend to rate leniently and that they do not differentiate strongly

between employees when evaluating performance.

The extent to which ratings are biased is not the same for different supervisors or for

different employees evaluated by the same supervisor. Supervisors are predicted to vary the

amount of bias applied to the performance ratings depending on how rating bias influences their

own utility.2 In determining the extent of bias, supervisors consider the potential negative

consequences of communicating performance ratings. Communicating harsh but accurate ratings

to employees will likely damage personal relationships and lead to discussions and criticism. By

offering employees lenient and compressed ratings, supervisors can reduce the real and

psychological cost of communicating evaluations (McGregor, 1957; Bernardin & Buckley,

1981). This leads to the following hypotheses:

Hypothesis 1a: The costs of communicating evaluations have a positive effect on the extent of

leniency bias that is applied to performance ratings.

Hypothesis 2a: The costs of communicating evaluations have a positive effect on the extent of

centrality bias that is applied to performance ratings.

The extent of bias that supervisors apply to performance ratings is also influenced by the

time and effort needed to gather the appropriate performance information. High information-

gathering costs will make supervisors less willing to invest the required time in information

collection (Harris, 1994). This is predicted to lead to leniency and centrality bias, because

2 In this paper I focus on incentive-consistent bias by supervisors. However, supervisors can also unconsciously bias

performance ratings due to their cognitive limitations. See, for example, Lipe and Salterio (2000) and O’Connor,

Deng and Shields (2006).

5

biasing imprecise performance evaluations reduces the chance that resulting imprecise ratings

lead to painful discussions and dissatisfied employees. This leads to the following hypotheses:

Hypothesis 1b: Information-gathering costs have a positive effect on the extent of leniency bias

that is applied to performance ratings.

Hypothesis 2b: Information-gathering costs have a positive effect on the extent of centrality

bias that is applied to performance ratings.

Since supervisors’ compensation and promotion possibilities are often linked to employee

performance, supervisors are also expected to take the effects of their rating decisions on

employee incentives into account (Prendergast & Topel, 1993), as well as how the performance

ratings reflect on the supervisors’ own management capabilities (Longenecker, Sims, & Gioia,

1987). By rating poorly performing employees more leniently, supervisors may try to hide

departmental problems in order to come across as more capable managers.

Hypothesis 1c: Employee performance has a negative effect on the extent of leniency bias that

is applied to performance ratings.

Supervisors are also predicted to take their superiors’ attitude towards rating accuracy

into account. Supervisors adjust their biasing behavior based on the expected rewards for

accurate ratings and/or the expected consequences of being perceived as a biased rater by their

own superiors (Longenecker et al., 1987). This results in the following hypotheses:

Hypothesis 1d: A positive attitude of the supervisor’s superiors towards rating accuracy has a

negative effect on the extent of leniency bias that is applied to performance

ratings.

Hypothesis 2d: A positive attitude of the supervisor’s superiors towards rating accuracy has a

negative effect on the extent of centrality bias that is applied to performance

ratings.

6

Finally, rating behavior is influenced by the utility supervisors get out of (dis)favoring

employees (Prendergast & Topel, 1996). Favoring and disfavoring certain employees leads to

biased ratings, although favoritism does not necessarily lead to leniency and centrality bias.

Employee Incentives and Supervisor Bias

Performance ratings are an important element of incentive contracts because they

summarize the outcome of the performance evaluation process and link performance to pay by

indicating how much compensation the employee will receive. Bias in performance ratings is

likely to influence the incentives created by the compensation contract. In this paper I identify

three ways in which bias in performance ratings is expected to influence employee incentives: 1)

through the effect bias has on the link between pay and performance, 2) through the effect bias

has on the perceived fairness of the compensation system, and 3) through the effect bias has on

the performance information received by the employees.

The Link between Pay and Performance

One of the main objectives of using a performance-based compensation system is

motivating employees to supply effort on the right job dimensions.3 According to agency theory,

linking pay to performance motivates employees to exert increased effort to improve

performance, because increased performance results in increased pay (Holmstrom, 1979).

Performance measures play a crucial role in this process because employees direct their attention

to those actions that are measured.

However, the incentive effect of a compensation plan does not depend solely on what is

measured; it also depends on how these actions are measured. Employees will not be motivated

3 Another important objective of performance measurement is to differentiate between highly skilled and less skilled

employees. Although the effect of less than optimal personnel decisions might be severe, the aim of this paper is not

to investigate the effect of bias on selection issues.

7

to increase effort unless improved performance is actually expected to translate into more

compensation.4 When performance is assessed subjectively, this may not take place consistently.

Bias clouds the link between pay and performance, affecting the incentive provision of the

compensation plan.

The Perceived Fairness of the Compensation System

Employee incentives are not solely determined by the relation between pay and

performance. The extent to which employees are motivated by a compensation system is also

influenced by the perceived fairness of the compensation plan (Akerlof & Yellen, 1988; Blinder

& Choi, 1990; Colquitt, Conlon, Wesson, Porter, & Yee Ng, 2001). Employees not only care

about how their rating compares to their performance, but also about how the received

performance rating compares to their expectations and to the ratings received by others. In the

organizational justice literature, two types of subjective perceptions of fairness are distinguished:

the fairness of the outcome distributions, or distributive justice, and the fairness of the

procedures used to determine these outcome distributions, or procedural justice (Greenberg,

1990). Inaccuracies in performance ratings caused by supervisor bias are predicted to influence

the perceived fairness of the compensation system as bias changes the outcome distribution of

the compensation plan.

Performance Information

Compensation systems also influence employee incentives through their effect on self-

perception. Performance ratings provide employees with information about how their

performance is perceived by the supervisor. This information influences employee incentives by

affecting the employee’s self-perceived marginal productivity of effort (Fang & Moscarini,

4 In expectancy theory (Vroom, 1964; Heneman & Schwab, 1972) this is referred to as the degree to which

performance is instrumental for the attainment of certain outcomes, and research shows that it is important in

motivating individuals.

8

2002). Higher self-confidence enhances motivation by affecting the expected result of effort,

which provides supervisors with clear incentives to provide employees with performance ratings

that increase or maintain their self-esteem (Bénabou & Tirole, 2002; 2003).

The Performance Effects of Leniency and Centrality Bias

Supervisor bias is studied extensively in the HR literature in the hope of finding ways to

reduce it, under the maintained assumption that inaccurate ratings per se are bad for

compensation contracting (see Landy & Farr, 1980; Rynes, Gerhart, & Parks, 2005). In this

paper, I argue that this assumption may be incorrect. There are several ways in which bias is

expected to influence the functioning of performance-based compensation contracts and they are

not all negative. In the following sections I discuss how the two biases considered in this paper,

leniency and centrality bias, are predicted to influence employee incentives.

Leniency Bias

Higher ratings can affect incentive provision positively by increasing congruence

between performance rating expectations and received performance ratings. Individuals have a

tendency to overestimate themselves5 and therefore to rate themselves higher than their

supervisors do (McFarlane Shore & Thornton, 1986; Harris & Schaubroeck, 1988). Employees

who believe they have received a lower performance rating (and consequently less

compensation) than they deserve, are expected to lower their performance in order to restore a

feeling of equity (Akerlof & Yellen, 1988; Colquitt et al., 2001). More lenient performance

5 There exists abundant evidence in the psychology literature that most people overestimate their abilities and past

achievements (e.g. Larwood & Whittaker, 1977; Arkin, Cooper, & Kolditz, 1980), that they tend to recall their

success more than their failures (e.g. Mischel, Ebbesen, & Zeiss, 1976), and that they have the tendency to be

unrealistically optimistic about their future life events (e.g. Weinstein, 1980).

9

ratings may minimize these feelings of unfairness and the resulting negative effects on

performance.

Moreover, more lenient performance ratings can maintain or increase the employee’s

confidence in his ability and efficacy, which can be valuable to the firm because it increases the

employee’s motivation to undertake ambitious projects and persist even when faced with

adversity (Bénabou & Tirole, 2002).

On the other hand, since performance ratings give employees a signal about how their

performance is perceived by their supervisor, lenient ratings may lead employees to falsely

conclude that they are performing their tasks as desired by their supervisor. Providing employees

with this mistaken impression will have a negative effect on performance because it motivates

them to continue to take wrong/suboptimal actions. Hence, leniency bias is predicted to have a

positive effect on effort, but this will only lead to improved performance if the behavior that is

stimulated is desirable. Since supervisors are expected to be cautious about stimulating

undesirable actions, the overall effect of leniency bias is expected to be positive. I state the

hypothesis as follows:

Hypothesis 3: Leniency bias has a positive effect on the effectiveness of a compensation

contract as an incentive provider.

Centrality Bias6

The lack of distinction between performance ratings of different employees is expected to

influence employee incentives in several ways. First, above-average performers are likely to feel

disenchanted when employees who perform worse are rewarded almost equally. This is expected

to negatively affect above-average employees’ future performance (Lazear, 1991). Moreover,

6 Although both biases will move up ratings for below-average performance, centrality bias also decreases the

variance in reported employee performance.

10

compression reduces the probability that the value of a marginal performance rating increase will

outweigh the cost of the extra effort needed to improve performance sufficiently to receive this

marginal performance rating increase. Centrality bias is thus predicted to have a negative effect

on the incentives of above-average performers. This results in the following hypothesis:

Hypothesis 4: Centrality bias has a negative effect on the effectiveness of a compensation

contract as an incentive provider for above-average performers.

The situation is different for below-average performers, as compression influences their

ratings in a positive way. Because of centrality bias, their performance seems similar to that of

top performers. Since most employees consider their own performance to be above-average

(Meyer, 1975), providing below-average employees with their true comparative position will be

a deflating experience for most of them, which can have a disruptive effect on their performance

(Pearce & Porter, 1986; Gabris & Mitchell, 1988). Compressed ratings might therefore have a

positive effect on the incentives of below-average performers as they protect them from

information that will lower their self-perception and the perceived prospects from providing

effort in the future (Prendergast & Topel, 1993; Fang & Moscarini, 2002).

On the other hand, just as with above-average performers, the lack of variance in

performance ratings might also negatively affect the motivation of below-average performers as

their marginal cost to improve performance ratings may be higher than the marginal benefit they

receive from the performance rating improvement.

Since I cannot predict which of these arguments explains the relation between centrality

bias and the performance change of below-average performers, their relative importance must be

empirically determined.

11

III. RESEARCH DESIGN

Research Setting

In this paper I use data provided by a Dutch financial service provider (FSP). FSP is one

of the main financial service providers in the Netherlands, and it has a considerable stake in the

European market. It serves over nine million customers and its assets have a total value of some

€500 billion (in 2005).

Recently, competition in the financial sector has become more intense. More specialized

firms have managed to increase their marketshare substantially, jeopardizing FSP’s market

leading position in the Dutch market. In order to defend its market position, FSP decided to

move to more results-oriented management, to strengthen the relationships with current

customers and to acquire new customers. To accomplish these objectives FSP restructured its

front-offices, introduced a more pro-active way of approaching potential new customers and

designed a variety of new management control systems. As part of this initiative, FSP introduced

a new incentive system for its branch offices in 2003. The compensation plan covers all branch

office employees, except for the local management teams.

The New Compensation Contract

The new compensation system consists of a fixed salary and a bonus, where the fixed

salary depends on the employee’s job and the periodic fixed salary increase (up to a stated

maximum), and the bonus on the employee’s overall performance rating.7 In the incentive

system, performance is not only objectively measured but also subjectively assessed. More

specifically, half of the overall rating is determined by an output-based performance rating and

half by a competence-based performance rating.

7 Although the performance ratings are taken into account when making promotion decisions, the system was

designed to determine yearly compensation, not promotions.

12

The output-based section of the system consists of three to six subjective or objective

performance measures that measure specific employee output. Examples of objective

performance measures are: ‘the number of insurance policies sold,’ ‘the percentage increase in

portfolio growth,’ or ‘the number of appointments made.’ Examples of subjective outcome

performance measures are ‘the value of the front-office support,’ ‘the quality of the management

information reports,’ or ‘timely preparation of customer reports.’ The designers of the system

have provided the supervisors with examples of performance measures, but the supervisors

decide wich performance measures to use. 8

Although the intention of the new compensation system is to improve output, the

designers of the system wanted to avoid a blind focus on improving measured performance. The

compensation system therefore includes the subjective assessment of competencies that are

considered essential to performing the employee’s tasks in an efficient and correct manner. The

supervisor must choose between one and four competence measures, which are subjective

assessments of specific behavior that the employee must demonstrate. Hence, the system not

only indicates what type of output improvement is expected, it also informs the employees how

output improvements are expected to be accomplished. The competencies ‘cooperative behavior’

and ‘customer focus’ are included in all performance documents (see Figure 1).

The weight on each performance measure is determined by the number of measures

chosen within each section. Each section determines half of the overall rating and the weights of

the measures within each section are equal. The supervisor also sets specific targets and margins

8 Although supervisors individually select the performance measures and set the targets, before presenting the

contracts to the employees, all supervisors and management meet to discuss and compare the chosen performance

measures and targets. In this way FSP hopes to increase the consistency in the use of the system.

13

for each of the selected measures.9 At the end of the year, the supervisor evaluates performance

according to these measures and targets, and assigns a rating between one and five (not

necessarily integers) to each measure. This leads to an overall rating which determines a)

whether the employee receives the periodic fixed salary increase, and b) the bonus percentage

range. The employee only receives the periodic fixed salary increase if the overall rating is above

1.5.10

Additional Discretion in the Compensation System

The use of subjective performance measures is not the only way in which the FSP has

introduced supervisor discretion into its compensation system. Supervisors also have discretion

over the awarded bonus percentage. The overall rating is linked only to a bonus percentage

range,11

not to a concrete percentage and, if deemed necessary, supervisors are allowed to

provide employees with a bonus percentage outside the stated range. These discretionary bonus

adjustments are intended to ensure that important actions that are not foreseen ex ante can still be

rewarded.

In sum, the incentive system links pay to performance and puts part of the compensation

at risk. It does not rely exclusively on objective measures, but combines objective performance

measurement with subjective performance appraisal, thereby giving supervisors a relatively wide

scope for discretion.

The New versus the Old Compensation System

Although both the new and the old system consist of a fixed salary and possibly a bonus

and/or a periodic fixed salary increase, the new compensation system is different in two

9 Margins indicate when a performance target is not met, almost met, met, surpassed and generously surpassed

indicated by the scores ‘bad,’ ‘regular,’ ‘good,’ ‘very good’ and ‘excellent,’ respectively. 10

This is only true for employees who have not yet reached the maximum of their salary scale. 11

A ‘bad’ rating (0-1.5) corresponds to a bonus of 0%, a ‘regular’ rating (1.5-2.5) to 1-3%, a ‘good’ rating (2.5-3.5)

to 4-8%, a ‘very good’ rating (3.5-4.5) to 9-12% and an ‘excellent’ rating (4.5-5) to 13-15%. See Figure 1.

14

important ways. First, the old system did not use any objective performance measures; only

subjective assessments were used to evaluate performance. Second, in the old system a higher

performance rating did not consistently translate into a higher bonus percentage. In theory, there

was a link between pay and performance, where a higher performance rating should result in a

higher bonus. In practice, however, supervisors seemed to simply increase the bonus percentage

with one or two percentage points when performance was acceptable. This resulted in a situation

where some employees would, for example, get a bonus of four percent when rated ‘excellent’,

while others received fourteen percent when rated ‘good’.

The lack of a clear link between pay and performanc was the main reason for FSP to

abandon the old system. By creating a clear link, the new system is expected to increase

employee incentives and focus on output.12

The new system was not introduced to reduce

compensation costs: FSP increased employee compensation so that no employee would receive

less compensation after the introduction of the new system.

Introduction and Implementation of the New Compensation System

FSP took the introduction of the new compensation system very seriously. All employees

and supervisors received information (through presentations, folders, letters, and internet sites)

on the functioning of the system, its formal steps, and the timelines of its implementation. The

directors of the branch offices also wrote a formal letter to all employees indicating top

management’s support for the new system. Supervisors were provided with workshops which

explained how to select the performance measures, how to set the performance targets, and how

to rate performance. In order not to destroy the incentive effect of the system, as had happened

with the old system, supervisors where explicitly asked to rate performance accurately and to

12

An internal study that examined the consequences of the introduction of the new compensation system finds that

both supervisors and employees think that employee effort and focus on output have increased after the introduction

of the system.

15

uphold the link between pay and performance.13

Moreover, empirical evidence (e.g.

Longenecker et al., 1987; Burney, Henle, & Widener, 2006) indicates that attaching importance

to the implementation and use of a compensation system reduces the raters’ tendency to bias

performance ratings. The setting of this study is therefore biased against finding the existence of

supervisor bias, and therefore their determinants and consequences.

Data

The analyses employ two years of proprietary archival data on the incentive plans for

employees of five offices of FSP14

. I limit my study to employees who were employed by the

bank in both 2003 and 200415

and to departments that made significant use of objective

performance measures (at least 25%). This results in 396 complete performance documents, 198

from 2003 and 198 from 2004.16

The documents (see Figure 1) provide information about the chosen performance

measures, the targets, the ratings per measure, the total rating, and the old and new total

compensation (salary scale, periodic salary increase and bonus). Furthermore, the documents

indicate the employee’s job, department and supervisor, and an extensive description of the

chosen performance measures. Based on these descriptions I classify each output-based

performance measure as being either objective or subjective, while the competencies are by

13

Although the supervisors were asked to rate accurately, the concepts of leniency and centrality bias were not

explained and supervisors were not explicitly asked to refrain from applying these biases. 14

The five studied offices provide a representative mix of different regions, different office sizes and different types

of areas (ranging from rural to major cities). 15

Using employees that were employed by FSP in 2003 and 2004 might lead to selection bias because employees

that left the company in 2003 are not included in the sample. To examine this issue, I have asked the HR managers

of each office whether there were employees that had left the company because of the new compensation system.

Four of the five Hr managers said that this had not happened, while one HR-manager mentioned that there were

some instances of employees leaving because of the new system but that this was a very small percentage (less than

3%). 16

The data gathered does not cover all the employees meeting these conditions because some of the performance

documents were incomplete or were never received by the human resource department. I have checked whether

certain departments were underrepresented in the sample but found no such bias.

16

definition subjective. This results in two performance dimensions: objective and subjective.17

I

also obtained personal characteristics of the supervisors and employees such as gender and age,

and I interviewed several employees, supervisors, and designers of the system to get a better

understanding of the compensation system and the company background. Finally, I used data

provided by an internal study that examined employee and supervisor reactions to the

compensation system.

Summary statistics for the main variables used in this paper (see Tables 1, 2 and 3)

indicate that supervisors in 2003, on average, provided higher ratings on the subjective than on

the objective dimension. Both the mean (2.94 versus 2.80) and the median (3.00 versus 2.75)

were significantly higher in 2003 (p < 0.005 two-tailed). This relationship reversed in 2004; both

the mean (3.14 versus 3.28) and the median (3.13 versus 3.33) subjective rating were

significantly lower (p < 0.005 two-tailed). The descriptive statistics also show that the variance

in the objective ratings was significantly larger (p < 0.001 two-tailed) than the variance in the

subjective ratings in both 2003 and 2004. As predicted, the subjective ratings seem to be more

lenient (although only in 2003) and more compressed, suggesting the presence of leniency and

centrality bias. In the next section, I examine the existence of leniency and centrality bias more

formally.

17

For 63 percent of the employees in the sample the subjective dimension solely consists of competencies, while for

the remaining employees only one or two subjective output-based measures were chosen. I examine whether the

results presented in this paper are influenced by the type of subjective measures that are used (only competencies or

also subjective output-based measures), by re-estimating all models for only those employees that have no

subjective output-based measures and by re-estimating all models while excluding the subjective output-based

measures. In both cases the results are similar to those presented.

17

Leniency and Centrality Bias

In order to examine the determinants and performance effects of supervisor bias, I first

examine whether the performance ratings are subject to leniency and centrality bias, as predicted

by previous research. To detect leniency, I test whether performance ratings on the subjective

dimension are higher, on average, than performance ratings on the objective dimension, after

controlling for other relevant influences. Similar to Moers (2005), a higher score on the

subjective dimension is considered to be evidence consistent with more lenient ratings. Leniency

bias can only be adequately captured in this way if the performance ratings on the objective

dimension are unbiased or at least significantly closer to the true performance value than the

subjective ratings. Another underlying assumption is that employees have similar ability levels

on the two dimensions. After controlling for effort allocation effects, actual employee

performance on the objective and subjective dimension (as opposed to supervisors’ potentially

biased assessments) is therefore assumed to be very similar. There should thus, on average, be no

difference between the ratings on the different dimensions unless the ratings are biased, after

controlling for other relevant influences.

As the dependent variable, I use the employees’ performance ratings on the objective and

subjective dimensions (RATING). This means that each employee is included twice for each

year, once with his objective rating and once with his subjective rating. The influence of

discretion on the ratings is examined by including a dummy variable that equals one if the

observation is a subjective rating and zero otherwise (DSubjectivity). To account for differences in

contract design that may cause different effort allocations, I control for the contractual weight on

the specific dimension (WEIGHT), as a higher weight is expected to lead to higher effort and

consequently to higher ratings. I also include the number of performance measures (NR_PM), as

18

a larger number of performance measures can lead to incentive dilution. To control for year

effects, I include a year dummy (Y2003) that is one if the observation relates to 2003 and zero

otherwise.

In order to determine whether leniency bias is specific to a certain group of employees, or

is a more general consequence of using subjective performance measures, I include several

additional control variables. First, supervisors are not expected to behave in identical ways when

it comes to rating performance. I include supervisor dummies (DSupervisor) to control for these

differences. Since rating behavior might also be dissimilar for different types of employees, I

also control for three employee variables: age (AGE), gender (SEX), and number of designated

contract hours per week (HOURS). Finally, the way the system is used (e.g., the type of

measures that are chosen) and/or the importance attached to the system (e.g., the time that is

provided for instructions and evaluations) might not be identical for different departments and

offices. To control for these differences, office (DOffice) and department dummies (DDepartment) are

included.18

To control for likely correlation of regression model errors for a given employee, I use a

random effects model with robust standard errors. This leads to the following specification:

�������� �� ��������������� ���������� ����_���� � !����� �"����� �#$�%� �&�'(�$��

∑ �� *�+* ���,�-��./-0 ∑ �1

"�1+ 2 �344���5 ∑ �6

#�6+" �7�,8-�9�:�;

<� ε=> ?1A

where i indicates performance ratings (i= 1, …, 792), t time, j supervisors, k offices, and l

departments.

18

Employees have different tasks and duties that are logically translated into different performance measures. I

partly control for these differences by adding supervisor, office, and department dummies. The chosen performance

measures per supervisor are, however, also not identical per se. To examine the influence of performance measure

differences within reference groups, I redo the analyses for only those observations where the chosen performance

measures in the reference group are at least 90% the same in the two consecutive years. The results show similar

inferences as those in Table 5, implying that the results are not driven by differences in performance measures

within reference groups.

19

I use random effects19

to obtain consistent estimates of all parameters, including coefficients of

time invariant regressors (Greene, 1993).

I investigate whether the use of subjective measures leads to centrality bias, by

calculating the ratio between employees’ ratings on the objective (subjective) dimension and the

mean rating on the objective (subjective) dimension (R_RATING = Max ((rating/mean rating),

(mean rating/rating)). To justify this method, I assume that the variance in performance is similar

over the two dimensions and that the variance in the objective performance ratings is similar to

the performance variance in the sample. All other variables are the same as in model 1, and a

random effects model with robust standard errors is used, leading to the following empirical

specification:

�_�������� �� ��������������� ���������� ����_���� � !����� �"����� �#$�%� �&�'(�$��

∑ �� *�+* ���,�-��./-0 ∑ �1

"�1+ 2 �344���5 ∑ �6

#�6+" �7�,8-�9�:�;

<� ε=> ?2A

Since the descriptive statistics show different patterns when comparing the objective and

subjective ratings for the two consecutive years, I examine the existence of leniency and

centrality on a yearly basis, in addition to the pooled sample.

The Determinants of Bias

In this section, I examine the determinants of leniency and centrality bias. Although all

the compensation contracts are subject to supervisor discretion, the extent to which ratings are

biased is expected to vary with supervisor preferences. In Table 4, descriptive statistics on

leniency and centrality bias and the discretionary bonus adjustment are presented. They show

that the average amount of leniency bias was larger in 2003 than in 2004, while the average

amount of centrality bias was very similar in the two consecutive years. Supervisors also took

19

The Hausman test indicates that random effects estimation is adequate for models 1, 2, 3 and 4.

20

advantage of their ability to make discretionary adjustments, especially in 2003. None of these

adjustments was negative, and the average adjustment was 2.7% (1.5%) in 2003 (2004).

I measure the extent of leniency bias in the subjective rating of a specific employee by

comparing the objective rating to the subjective rating of that employee. As differences between

the objective and subjective rating can be caused by differences in effort allocation on the

different dimensions, I control for both the contractual weight and the number of performance

measures. Moreover, the difference between the objective and subjective rating captures both

leniency and compression applied to the rating (even after controlling for effort allocation

effects). In order to control for the effort allocation effect, and to separate leniency bias from

centrality bias, I use a residual model. I regress the ratio between subjective and objective ratings

on the weights, the number of performance measures and on the centrality bias (explained

below), and use the residuals of this regression as a measure of leniency bias (LEN_BIAS).20

This

proxy for leniency bias will likely contain some measurement error because possible differences

in ability levels on the two dimensions are not taken into account. However, since measurement

error that is caused by different ability levels is not correlated with the independent variables, the

regression estimates will be unbiased. Measurement error in the dependent variable will however

decrease the power of the test, which will work against finding any results.

Several variables are predicted to affect the extent of leniency bias applied by

supervisors. Since supervisors prefer to avoid negative consequences related to rating

performance, and painful confrontations are more likely to occur when performance is

inadequate, employee performance is predicted to have a negative effect on the extent of

leniency bias. Supervisors might also rate more leniently, especially when employee

20

A problem with this measure is that it assumes leniency bias and centrality bias to be independent. However, if I

use the simple ratio between subjective and objective ratings and control for the extent of centrality bias in the

regression, I get similar results.

21

performance is poor, to hide departmental problems. I capture employee performance by

including the objective performance ratings (OBJ_R) (see Table 6 for variable definitions).

Confronting employees with harsh but accurate ratings might be less costly to the

supervisor when the personal relationship with the employee is not that strong. I therefore predict

that ratings are less lenient when the employee and supervisor have only started working together

recently. Moreover, since supervisors care about the performance of their department, they are

expected to reduce leniency when rating leniently might give employees the mistaken idea that

they are performing their duties correctly when they are not. Since misunderstandings on how

tasks should be preformed are more likely to occur when the employee and supervisor have

worked together for a short period of time, supervisors are also expected to be less lenient in the

initial stage of the relationship to avoid stimulating dysfunctional behavior. To capture the

influence of the duration of the employee-supervisor relationship, I include a dummy variable

that equals one if the employee joined the company within the last three years, if the employee

recently changed jobs within the company or if a new supervisor was assigned, and zero

otherwise (NEW_R).

The extent of leniency bias is also influenced by the time and effort the supervisor must

invest in collecting performance information. High information-gathering costs are predicted to

result in less precise evaluations, which are expected to lead to more lenient ratings because

leniency decreases the possibility that imprecise ratings lead to problems. I capture information-

gathering costs using the supervisor’s position in the organizational hierarchy. More specifically,

22

I proxy for information-gathering costs (INFO_C) using the difference between the

organizational level of the supervisor and the employee.21

The effect of the attitude of the supervisor’s superiors towards rating accuracy is

analyzed by examining the financial performance of the offices, as financial conditions are

expected to affect attitudes towards leniency bias. More specifically, poor financial performance

is predicted to put pressure on the office to keep compensation costs down, making the superior

less tolerant of lenient ratings (Longenecker et al., 1987). I measure the financial position of the

offices by including their profit growth (GROWTH) and the difference between their budgeted

and actual profit (BUDGET).

Supervisor and employee characteristics are also expected to influence supervisor bias as

they influence the supervisor’s preferences. I include supervisor gender (SUP_SEX) and

employee gender (SEX) as independent variables because some evidence in the psychology

literature indicates that female supervisors rate more leniently (Tsui & O'Reilly, 1989), and that

female employees are rated lower than male colleagues (Rosen & Jerdee, 1974). Finally, the age

difference between the supervisor and employee (DIF_AGE) is included because research on

relational demography shows that individuals with comparative demographic characteristics are

more likely to develop a close relationship (Tsui & O'Reilly, 1989). A closer relationship is

expected to have a positive effect on leniency bias (Varma, DeNisi, & Peters, 1996), because it

can increase the utility received from favoring the employee, and/or increase the pain suffered

when the employee needs to be confronted.

Discretionary bonus adjustments provide supervisors with an additional way to influence

compensation. To control for a possible substitution effect, I include the number of percentage

21

The influence of the supervisor’s position in the organizational hierarchy on rating behavior has been analyzed in

the psychology literature, and several studies indicate little interrater agreement on performance ratings provided by

supervisors from different organizational levels (Berry, Nelson, & McNally, 1966; DeCotiis & Petit, 1978).

23

points that the bonus is adjusted beyond the stated range (DIS_ADJ). Finally, I include year

(Y2003) and department dummies (DDepartment) as additional controls.22,

.23

A random effects model

at the employee level24

with robust standard errors is used to test the following empirical

specification:

C��_D��$�� �� ��'DE_��� �����_��� ����F'_G�� � ��'����� �"D(������ �#$(�_$�%�

�&$�%� �*��F_����� �2��$_��E�� ���!����� ∑ �6�&6+�� �7�,8-�9�:�;

<� ε=> ?3A

where i relates to employee ratings (i= 1, …, 396), t to time and l to departments.

In order to measure centrality bias, I calculate the ratio between the standard deviation of all

ratings on the objective dimension and the standard deviation of all ratings on the subjective

dimension for each reference group25

(CEN_BIAS). This ratio is the same for all employees in a

reference group, and a high ratio indicates that the subjective ratings are compressed relative to

the objective ratings.

Since the extent of centrality bias is measured per reference group, I examine the

determinants of centrality bias at the supervisor level. I include the same explanatory variables as

in model 3 except that I use the median objective performance rating of the reference group

instead of the individual performance ratings (OBJ_R), and I include supervisor age (SUP_AGE)

instead of the age difference between supervisor and employee (DIF_AGE). Employee gender

(SEX) is also excluded. The predicted signs of the explanatory variables are similar to those in

model 3 apart from the expected sign on office performance. Pressure to keep compensation

22

Due to high correlation between the variables measuring office performance and the office dummies, the office

dummies are not included in the model. 23

The introduction of additional variables that control for the number of employees evaluated by the supervisor and

contract choices made by the supervisor, such as the contractual weight placed on subjective measures, the number

of subjective measures included and the consistency in the chosen subjective measures within the reference group,

leads to similar results. 24

Random effects at the supervisor level lead to similar results except for the age difference between the supervisor

and the employee, which in this specification no longer has a negative effect on the extent of leniency bias. 25

A reference group consists of all employees who have a similar function and who are evaluated by the same

supervisor.

24

costs down is predicted to lead to more, instead of less, compression. Furthermore, the standard

deviation of all objective performance ratings per reference group (REF_SD) is included as an

additional control variable, since the extent to which ratings can be compressed depends on

existing performance variation in the reference group. This leads to the following specification

that is tested with a random effects model with robust standard errors:26, 27

G��_D��$�� �� ��'DE_��� �����_��� ����F'_G�� � ��'����� �"D(������ �#$(�_$�%�

�&$(�_����� �*��$_��E�� �2��F_$��� ���!����� <� ε=> ?4A

where i relates to supervisor rating decisions (i= 1, …, 67), and t to time.

Performance Effects

In this part of the analysis, I investigate how leniency and centrality bias affect the

effectiveness of the compensation system as an incentive provider. I capture the differences in

incentives provided by the compensation system by examining the change in employee

performance following exposure to supervisor discretion. I measure the change between 2003

and 2004 as the supervisors’ influence on incentive provision becomes apparent after the

evaluation of 2003. Changes in objective performance (∆PERF_O), subjective performance

(∆PERF_S), and total performance (∆PERF_T) are examined because bias is hypothesized to

have an overall influence on employee incentives. Performance change is expected on both the

objective and the subjective dimensions. However, since prior analyses indicate that the

subjective (and therefore the total) ratings are biased, they must be interpreted with care.

26

Estimating the model at the employee level with random effects at the supervisor level leads to similar results. 27

As with employee performance, I use the median bonus adjustments per reference group instead of the individual

bonus adjustments (DIS_ADJ). Moreover, because of the limited number of observations at the supervisor level (n =

67) the department dummies are not included.

25

Existing variation in the biases provides an opportunity to examine the influence of these

biases on future performance.28

Leniency (LEN_BIAS) and centrality bias (CEN_BIAS) are

measured as before. However, in order to analyze the separate effects of centrality bias on above-

average and below-average performers, I split CEN_BIAS into two variables: CEN_BIASA and

CEN_BIASB. CEN_BIASA (CEN_BIASB) takes the value of the ratio when the employee

performs above (below) the average of his reference group on the objective dimension, and zero

otherwise. As mentioned before, whether I am able to adequately capture leniency and centrality

bias depends on the validity of the assumption that the objective performance rating is a good

benchmark. This assumption is however less essential when examining employee reactions

because even if the objective performance rating is not a good benchmark of actual performance,

the objective performance rating is likely to be used by employees as a benchmark to compare

their subjective rating to and subsequently to form their perceptions of bias with. Thus, since

employees react to their perceptions, this analysis is less dependent on the validity of the earlier

made assumptions.

I control for several variables that might influence the change in employees’ performance

ratings. First, discretionary bonus adjustments are expected to have a positive effect on employee

incentives. By adjusting ‘incorrect’ ratings, the supervisor signals that providing employees with

a fair reward is considered important. This reinforces employees’ beliefs that improving

performance will result in higher rewards, which is essential in motivating employees to enhance

performance (Dubinsky & Levy, 1989). Moreover, discretionary adjustments are expected to

28

When developing hypotheses on the influence of centrality bias on incentives, I assume that the employees had at

least some knowledge of the rating distribution of their reference group. During the interviews, I asked several

employees about this matter and they confirmed that they had some, though not full, knowledge of the performance

ratings of their peers. Moreover, through the internal study conducted by the company (see page 15), employees

confidentially provided information on the perceived fairness of the evaluation procedures and the outcome

distributions of the new compensation system, which implies that they possessed knowledge of how rewards were

distributed.

26

have a positive effect on incentives by creating a feeling of reciprocity.29

I control for the effect

of discretionary adjustments using the DIS_ADJ variable defined earlier. Discretionary

adjustments are not only expected to influence the motivation of the employees who receive

them, but also the motivation of those who do not but are confronted by their existence.

Discretionary bonus adjustments create a lack of consistency in rewarding procedures. For some

employees actions not included in the compensation contract are taken into account, while for

others they are not. This lack of consistency influences incentives by influencing employee

perceptions of procedural justice within their organization (Parker & Kohlmeyer, 2005). To

control for the influence of discretionary bonus adjustments on employees who do not receive

them, I use a dummy variable that equals one if an adjustment has taken place within the

reference group and zero otherwise (REF_DISADJ).

Third, I control for the maximum amount the performance of the employee can still

improve (the difference between the maximum performance rating (5) and the obtained

performance rating in 2003, MGROWTH), for changes in the contractual incentive weight on the

objective dimension (DIF_WO) and for changes in the number of performance measures

(DIF_NR).

I also include several personal conditions that might make an employee less sensitive to

the incentives provided by the compensation contract. First, employees who have reached the

upper limit of their salary scale might be less receptive to changes in provided incentives as

performance improvement can only lead to a bonus increase, not to a periodic salary increase. I

also include contract hours and age, as I suspect that part-time and older employees are not

motivated in the same way (e.g., they are expected to be less concerned about their future

29

The employee receives more from the company than the required minimum, making the employee willing to

provide more than the minimum effort required (Hannan, Kagel, & Moser, 2002).

27

career). Some employees have occupied a different position and/or were appraised by a different

supervisor in 2004. To ensure this does not drive the results, two dummy variables are included

(DDIF_FUN & DDIF_SUP) that equal one if the employee’s function and supervisor, respectively,

have changed, and zero otherwise. Finally, I add office (DOffice) and department dummies

(DDepartment) to control for office-specific and job-specific elements, such as local market

conditions, that might influence employee performance. This leads to the following empirical

specification:30

∆���F� L� L�C��_D��$� L�G��_D��$�� L�G��_D��$D� L ��$_��EM L"��F_��$��E�

L#���'���� L&��F_�'� L*��F_��� L2�7MN_�OPQ L���7MN_R3SQ L����%_$G�

L���'(�$� L������ ∑ L1�*1+� �344���5 ∑ L6

�"6+�2 �7�,8-�9�:�;

T� ?5A

where i relates to employees (i= 1, …, 198) and ∆PERF can be ∆PERF_T, ∆PERF_O or ∆PERF_S.

IV. RESULTS

Leniency and Centrality Bias

As a first step in my analysis, I examine whether the performance ratings are, on average,

subject to leniency and centrality bias (Table 5). The results show that the ratings on the

subjective dimension are higher than the ratings on the objective dimension, after controlling for

relevant influences, implying that subjective performance ratings are subject to leniency bias.

However, the results of the annual analyses show that the subjective performance ratings are not

more lenient in 2004. Subjectivity even has a marginally significant negative effect on the

ratings. These results indicate that supervisors in general use the leeway they have in subjective

30

I have chosen to model the relationship between centrality and leniency bias and performance change as a linear

relationship because I expect the effect of more leniency and centrality bias to be relatively constant over the limited

range found in this setting.

28

performance appraisal to bias performance ratings, but that this does not necessarily result in

more lenient ratings.

Consistent with earlier empirical work, I find that the ratings on the subjective

performance dimensions are more compressed than the rating on the objective dimension. The

number of performance measures has a negative effect on the ratings, suggesting that a higher

number of measures leads to lower performance ratings, consistent with the dilution argument.

The Determinants of Bias

Determinants of leniency and centrality bias are investigated in Table 6. Employee

performance is found to be one of the main determinants of leniency bias. The objective

performance ratings negatively affect the extent of leniency bias, which indicates that

supervisors rate more leniently when employee performance is low. The performance of the

reference group has no effect on centrality bias.

Ratings are found to be less lenient when the employee and supervisor only recently

started to work together, indicating that supervisors are more willing to be accurate when

personal ties are not that strong, and when lenient ratings are more likely to provide a wrong

signal on the correctness of employee actions. Although the lack of a strong personal relationship

was predicted to make supervisors more willing to differentiate between employees, ratings are

found not to be less, but more compressed in the initial period of the employee-supervisor

relationship. A possible explanation for this is that supervisors refrain from differentiating

strongly until they have more information on employee performance.

The proxy for information-gathering costs positively affects the extent of leniency and

centrality bias, indicating that supervisors provide more lenient and compressed ratings when

29

gathering information is more costly. The evidence also indicates that the office’s financial

position affects supervisor bias. As predicted, both financial performance variables have a

positive effect on the extent of leniency bias and a negative effect on the extent of centrality bias.

This indicates that supervisors become more lenient and compress less when the office is

performing better financially.

Supervisor gender is also found to influence rating behavior. I find that male supervisors

are more lenient, not female supervisors, as predicted by previous studies. A speculative

explanation for this finding is that female supervisors are aware of the stereotype and

consequently exercise more caution when rating. Employee gender was not found to influence

the extent of bias.31

Finally, age differences between supervisors and employees negatively affect

the extent of leniency bias, suggesting that it is more costly for supervisors to rate accurately

when they have a strong relationship with the employees.

In sum, the results indicate that supervisors vary the amount of bias they apply to

performance ratings, and that more bias is applied when biased performance ratings increase the

utility the supervisor receives out of rating employee performance.

Performance Effects

The performance effects of leniency and centrality bias are examined in Table 7. The

results32

indicate that supervisor bias has both positive and negative effects on future

31

Most studies that find that female employees receive lower ratings than male employees are conducted in settings

where the occupation would likely be perceived as masculine (see Landy & Farr, 1980), while this setting has both

masculine- and feminine-type positions. However, even after interacting the position’s stereotype with employee

gender, I find no evidence of employee gender affecting the extent of bias applied to the ratings. 32

The Breusch-Pagan and White's tests for heteroskedasticity indicate that the data does not suffer from severe

heteroskedasticity problems and the Variance Inflation Factors (VIF) indicate the absence of multicollinearity

problems.

30

performance.33

First, leniency bias has a positive effect on objective and total performance

changes, indicating that employees who receive more lenient ratings on the subjective dimension

show more performance improvement on the objective and total dimensions in the following

year. The results also show that centrality bias, for above- and below-average performers, has a

negative effect on performance change on all dimensions, indicating that more compressed

ratings have a negative influence on the performance of all employees.

Regarding the impact of discretionary bonus adjustments, I find that discretionary

adjustments have a positive effect on performance change, while discretionary adjustments made

in the reference group have no effect. This implies that receiving a discretionary adjustment

increases future performance. An alternative explanation is that the ratings in 2004 now portray

the true level of overall performance. Instead of needing an ex post adjustment to give the

employee the fair amount of compensation, the true value is now directly captured by the

measures, resulting in a higher rating. However, even if we cannot be sure that incentives will

improve after employees have received an adjustment, the results show that employees who do

not receive an adjustment do not show a significant negative performance reaction, thereby

taking away an important reason for not adjusting rewards.

The results also show that the employee’s growth potential (MGROWTH) has a positive

effect on performance change. This shows that employees with more room for improvement

improve their ratings to a greater extent. Furthermore, I find that changes in weights and number

of performance measures influence performance change. Being evaluated by a different

supervisor is also found to influence performance; it negatively affects performance changes.

33 I also examine the performance effects on a more aggregate level. More specifically, I investigate whether the

average performance change of all employees who were appraised by the same supervisor is larger (smaller) when

that supervisor has a stronger tendency to apply leniency (centrality) bias. To test this I use model 5, but measure the

performance changes and the biases at the supervisor level. The results show that the average extent of leniency

(centrality) bias is positively (negatively) related to the average performance change.

31

Finally, the results show that age has a negative effect on performance change, implying that

younger employees are more sensitive to incentives.

Robustness Checks

The results show that performance change on the subjective dimension is not

significantly affected by leniency bias. A plausible explanation for this finding is that the

motivational effect caused by the leniency bias in 2003 is offset by the less lenient average

subjective performance ratings provided in 2004. An alternative explanation is that employees

who score higher on the subjective dimension than on the objective dimension (the majority of

the employees) reallocate their effort to the objective dimension because improving a high rating

even further requires more than a linear effort increase. I examine this possibility by including

the quadratic term of the maximum growth potential (MGROWTH) to control for the possible

non-linear effort requirements. The results (not reported) show that the quadratic terms have no

significant effect on performance change and all other inferences remain the same.

I also investigate the influence of effort reallocation on performance change. I do this by

examining the group of employees who are most likely to reallocate their effort from the

subjective to the objective dimension because of non-linear effort requirements; i.e., employees

scoring high on the subjective dimension (top 20%) and scoring higher on the subjective than on

the objective dimension. I use model 5 and replace the leniency bias variable by a dummy

variable that indicates one if the employee belongs to this group and zero otherwise. The results

(not reported) show no relation between the dummy variable and performance change on the

objective dimension, suggesting that the positive relationship between leniency bias and

32

performance change on the objective dimension is not driven by the reallocation of effort from

the subjective to the objective dimension.

Previously, I implicitly assumed that the performance ratings are exactly proportional to

the bonus percentage. The ratings are, however, linked to a bonus range, not to a concrete

percentage. The actual bonus might therefore not be exactly proportional to the rating, even if no

discretionary bonus adjustment has occurred. To ensure that this is not driving the results, I

calculate the difference between the actual bonus percentage and the bonus percentage that

would have been paid if the relation between the rating and the bonus percentage had been

completely proportional. After replacing the discretionary adjustment variable (DIS_ADJ) with

this newly constructed difference variable, I re-estimate specification 5. The results (not

reported) are similar to those in Table 7.34

Additional Analysis

In order to capture the incentive effect, I measure the performance change by comparing

the performance rating of 2003 with the rating of 2004. However, the subjective and therefore

the total rating are biased and the amount of bias applied by the supervisor is not necessarily

consistent over the years. The measurements of the performance change on the subjective and

total dimension are therefore clouded.

To address this limitation, I preform an additional analysis with data collected through a

questionnaire that was part of an internal study performed by FSP.35

In this analysis, instead of

the change in performance, I use the change in effort to capture the incentive effects. The change

in effort is measured using the response to a survey question that asked employees how their

34

The results of model 3 and 4 also remain unchanged when DIS_ADJ is replaced by this new difference variable. 35

The questionnaire was conducted in august of 2004 and administrated by the designers of the system. I had no

influence on the design or execution of the questionnaire.

33

effort was affected by the new incentive system (see Table 8 for variable definition). The

questionnaire also asked the employees to provide details on the design of their performance

document and the outcome of the performance measurement process, which allows me to

measure both leniency and centrality bias. The information from the questionnaire is less precise

and elaborate than the archival data (the questionnaire was kept anonymous so I was unable to

link the questionnaire to the archival data). The calculations of the biases are therefore not

identical to those performed for the main analysis.36

Finally, the questionnaire provides

information on some of the control variables included in the main model. This leads to the

following specification that is tested with an ordered probit regression model:

�FF'��� L� L�C��_D��$� L�G��_D��$�� L�G��_D��$D� L ��%_$G� L"F(CC�����

L#���� L&$�%� T� ?6A

where i relates to employees (i= 1, …, 70).

The results, presented in Table 8, confirm the findings of the main analysis. Leniency

bias has the hypothesized positive effect on effort, while centrality bias has the expected negative

effect. In summary, I find strong support for the prediction that leniency and centrality bias affect

the effectiveness of the compensation system as an incentive provider.

V. CONCLUSIONS AND FUTURE RESEARCH

Rating inaccuracy caused by supervisor bias is perceived to be one of the main problems

from introducing subjectivity into compensation contracts. This is due to its assumed negative

36

For the calculation of leniency bias in the main analysis I used the percentage of objective and subjective

performance measures. Since the questionnaire does not provide any information that allows me to distinguish

between objective and subjective output-based measures (competences are by definition subjective), I only include

those employees that indicated that their output-based measures were very measurable (hence, assuming that for

these employees all output-based performance measures were objective). The way centrality bias is calculated is

identical but since the questionnaire was only completed by a limited number of employees, the information on the

performance of the reference group is very limited.

34

effect on the compensation system’s ability to motivate employees and provide valuable

information for personnel decisions. Our knowledge of the influence of supervisor bias on the

effectiveness of compensation contracting is, however, extremely limited. Empirical studies

examining the consequences of biased performance ratings are especially lacking.

This paper contributes to the literature by investigating what causes supervisors to bias

performance ratings and by analyzing how biased ratings affect employee incentives. The results

indicate that the use of subjective measures leads to more lenient and compressed ratings and

that supervisors bias ratings to a greater extent when bias enhances the utility received out of

rating performance. The results also provide strong support for the prediction that supervisor bias

not only affects current performance ratings, but also future employee incentives. I find that

leniency bias has a positive effect on performance improvement, while centrality bias has a

negative performance effect for both above- and below-average performers.

The empirical finding that supervisor bias can have a positive influence on employee

incentives provides an explanation for earlier empirical findings that show that supervisors in

general do not receive rewards for rating accurately (e.g., Napier & Latham, 1986). Companies

seem to be interested in the effectiveness of performance-based compensation contracts in

increasing employees’ future performance, not necessarily in the accuracy of the performance

appraisal as such (at least not for incentive purposes). Motivation to manage the performance

appraisal process in an efficient manner seems to be provided by linking the supervisors’

compensation and promotion possibilities to their unit’s performance, not to the accuracy of their

unit’s performance ratings.

The findings from this study are not without limitations. First, although there are no

theoretical reasons to expect that the results would not extend to other settings, its

35

generalizability is reduced by relying on data from one firm. Moreover, I was only able to obtain

information for two consecutive years. As a consequence, I was unable to examine whether the

documented effects of leniency and centrality bias persist over the years. More extensive time-

series data from multiple organizations would remove these limitations. Second, the focus of this

study has been on intentional bias. Supervisors can also unintentionally bias performance ratings

due to their cognitive limitations. I was unable to distinguish the extent of distortion caused by

these cognitive limitations and therefore unable to investigate its effect.

Another opportunity for future research is to examine the effect of inaccurate ratings on

other functions of performance ratings. I have limited the study to incentive provision, but

performance ratings are also likely to influence training and promotion decisions, among others.

Finally, this study focuses only on the effects of biased performance ratings on future

performance. Subjectivity in compensation contracting is likely to give rise to additional

concerns, e.g., uncertainty about performance criteria. Investigating how these other concerns, in

combination with biased ratings, influence the effectiveness of compensation contracts would

make an important contribution to the literature.

36

REFERENCES

Akerlof, G. A. & J. L. Yellen, 1988, Fairness and Unemployment, The American Economic

Review, 78(2): 44-49.

Arkin, R., H. Cooper, & T. Kolditz, 1980, A Statistical Review of the Literature Concerning the

Self-Serving Attribution Bias in Interpersonal Influence Situations, Journal of Personality, 48:

435-48.

Baiman, S. & M. V. Rajan, 1995, The Informational Advantages of Discretionary Bonus

Schemes, The Accounting Review, 70(4): 557-79.

Baker, G. P., R. Gibbons, & K. J. Murphy, 1994, Subjective Performance Measures in Optimal

Incentive Contracts, Quarterly Journal of Economics, 109(4): 1125-56.

Bénabou, R. & J. Tirole, 2003, Intrinsic and Extrinsic Motivation, Review of Economic Studies,

70(3): 489-520.

Bénabou, R. & J. Tirole, 2002, Self-Confidence and Personal Motivation, The Quarterly Journal

of Economics, 117(3): 871-915.

Bernardin, J. H. & R. M. Buckley, 1981, Strategies in Rater Training, Academy of Management

Review, 6(2): 205-12.

Berry, N. H., P. D. Nelson, & M. McNally, 1966, A Note on Supervisor Ratings, Personnel

Psychology, 19: 423-26.

Blinder, A. S. & D. H. Choi, 1990, A Shred of Evidence on Theories of Wage Stickiness, The

Quarterly Journal of Economics, 105(4): 1003-15.

Bretz, R. D., G. T. Milkovich, & W. Read, 1992, The Current State of Performance Appraisal

Research and Practice: Concerns, Directions, and Implications, Journal of Management, 18(2):

321-52.

Burney, L. L., C. A. Henle, & S. K. Widener, 2006, Do Characteristics of Strategic Performance

Measurement Systems Used in Incentives Enhance Organizational Fairness?, Working Paper: 1-

41.

Colquitt, J. A., D. E. Conlon, M. J. Wesson, C. O. L. H. Porter, & K. Yee Ng, 2001, Justice at

the Millennium: A Meta-Analytic Review of 25 Years of Organizational Justice Research,

Journal of Applied Psychology, 86(3): 425-45.

DeCotiis, T. & A. Petit, 1978, The Performance Appraisal Process: A Model and Some Testable

Propositions, Academy of Management Review, 3(3): 635-46.

37

Dubinsky, A. J. & M. Levy, 1989, Influence of Organizational Fairness on Work Outcomes of

Retail Salespeople, Journal of Retailing, 65(2): 221-52.

Fang, H. & G. Moscarini, 2002, Overconfidence, Morale and Wage-Setting Policies: Mimeo,

Yale University.

Gabris, G. T. & K. Mitchell, 1988, The Impact of Merit Raise Scores on Employee Attitudes:

The Matthew Effect of Performance Appraisal, Public Personnel Management, 17(4): 369–89.

Gibbs, M., K. A. Merchant, W. A. Van der Stede, & M. E. Vargus, 2004, Determinants and

Effects of Subjectivity in Incentives, The Accounting Review, 79(2): 409-36.

Greenberg, J., 1990, Organizational Justice: Yesterday, Today, and Tomorrow, Journal of

Management, 16(2): 399-432.

Greene, W. H., 1993, Econometric Analysis, 5 ed, New Jersey: Pearson Education, Inc.

Hannan, L. R., J. H. Kagel, & D. V. Moser, 2002, Partial Gift Exchange in an Experimental

Labor Market: Impact of Subject Population Differences, Productivity Differences, and Effort

Requests on Behavior, Journal of Labor Economics, 20(4): 923-51.

Harris, M. M., 1994, Rater Motivation in the Performance Appraisal Context: A Theoretical

Framework, Journal of Management, 20(4): 737-56.

Harris, M. M. & J. Schaubroeck, 1988, A Meta-Analysis of Self-Supervisor, Self-Peer, and Peer-

Supervisor Ratings, Personnel Psychology, 41: 43 - 62.

Hayes, R. M. & S. Schaefer, 2000, Implicit Contracts and the Explanatory Power of Top

Executive Compensation for Future Performance, Rand Journal of Economics, 31(2): 273-93.

Heneman, H. G. & D. P. Schwab, 1972, Evaluation of Research on Expectancy Theory

Predictions of Employee Performance, Psychological Bulletin, 78(1): 1-9.

Holmstrom, B., 1979, Moral Hazard and Observability, Bell Journal of Economics, 10(1): 74-91.

Indjejikian, R. J., 1999, Performance Evaluation and Compensation Research: An Agency

Perspective, Accounting Horizons, 13(2): 147-57.

Ittner, C. D., D. F. Larcker, & M. W. Meyer, 2003, Subjectivity and the Weighting of

Performance Measures: Evidence from a Balanced Scorecard, The Accounting Review, 78(3):

725-58.

Jawahar, I. M. & C. R. Williams, 1997, Where All the Children Are above Average: The

Performance Appraisal Purpose Effect, Personnel Psychology, 50: 905-26.

Landy, F. J. & J. L. Farr, 1980, Performance Rating, Psychological Bulletin, 87(1): 72-107.

38

Larwood, L. & W. Whittaker, 1977, Managerial Myopia: Self-Serving Biases in Organizational

Planning, Journal of Applied Psychology, 62(2): 194–98.

Lazear, E. P., 1991, Labor Economics and the Psychology of Organizations, Journal of

Economic Perspectives, 5(2): 89-110.

Lipe, M. G. & S. E. Salterio, 2000, The Balanced Scorecard: Judgmental Effects of Common and

Unique Performance Measures, The Accounting Review, 75(3): 283-98.

Longenecker, C. O., H. P. Sims, & D. A. Gioia, 1987, Behind the Mask: The Politic of Employee

Appraisal, The Academy of Management Executive, 1(3): 183-93.

McFarlane Shore, L. & G. C. Thornton, 1986, Effects of Gender on Self-and Supervisory

Ratings, Academy of Management Journal, 29(1): 115 - 29.

McGregor, D. M., 1957, An Uneasy Look at Performance Appraisal, Harvard Business Review,

35(3): 89-94.

Meyer, H. H., 1975, The Pay-for-Performance Dilemma, Organizational Dynamics, 3: 39-50.

Mischel, W., E. B. Ebbesen, & A. M. Zeiss, 1976, Determinants of Selective Memory About the

Self, Journal of Consulting and Clinical Psychology, 44(1): 92-103.

Moers, F., 2005, Discretion and Bias in Performance Evaluation: The Impact of Diversity and

Subjectivity, Accounting, Organizations and Society, 30(1): 67-80.

Murphy, K. R. & J. Cleveland, 1991, Performance Appraisal: An Organizational Perspective,

Boston: Allyn and Bacon.

Napier, N. K. & G. P. Latham, 1986, Outcome Expectancies of People Who Conduct

Performance Appraisals, Personnel Psychology, 39(4): 827-37.

O'Connor, N., F. J. Deng, & M. D. Shields, 2006, Determinants of the Subjective Performance

Measurement of Managerial Behavior, Working Paper: 1-33.

Parker, R. J. & J. M. Kohlmeyer, 2005, Organizational Justice and Turnover in Public

Accounting Firms: A Research Note, Accounting, Organizations and Society, 30: 357-69.

Pearce, J. L. & L. W. Porter, 1986, Employee Responses to Formal Performance Appraisal

Feedback, Journal of Applied Psychology, 71(2): 211-18.

Prendergast, C., 1999, The Provision of Incentives in Firms, Journal of Economic Literature,

37(1): 7-63.

Prendergast, C. & R. H. Topel, 1993, Discretion and Bias in Performance Evaluation, European

Economic Review, 37(2-3): 355-65.

39

Prendergast, C. & R. H. Topel, 1996, Favoritism in Organizations, Journal of Political Economy,

104(5): 958-78.

Rosen, B. & T. H. Jerdee, 1974, Influence of Sex Role Stereotypes on Personnel Decisions,

Journal of Applied Psychology, 59(1): 9-14.

Rynes, S. L., B. Gerhart, & L. Parks, 2005, Personnel Psychology: Performance Evaluation and

Pay for Performance, Annual Review of Psychology, 56: 571-600.

Tsui, A. S. & C. A. O'Reilly, 1989, Beyond Simple Demographic Effects: The Importance of

Relational Demography in Superior-Subordinate Dyads, The Academy of Management Journal,

32(2): 402-23.

Varma, A., A. S. DeNisi, & L. H. Peters, 1996, Interpersonal Affect and Performance

Appraisals: A Field Study, Personnel Psychology, 49: 341-60.

Vroom, V. H., 1964, Work and Motivation: Wiley New York.

Weinstein, N. D., 1980, Unrealistic Optimism About Future Life Events, Journal of Personality

and Social Psychology, 39(5): 806-20.

40

FIGURE 1

The performance measurement document

Name:

Department:

Position:

Supervisor:

EvaluationBad Regular Good Very Good Excellent

1) Ο Ο Ο Ο Ο

2) Ο Ο Ο Ο Ο

3) Ο Ο Ο Ο Ο

4) Ο Ο Ο Ο Ο

5) Ο Ο Ο Ο Ο

6) Ο Ο Ο Ο Ο

Comments

EvaluationBad Regular Good Very Good Excellent

1) Ο Ο Ο Ο Ο

2) Ο Ο Ο Ο Ο

3) Ο Ο Ο Ο Ο

4) Ο Ο Ο Ο Ο

5) Ο Ο Ο Ο Ο

6) Ο Ο Ο Ο Ο

Comments

Total rating output dimension 50%

Total rating competence dimension 50%

Total score

Salary Scale Growth % Bonus

Salary Scale Growth % Bonus

PERFORMANCE DOCUMENT

Salary Determination

Good Very Good

Output performance measures

Competence performance measures

Current Salary

New Salary

Excellent

0% 1-3% 4-8% 9-12% 13-15%

Bad Regular

41

TABLE 1

Descriptive statistics: Part I

2003 2004

Variable Mean SD Median Range n Mean SD Median Range n

Overall performance rating 2.87 0.52 2.86 1.6 – 4.7 198 3.20 0.50 3.23 1.6 – 4.4 198

Objective performance rating 2.80 0.76 2.75 1.0 – 5.0 198 3.28 0.74 3.33 1.0 – 5.0 198

Subjective performance rating 2.94 0.46 3.00 1.5 – 4.3 198 3.14 0.45 3.13 2.1 – 4.6 198

Total # of performance measures 9.70 1.77 10.00 5 – 12 198 10.68 1.49 11.00 5 – 12 198

# of objective performance measures 4.04 1.35 4.00 1 – 6 198 4.48 1.30 5.00 2 – 6 198

# of subjective performance measures 5.66 1.36 6.00 2 – 9 198 6.20 1.06 6.00 3 – 8 198

% of total rating objectively determined 44.33 8.72 50.00 25 – 50 198 44.37 7.57 50.00 25 – 50 198

% of total rating subjectively determined 55.67 8.72 50.00 50 – 75 198 55.63 7.57 50.00 50 – 75 198

42

TABLE 2

Descriptive statistics: Part II

Variable Mean SD Median Range n

Age 38.39 8.60 37 23 – 60 198

Supervisor age 39.76 7.54 37 28 – 57 41

Designated contract hours in 2003 32.68 6.52 36 12 – 40 198

Designated contract hours in 2004 32.27 6.80 36 12 – 40 198

Profit growth in 2003a

1.09 2.62 1.60 -6.2 – 11.9 198

Profit growth in 2004a

5.68 18.21 1.27 -20.3 – 67.2 198

Budgeted versus actual profit in 2003b

3.61 3.56 1.64 1.6 – 12.6 198

Budgeted versus actual profit in 2004b

4.14 4.05 4.24 -4.2 – 7.81 198

Variable 2003 2004

% of employees at their salary scale maximum 26.26 31.31

% of female employees 56.06 56.06

Number of supervisors 35 32

% of female supervisors 28.57 40.63

a The percentage difference between the offices’ profit this year and the offices’ profit last year.

b The percentage difference between the offices’ budgeted profit growth and the actual profit growth.

43

TABLE 3

Correlation between variables

198 observations for 2003 1 2 3 4 5 6 7 8 9 10

1. Overall performance rating

2. Objective performance rating 0.90***

3. Subjective performance rating 0.80*** 0.47***

4. Total # of performance m. -0.14** -0.11 -0.13*

5. # of objective performance m. -0.12* -0.17** -0.02 0.65***

6. # of subjective performance m. -0.06 0.03 -0.15** 0.66*** -0.15**

7. % objectively determined -0.10 -0.16** 0.02 0.10 0.52*** -0.39***

8. Employee age -0.02 -0.03 -0.03 -0.04 -0.02 -0.03 -0.03

9. Employee contract hours 0.18** 0.19*** 0.12* 0.30*** 0.13* 0.26*** 0.04 -0.04

10. Profit growth -0.16** -0.15** -0.12 -0.12* -0.05 -0.10 -0.01 0.04 -0.17**

11. Budgeted versus actual profit 0.24*** 0.12* 0.35*** -0.06 0.17** -0.25*** 0.09 -0.16** 0.02 -0.11

198 observations for 2004

1. Overall performance rating

2. Objective performance rating 0.88***

3. Subjective performance rating 0.78*** 0.41***

4. Total # of performance m. -0.08 -0.08 -0.05

5. # of objective performance m. 0.02 -0.03 0.07 0.72***

6. # of subjective performance m. -0.14** -0.08 -0.16** 0.53*** -0.21***

7. % objectively determined 0.18*** 0.10 0.19*** -0.05 0.57*** -0.77***

8. Employee age -0.22*** -0.25*** -0.09 0.04 -0.02 0.08 -0.06

9. Employee contract hours 0.14** 0.15** 0.11 0.10 0.08 0.04 -0.01 0.00

10. Profit growth -0.08 -0.09 -0.04 -0.28*** -0.19*** -0.16** -0.10 -0.01 0.08

11. Budgeted versus actual profit 0.12 0.13* 0.03 0.36*** 0.10 0.39*** -0.02 0.07 0.14* -0.30***

***, **, * is statistically significant at respectively the 1%, 5%, and 10% level (two-tailed).

44

TABLE 4

Descriptive statistics: Part III

Variable Mean SD Median Range n

2003

Leniency biasa 0.06 0.32 0.00 -0.44 – 1.75 198

Centrality biasb 2.18 1.41 1.92 0.32 – 5.95 198

Leniency bias per supervisorc 0.06 0.22 0.03 -0.29 – 0.80 35

Centrality bias per supervisor 2.22 1.36 2.17 0.40 – 5.95 35

2004

Leniency bias -0.06 0.27 -0.11 -0.50 – 1.61 198

Centrality bias

2.20 1.15 1.92 0.52 – 6.05 198

Leniency bias per supervisor -0.06 0.14 -0.10 -0.25 – 0.40 32

Centrality bias per supervisor 2.10 1.00 1.88 0.68 – 5.10 32

Variable 2003 2004

Discretionary bonus adjustment 35% 19%

Average bonus adjustmentd 2.7% 1.5%

a Leniency bias is captured by regressing the ratio between the subjective and objective ratings on the

weights, the number of performance measures and on the centrality bias, and using the residuals of this

regression as a measure of leniency bias. b Centrality bias is measured as the ratio between the standard deviation of all performance ratings on the

objective dimension and the standard deviation of all performance ratings on the subjective dimension for

each reference group. c The average amount of bias applied per supervisor.

d The average bonus adjustment once an adjustment is made. All adjustments were of a positive nature.

45

TABLE 5

The impact of subjectivity on the performance rating and on its variation

2003

2004

Independent variables a

RATING R_RATING RATING R_RATING RATING R_RATING

DSubjectivity

0.10**

(2.11)

-0.14***

(-7.35)

0.29***

(4.80)

-0.15***

(-5.70)

-0.09

(-1.44)

-0.11***

(-4.82)

WEIGHT 0.00

(0.20)

0.00

(1.33)

-0.00

(-0.86)

0.01**

(2.04)

0.01

(0.88)

0.00

(1.54)

NR_PM -0.06***

(-2.76)

-0.00

(-0.25)

-0.06**

(-2.31)

-0.01

(-1.27)

-0.07

(-1.19)

-0.02

(-1.32)

Y2003 -0.40***

(-9.80)

0.05***

(3.28)

AGE -0.01**

(-2.47)

0.01*

(1.76)

-0.01

(-1.34)

-0.00

(-0.20)

-0.01***

(-2.69)

0.01***

(2.66)

SEX -0.12

(-1.38)

0.05*

(1.81)

-0.10

(-1.14)

0.04

(1.17)

-0.17

(-1.55)

0.07*

(1.87)

HOURS 0.01

(1.43)

-0.00

(-1.14)

0.01

(1.50)

-0.00

(-1.12)

0.01

(1.08)

-0.00

(-1.05)

R2

Within

Between

Overall

0.22

0.41

0.31

0.14

0.32

0.20

0.09

0.58

0.43

0.22

0.42

0.33

0.05

0.37

0.26

0.14

0.32

0.24

***, **, * indicate statistical significance at respectively the 1%, 5% and 10% level (two-tailed). Z-values are in parentheses

and n = 792 for the first two regressions and n = 396 for the last four. (Continued on next page)

46

TABLE 5 (continued)

RATING Performance rating, on the objective or the subjective dimension

R_RATING Performance ratings variation, measured as the ratio between employees’ performance ratings on the objective

(subjective) dimension and the mean performance rating on the objective (subjective) dimension: Max

((rating/mean rating), (mean rating/rating))

DSubjectivity Dummy variable that equals 1 if the observation refers to a subjective performance rating and 0 otherwise

WEIGHT Contractual incentive weight on the dimension (objective or subjective) to which the observation relates

NR_PM Number of performance measures on the dimension (objective or subjective) to which the observation relates

Y2003 Dummy variable that equals 1 if the observation relates to 2003 and 0 otherwise

AGE Employee age

SEX Employee gender (0 for female and 1 for male)

HOURS Employee designated contract hours per week a An intercept, as well as supervisor, department and office dummies are included but not reported.

47

TABLE 6

The determinants of leniency and centrality bias

Independent variables

a

LEN_BIAS

CEN_BIAS

OBJ_R

b -0.33***

(-16.07)

0.19

(0.52)

NEW_R -0.05**

(-2.12)

0.64*

(1.78)

INFO_C 0.04*

(1.78)

0.48*

(1.69)

GROWTH 0.02**

(2.08)

-0.02**

(-2.54)

BUDGET 0.01***

(3.44)

-0.04

(-1.54)

SUP_SEX 0.11***

(4.05)

-0.08

(-0.23) SEX 0.03

(1.26)

DIF_AGE -0.01*

(-1.73)

SUP_AGE 0.02

(1.38) DIS_ADJ

b 0.00

(0.44)

-0.23

(-1.33) Y2003 -0.06***

(-3.22)

0.58*

(1.67) REF_SD 2.30***

(3.71)

R2

Within

Between

Overall

0.73

0.63

0.67

0.55

0.29

0.33

***, **, * indicate statistical significance at respectively the 1%, 5% and 10% level (two-

tailed). Z-values are in parentheses and n = 396 for the first and n = 67 for the second

regression .

(Continued on next page)

48

TABLE 6 (continued)

LEN_BIAS Leniency bias, measured as the residuals of the regression of the ratio

between the subjective performance rating and the objective

performance rating on the contractual weights, the number of

performance measures and the centrality bias

CEN_BIAS Centrality bias, measured as the ratio between the standard deviation of

all performance ratings on the objective dimension and the standard

deviation of all performance ratings on the subjective

dimension for each reference group

OBJ_R Objective performance rating

NEW_R Dummy variable that equals 1 if the employee joined the company in the

last three years, if the employee recently changed jobs within the

company or if a new supervisor was assigned, and 0 otherwise

INFO_C Dummy variable that is 0 if the supervisor and the employee have

the same organizational level and 1 if the supervisor has a higher

organizational level

GROWTH Percentage difference between the profit of this year and the profit of

last year calculated per office per year

BUDGET Percentage difference between budgeted profit growth and actual profit

growth calculated per office per year

SUP_SEX Supervisor gender (0 for female and 1 for male)

SEX Employee gender (0 for female and 1 for male)

DIF_AGE Absolute difference between supervisor age and employee age

SUP_AGE Supervisor age

DIS_ADJ Difference between the rewarded bonus percentage and the maximum

of the stated bonus range

Y2003 Dummy variable that equals 1 if the observation relates to 2003 and 0

otherwise

REF_SD Standard deviation of all objective performance ratings within the same

reference group a An intercept and department dummies are included but not reported.

b Since the determinants of centrality bias are examined at the supervisor level the median

objective performance rating per reference group and the median bonus adjustment per

reference group are used in the second regression.

49

TABLE 7

Regression of the effects of supervisor discretion on incentive provision (I)

Independent variables

a

∆PERF_O

∆PERF_S

∆PERF_T

LEN_BIAS 0.73***

(3.00)

-0.10

(-0.94)

0.22*

(1.89) CEN_BIASA -0.14***

(-3.12)

-0.04*

(-1.72)

-0.08***

(-3.08) CEN_BIASB -0.13***

(-3.15)

-0.06***

(-2.69)

-0.09***

(-3.69) DIS_ADJ 0.06*

(1.86)

0.07***

(4.01)

0.07***

(3.65) REF_DISADJ 0.01

(0.11)

0.07

(0.93)

0.02

(0.27) MGROWTH

b 0.27**

(2.26)

0.45***

(6.78)

0.36***

(4.66) DIF_WO 0.01**

(2.07)

0.00

(0.67)

0.01**

(2.36) DIF_NR

c -0.02

(-0.53)

-0.06*

(-1.79)

-0.02

(-1.34) DDIF_ SUP -0.18

(-1.46)

-0.13*

(-1.91)

-0.16**

(-2.14) DDIF_JOB 0.17

(0.85)

-0.04

(-0.32)

0.07

(0.56) MAX_SC -0.18

(-1.42)

0.05

(0.68)

-0.06

(-0.80)

HOURS

-0.01

(-0.73)

0.00

(0.10)

-0.00

(-0.42)

AGE

-0.02***

(-2.64)

-0.01**

(-2.27)

-0.01***

(-3.18)

R2-ADJ

0.47

0.46

0.51

***, **, * indicate statistical significance at respectively the 1%, 5%, and 10% (two-

tailed). T-values are in parentheses and n = 198 for all three regressions.

(Continued on next page)

50

TABLE 7 (continued)

∆PERF_O Difference between the objective performance rating of 2003 and the

objective performance rating of 2004

∆PERF_S Difference between the subjective performance rating of 2003 and

the subjective performance rating of 2004

∆PERF_T Difference between the total performance rating of 2003 and the total

performance rating of 2004

LEN_BIAS Leniency bias, measured as the residuals of the regression of the ratio

between the subjective performance rating and the objective

performance rating on the contractual weights, the number of

performance measures and the centrality bias

CEN_BIASA Variable that equals CEN_BIAS if the observation relates to an

above-average performer and 0 otherwise

CEN_BIASB Variable that equals CEN_BIAS if the observation relates to a below-

average performer and 0 otherwise

DIS_ADJ Difference between the rewarded bonus percentage and the maximum

of the stated bonus range

REF_DISADJ Dummy variable that equals 1 if a bonus adjustment was made

within the reference group and 0 otherwise

MGROWTH Growth potential, measured as the difference between the maximum

performance rating (5) and the obtained performance rating in 2003

DIF_WO Difference between the contractual incentive weight on the objective

dimension in 2003 and in 2004

DIF_NR Difference between the number of performance measures included in

2003 and the number included in 2004

DDIF_SUP Dummy variable that equals 1 if the employee’s supervisor is

different in 2004 than in 2003 and 0 otherwise

DDIF_JOB Dummy variable that equals 1 if the employee’s function is different

in 2004 than in 2003 and 0 otherwise

MAX_SC Dummy variable that equals 1 if the employee has reached the

maximum of his salary scale and 0 otherwise

HOURS Employee designated contract hours per week

AGE Employee age a An intercept, as well as department and office dummies are included but not reported.

b For regressions with the dependent variable ∆PERF_O, ∆PERF_S and ∆PERF_T I use

the objective rating, the subjective rating and the total rating of 2003, respectively, to

calculate MGROWTH. c

For regressions with the dependent variable ∆PERF_O, ∆PERF_S and ∆PERF_T I use

the number of objective, subjective and total performance measures, respectively, to

calculate DIF_NR.

51

TABLE 8

Regression of the effects of supervisor discretion on incentive provision (II)

Independent variables

a

EFFORT

LEN_BIAS 0.39*

(1.65) CEN_BIASA -0.27

†

(-1.44) CEN_BIASB -0.72**

(-2.24) MAX_SC -0.58

(-1.43) FULLTIME 0.39

(0.97) AGE -0.02

(-1.19) SEX -0.29

(-0.81)

McFadden’s Pseudo R2

0.10

***, **, *, † indicate statistical significance at respectively the 1%, 5%, 10% and 15%

(two-tailed). T-values are in parentheses and n = 70.

EFFORT Change in employee effort. The change is measured by using the

employee’s assessment (captured on a 5-item Likert scale) of how the

new incentive system has affected his effort.

LEN_BIAS Leniency bias, measured as the residuals of the regression of the ratio

between the performance rating on the competence dimension and the

performance rating on the output-based dimension on the number of

performance measures and the centrality bias, for all those employees

who have indicated in the questionnaire that their output-based

performance measures were very measurable

CEN_BIASA Variable that equals CEN_BIAS if the observation relates to an

above-average performer and 0 otherwise

CEN_BIASB Variable that equals CEN_BIAS if the observation relates to a below-

average performer and 0 otherwise

(Continued on next page)

52

TABLE 8 (continued)

MAX_SC Dummy variable that equals 1 if the employee has reached the

maximum of his salary scale and 0 otherwise

FULLTIME Dummy variable that equals 1 if the employee works fulltime and 0

otherwise

AGE Employee age

SEX Employee gender (0 for female and 1 for male) a The intercept is included, but not separately reported.