Introduction

Policy Studies Institute seminar, 26 June 2007

1

Smarter task assignment or greater effort: what makes a difference in team performance?

Burgess, Propper, Ratto, Scholder, Tominey


2

Introduction

Two main motivations: Are financial incentives useful in the

public sector? Do they have a role to play in public service reform?

Compared to theoretical work, relatively little empirical evidence on incentives in organisations, and how they work.


3

What we do:

Study a pilot scheme of team-based financial incentives in HM Customs & Excise (as was)

Use rich and disaggregate dataset based on their personnel records and MIS

Research design allows diff-in-diff-in-diff, and allows us to understand in detail how the scheme worked.


4

What we find: At team level the incentive scheme did

increase performance The incentive structure did raise individual

productivity. Managers reallocated efficient workers to

the incentivised tasks; this happened disproportionately in one of the treatment teams.

This reallocation was the more important contributor to the overall outcome.

Why? Maybe career concerns …


5

Plan

(Brief) details of the scheme Data Results:

Team level Worker productivity Reallocation Decomposition

Conclusions – career concerns?


6

UK experience on PRP in the public sector November 1998 Public Services Productivity Panel. Advise the government of ways of improving the productivity and efficiency of government departments.

Programme to improve efficiency and productivity in the public sector. Makinson report (2000). Some directions and particular emphasis on teamwork.

Team-based pilot schemes in DWP and HM Customs and Excise.


7

(Brief) details of the scheme 1 Pilot ran 1 April – 31 Dec 2002, with 12

trials Team-based: team = division, a few

offices. Bonus for team as a whole. Pairs of trial teams, plus blind control,

with different bonus schemes. We use pair of VAT Assurance

divisions, team 1 (154 workers); team 2 (158); and control team (281)

Mean potential bonus about 4% salary.


8

Team 1 Team 2 Jobband 2 420 740 Jobband 3 515 740 Jobband 4 550 740 Jobband 5 453* 740 Jobband 6 720 740 Jobband 7 860 740 Jobband 8 970 740 Jobband 9 1145 740 Jobband 11 1520 740 * only observation, however, the person did not work all 275 days, so did not get the total bonus..

Bonus amount


9

(Brief) details of the scheme 2 Each incentive target was baseline + 5% Aware of multi-task issues, and controlled. Some tasks incentivised, some not. Output split into different activities, different

client groups and different outputs 5 targets: 2 on visits (time), 3 on yield (££) Note different production function effects of

time and yield.


10

Activities of VAT assurance office

Trader groups No. of audits (time)

Tax revenue (yield)

Non-trader work - Other trader work - None-core - Unregisterable entity - Trader audit work divided into the following trader groups: Non-VAT - - Non-trader audit - - New registration Low risk - - Medium risk - - High risk - - Exceptional risk Large traders - Corporate groups - - Branches - - Insolvent - - Deregistered - - Missing traders - -


11

Data

9 months of the scheme, same 9 months before

We have records for each worker, in each week, and in each trader group.

Outcomes: time allocated by each worker to each trader group (no direct measure of visits), total yield per worker per trader group, positive and negative yield per worker per trader group, and productivity.

Focus on continuously employed, front-line staff

1 Apr 2001

31 Dec 2001

1 Apr 2002

31 Dec 2002


12

Data 2 Time:

Trader audit is more than half of time Mean time per worker/week/group is 8 hrs Mean groups worked on per w/w is 3 Each incentivised group is about 30% of time

Yield: Mean +ve yield per w/w/g is £302k; but median

is just £5k outliers! One obs w/w/g of £280m. Largest share of +ve yield from exceptional

risk TG Av. –ve yield much smaller, follows same

distribution


13

Results

Team level Worker productivity Strategic task allocation Decomposition


14

Results - Team level

Exploit variation in incentivisation across: Time Treatment status Trader group

Diff-in-diff-in-diff, controlling for time effects, team effects and TG effects.

Use median for anything involving ££ Short answer: scheme worked in T2

more than T1.


15

Table 1a Table 1b Table 1c For all trader groups For all incentivised For all non-incentivised Differences between incentivised & pre-incentivised period

Differences between incentivised & pre-incentivised period

Differences between incentivised & pre-incentivised period

Variable Team

1 Team 2 C Variable Team

1 Team 2 C Variable Team

1 Team

2 C

Total Time 50.5 103.35 41.5 Total Time -3.8 73.89 -1.5 Total Time 13.5 -13.4 1.099

Total Yield 67193 73441.5 0 Total Yield 10901 31725 0 Total Yield 6406 0 0 Positive Yield 84983 80877 0

Positive Yield 8370 46550.5 0

Positive Yield 6406 0 0

Negative Yield 0 0 0



Productivity 137.54 120.13 0 Productivity 58.70 89.78 9.17 Productivity 58.24 0 0

Table 1d Table 1e Table 1f Difference-in-Difference

Difference-in-Difference

Difference-in-Difference

Variable T1-C T2-C Variable T1-C T2-C Variable T1-C T2-C

Total Time 9 61.85 Total Time -2.3 75.40 Total Time 12.4 -14.5

Total Yield 67193 73441.5 Total Yield 10901 31725 Total Yield 6406 0 Positive Yield 84983 80877

Positive Yield 8370 46550.5

Positive Yield 6406 0

Negative Yield 0 0

Negative Yield 0 0

Negative Yield 0 0

Productivity 137.54 120.13 Productivity 49.53 80.61 Productivity 58.24 0

Variable M*(T1-TC) – nonM*(T1-TC) M*(T2-TC) – nonM*(T2-TC)

Total Time -14.7 89.89

Total Yield 4495 31725

Positive Yield 1964 46550.5

Negative Yield 0 0

Productivity -8.71 80.61

Difference:

Diff in diff:

Diff in diff in diff:


16

Analysis – Worker level (unit: worker-trader group) Same five outcomes available at w/g level,

adding up over 9 months, and differencing Regress:

Specifications: With & w/out non-VAT; with & w/out

trimming Treated teams combined (intercept

dummies), and each treated team + control separately

gigiiig Incentivey γ.GTeamδXβ ...


17


18

Worker Effort and yield

These are the coefficients on incentivisation

OLS on difference in outcome (so implicitly FE).

Unit is worker/group per 9-month spell, in £

This specification is trimmed of outliers (top & bottom 5%), and excludes non-VAT outcomes.

Other controls: TG effects, team effects, years in grade, age, mean age squared, gender, part-time/full-time worker, job band.


19

At team level, treated teams spend more time on all trader groups and recover more yield than the control team.

At individual level, the increase in productivity in incentivised trader groups is larger in Team 1 than in Team 2, and statistically significant only for Team 1.

Mismatch between individual-level and team-level outcomes: individual productivity increases more in T1 than in

T2. at team level T2 performs better than T1

Summary of results

Different strategy at team level between the treated teams


20

Results – Task Allocation

Identify efficient workers in pre-scheme period as those in top quartile of productivity

Consider reallocation across a four way split: incentivised vs non-incentivised tasks, efficient vs non-efficient workers.

Is there a significant difference in the reallocation of time on the incentivised and the non-incentivised trader groups by efficient workers and other workers in each team?


21

Inc

Non-inc

Time allocation

Increasing

total time

Reallocating fixed time

Horizontal axis = time on incentivised trader groups

Vertical axis = time on non-incentivised trader groups

Teams can increase total time on audit, and/or reallocate time between the two types of groups


22

Inc

Non-inc

Time allocation

Any point (x, y) derives from:

- an overall increase in time of (x – y)/2, and

- a net reallocation of (x + y)/2.

- The latter is half of the vertical distance between (x, y) and the positively sloped 45o line.

-The gross reallocation is twice that, namely the full vertical distance between (x, y) and the positively sloped 45o line

x, y


23

Strategic task allocation: efficient workers

-122

Change in time spent on NI_TG

T2

87

T1

49

-89C

29

Change in time spent on I_TG276


24

All teams Team 1 Team 2 Control team Efficient workers 131.94 87.12 275.91 48.51 Others 56.89 35.02 98.84 45.92

t = 2.98 t = 1.50 t = 3.46 t = 0.06

Significant at 1% Significant at 10% Significant at 1% Not significant

All teams Team 1 Team 2 Control team Efficient workers -63.56 29.31 -122.17 -89.29 Others 26.04 54.05 -8.31 28.73

t = -4.30 t = -0.741 t = -2.66 t = -3.75

Significant at 1% Not significant Significant at 1% Significant at 1%

Individual level difference in time spent per month Incentivis

ed trader groupsNon-incentivised trader groups

Reallocation between incentivised and non-incentivised trader groups All teams Team 1 Team 2 Control team

Efficient workers 195.50 57.80 398.08 137.79 Others 30.84 -19.03 107.15 17.18

t = 4.56 t = 1.52 t = 3.91 t = 2.1386

Significant at 1% Significant at 10% Significant at 1% Significant at 5%

Efficient workers

Team 1 vs. Team 2 Team 1 vs. Team C Team 2 vs. Team C Team X 57.80 57.80 398.08 Team Y 398.08 137.79 137.79

t = -4.25 t = -1.13 t = 2.92

Significant at 1% Not significant Significant at 1%

Reallocation of efficient workers

Team 1 vs. Team 2 Team 1 vs. Team C Team 2 vs. Team C Team X -19.03 -19.03 107.15 Team Y 107.15 17.18 17.18

t = -2.96 t = -1.0241 t = 2.21

Significant at 1% Not significant Significant at 5%

Reallocation of other workers


25

Team 1 Team 2

Efficient worker * New registrations -15.001 (21.992)

43.716 * (25.351)

Efficient worker * Low risk -3.071 (8.022)

-26.502 *** (9.769)

Efficient worker * Median risk 14.049 (9.806)

-18.213 ** (8.882)

Efficient worker * High risk -15.169 (20.790)

-45.998 (29.948)

Efficient worker * Exceptional risk 62.801 ** (25.398)

122.785 *** (42.559)

Efficient worker * (999) Large traders -14.579 (23.095)

-55.919 (41.144)

Efficient worker * Corporate groups 0 (0.000)

-9.942 (6.982)

Efficient worker * Branches 0.495 (4.063)

0 (0.000)

Efficient worker * Insolvent 9.547 ** (3.865)

4.715 (3.299)

Efficient worker * Deregistered 3.002 (3.116)

9.241 ** (3.895)

Efficient worker * Missing traders 10.339 (6.056)

-0.232 (2.515)

Time change per trader group


26

Results - Decomposition

How much of the change in a team’s yield is due to increased worker effort, and how much to better deployment of workers?

Tijt is time spent by i on group j period t, and yijt is the corresponding productivity

Decomposition of total team yield: 21211212 ijj

ijiji

ijj

ijiji

TTTYYY


27

Change in yield due to: Team 1 Team 2 Control Team

Reallocation 10,500,514 19,785,364 4,330,667 Increased Effort 3,445,083 1,378,579 752,514 Total 13,945,597 21,163,943 5,083,181 Reallocation as % 75.3 % 93.5 % 85.2 % Increased Effort as % 24.7 % 6.5 % 14.8 %

Decomposition of the change in total yield

We have taken out the highest single yield datapoint in T2 of £280m


28

Conclusions

The incentive scheme: raised individual yield and productivity led to the reallocation of efficient workers

towards the incentivised tasks. Reallocation was the more important contributor.

Comparing the teams: The increase in individual yield collected was

essentially the same in the two teams. But T2 engaged in reallocation to a significantly

greater degree than T1 T2 hit its targets and collected the bonus. Why?


29

Conclusions 2 Why: Staff composition: more experience in T2? No.

Different managerial strategy prior the scheme? No.

Nature of bonus scheme? No. Intrinsic incentives – career concerns?

Office managers in T2 were younger, and had been in that job band for fewer years.

Division manager in T2 was much younger than in either the control or T1, and had less experience.

Managers in T2 may have been more driven to perform well for longer term career concerns


30


31

Job band 9 Job band 11

All Team 1 Team 2 Control All Team 1 Team 2 Control

gender 0.24 0.10 0.33 0.27 0.67 1.00 1.00 0.00age 50.62 52.90 46.78 51.40 48.00 48.00 40.00 56.00job_band 9.00 9.00 9.00 9.00 11.00 11.00 11.00 11.00jobband_yrs 4.09 4.60 2.78 4.53 4.33 5.00 2.00 6.00service 27.94 27.70 25.44 29.60 28.00 27.00 21.00 36.00pay 30970.32 30829.90 29558.22 31911.20 39651.00 38667.00 38279.00 42007.00gmpay 2526.78 2485.22 2449.04 2601.13 3304.25 3222.25 3189.92 3500.58gmpot 2526.78 2485.22 2449.04 2601.13 3304.25 3222.25 3189.92 3500.58otm 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00pt 0.12 0.10 0.11 0.13 0.00 0.00 0.00 0.00ptpw 28.33 24.00 . 30.50 . . . .sick 0.03 0.00 0.11 0.00 0.00 0.00 0.00 0.00days 271.61 268.90 275.00 . 275.00 275.00 275.00 .bonus 915.95 1079.40 734.33 . 1130.00 1520.00 740.00 .

Managers Profile


32

The Teams

There are 12 teams, consisting of trial sites in RBS and LBG. These are:

- VAT Assurance Division, Solent, RBS South

- VAT Assurance Division, North East, RBS Central

- Excise Assurance, RBS Scotland

- Excise Assurance, RBS North

- International Trade Assurance, RBS North

- International Trade SME Assurance, RBS Scotland

- Risk Team (Operational Analysis Division), RBS North

- Risk Team, RBS Northern Ireland

- National Function – Registration Unit

- National Function – Debt Management Unit

- National Function - Nation Advice Service

- LBG Scotland and Northern Ireland


33

Are the obligations/targets in all other areas of the team activity which are not subject of the incentives targets met?

No

Yes

The team qualifies for admission to bonus scheme

Does the team meet the baseline target and at least 50% of the stretch on all incentive targets?

No

Yes

Payment of bonus:

- 100% bonus if all the incentive targets are met in full- 50% bonus if incentive targets are not met in full but baseline target and at least 50% of the stretch is met on all incentive targets

Bonus award process


34

All staff Continuous frontline staff

Overall Team 1 Team 2 Control Overall Team 1 Team 2 Control

Gender 0.47 0.39 0.44 0.52 0.44 0.40 0.40 0.50

Age 44.2 45.7 45.4 42.8 44.6 45.0 45.5 43.7

Job band 5.9 6.2 5.9 5.8 6.2 6.3 6.3 6.0

Years in job band 3.9 4.0 4.3 3.6 4.2 4.0 4.7 4.1

Years at Customs &Excise 18.1 19.1 19.1 17.1 19.5 19.3 20.0 19.2

Annual pay (£) 20247 20797 20066 20047 20671 20742 20789 20551

Gross monthly pay (£) 1594 1630 1599 1571 1632 1619 1657 1625

Gross monthly pay + overtime (£) 1599 1632 1599 1582 1639 1621 1657 1639

Overtime per month (hours) 0.38 0.16 0.02 0.70 0.44 0.15 0.02 0.88

Part-time marker 0.16 0.16 0.13 0.17 0.16 0.19 0.12 0.16

Part-time hours per week 24.8 24.1 - 25.1 24.7 23.8 - 25.2

Days of sickness 1.1 1.1 1.5 0.9 1.0 1.2 1.3 0.8

Days in scheme 260.2 258.8 261.5 - 264.0 262.4 265.6 -

Potential bonus (£ per person) - 688 681 - - 695 696 -

Team staff mean characteristics.


35

400

06

000

800

01

0000

120

001

4000

2001m1 2001m7 2002m1 2002m7 2003m1Month

Trial Team 1 Trial Team 2Control Team

Total monthly time (hours)

Time series variation in time.

Documents

Introduction