TEST SCORE COMPENSATION, MORAL HAZARD, AND STUDENT REACTION TO HIGH-SCORER DISINCENTIVES IN...

TEST SCORE COMPENSATION, MORAL HAZARD, AND STUDENT

REACTION TO HIGH-SCORER DISINCENTIVES IN

ECONOMICS PRINCIPLES

Johnnie B. Linn III

Concord University

THE PROBLEM

• To what extent the “curve” used for exam scores generates disincentives among the better scorers,

• and to what extent these incentives are anticipated by the students in general.

HOW THE CURVE IS COMPUTED

1. If the mean score in terms of percentage of questions answered correctly is less than 77%, the percentage score of all test takers is increased by the difference of 77% and the mean score percentage.

2. If Step 1 results in a percentage value of the highest score greater than 100%, a linear transformation of the raw scores is made so that the mean is 77% and the maximum is 100%.

THE VARIABLES

• x untransformed score• y transformed score• k maximum allowed transformed score• h highest untransformed score• m linear coefficient of transformation• b constant term of transformation

THE TRANSFORMATION

bmxy

.77

,77

xh

xkhb

xh

km

THE MODEL

• What curve will be “selected” by the students?

• In what ways will the students’ performance be affected by the curve that they expect to be given?

ASSUMPTIONS

• Test takers maximize utility functions whose arguments are test score and effort.

• Effort on the margin by any one student has negligible effect on the mean score.

• Effort on the margin by any one student has negligible impact on the expected score transformation (this assumption will be relaxed later)

WHAT ABOUT GUESSING?

• We assume for now that there is no guessing.

THE UTILITY FUNCTION AND PARAMETERS

],)),(([ eexyUU

.

,

e

U

U

e

y

U

U

y

MEANING OF THE PARAMETERS

• All students have a positive benefit, , from higher test scores.

• Students for whom leisure is a normal good have positive values of .

• Students who enjoy the challenge of taking test questions will have negative values of .

THE FIRST-ORDER CONDITION

,0

)(

e

U

e

Uexy

.)( exy

EFFORT

SCORE

SCORE DISTRIBUTION, SPECIAL SCENARIOS

• Same utility function, different abilities.

• Same abilities, different utility functions.

DIFFERENT ABILITIES

EFFORT

SCORE

DIFFERENT UTILITY FUNCTIONS

EFFORT

SCORE

“INCOME” AND SUBSTITUTION EFFECTS OF THE CURVE

EFFORT

SCORE

T1

b2

T2

O

77

U2 U1

U3

T3

b3

MORAL HAZARD

• Without the curve, the student selects T1.

• With an “income-compensated” curve, the student would select T2.

• The mean-scoring student would expect an overcompensation of income--an increase in the constant term of the transformation from Ob2 to Ob3--to raise the selected score to 77 at T3.

MORAL HAZARD, CONTINUED

• For most students, for whom leisure is a normal good, the income and substitution effects offset each other.

• For students with negative , the income and substitution effects reinforce each other.

• If there is variance in , the curve enhances the effect of that variance on the raw scores.

WHAT ABOUT THE HIGHEST SCORER?

• for the highest-ranking student we must abandon the assumption made earlier that performance on the margin has a negligible effect on m and b.

• the value of y’ for the highest-ranking student is zero for values of x greater than the expected raw score of the second-highest ranking student.

INCENTIVES FOR THE HIGHEST SCORER

EFFORT

SCORE

k

CLASS SIZE AND THE HIGHEST SCORER

• For small class size, likelihood that the high scorer has a negative is less, hence, the possibility of high-scorer moral hazard is greater.

• For large class size, probability of negative is greater and expected second-highest score is greater, hence, less moral hazard for highest scorer; in the limit, none.

HIGHEST SCORER’S IMPACT ON THE CURVE

• Effect on b is negligible, since h appears both in numerator and denominator

• Effect on m is not negligible, since h appears only in the denominator.

.77

,77

xh

xkhb

xh

km

WHAT ABOUT GUESSING?

• likelihood c of a choice being correct.

• The expected opportunity cost of attempting to answer a question is c times the value of the question.

• the net marginal product of attempting a question is (1 - c) times the value of the question.

UTILITY FUNCTION WITH GUESSING

nnNcnxyUU )),()((

FIRST-ORDER CONDITION WITH GUESSING

))(( cnxy

EFFECTS OF GUESSING ON INCENTIVES

• for a given number of questions attempted, an higher value of c will be associated with a higher value of m.

• no penalty for guessing will result in less effort.

EFFECT OF GUESSING ON INCENTIVES

Effort

Score

T

A

D C

B

O N

U2

U1

1 E

F

U3

A’

DO STUDENTS ANTICIPATE THE HIGHEST SCORE?

• If students pay no attention to what h will be when they select m and b, the observed values of m and b will conform to what is expected when the envelope theorem is applied to a utility function whose arguments are only m and b.

THE ENVELOPE THEOREM

0

U Udm db

m b

0

y ydm db

m b

dbx

dm

FOR THE AVERAGE SCORING STUDENT

db dm

y b m

ln( ) ln( )y b m C

y bx

m

THE NULL HYPOTHESIS

• a plot of the ratio of the logarithm arguments should have no slope.

WHICH SIGN FOR THE ONE-TAILED TEST?

• Ratio of the two logarithm arguments should have a positive slope.

• Students expect a negative correlation between h and m.

• Transformation as viewed by students is the equation on the right.

(1 )y m ph x b

RESULT ON THE ENVELOPE THEOREM

((1 ) ) 0xd ph m db mpxdh

((1 ) ) ((1 ) )

db dhx mpx

d ph m d ph m

APPROXIMATION

dbx

dm

db dm

y b m

ln( ) ln( ) ( 1) ln( )y b m m C

THE DATA (MICROECONOMICS)

14

141

4

1411

2 343 32

32

223

324

0

10

20

30

40

50

60

70

1.20 1.60 2.00 2.40 2.80 3.20 3.60 4.00

m

b

THE DATA (MACROECONOMICS)

4

1

2

3

4

1

33

4

1

2

41

23

4

1

23

41

23

41

23

1

2

34

2

0

10

20

30

40

50

60

70

1.20 1.60 2.00 2.40 2.80 3.20 3.60 4.00

m

b

COLOR CODE FOR DATA

• Red represents the spring term of academic year 2002-2003 (second edition of text, exams from database)

• White represents the academic year 2003-2004 (second edition of text, exams by instructor)

• Green represents the fall term of the academic year 2004-2005. (third edition of text, exams from database)

FIRST EXAM EXCLUDED, MICROECONOMICS

423

322

2 3233 4 32

44

4

4

0

10

20

30

40

50

60

70

1.20 1.60 2.00 2.40 2.80 3.20 3.60 4.00

m

b

FIRST EXAM EXCLUDED, MACROECONOMICS

2

4 3

2

32

4

324

324

3 2

4

24

33

4

3

2

4

0

10

20

30

40

50

60

70

1.20 1.60 2.00 2.40 2.80 3.20 3.60 4.00

m

b

THE TEST RESULTS

• Microeconomics: Results are not significant.• Macroeconomics: Results are significant.

TEST STATISTICS

Course Sample Size

Coefficient Constant t for coefficient

Microeconomics 16 -0.020 (0.205)

2.66 (.226)

-0.099

Macroeconomics 23 0.46 (0.106)

2.16 (0.113)

4.39

TEST RESULTS, MICROECONOMICS

2.3

2.4

2.5

2.6

2.7

2.8

2.9

0.6 0.7 0.8 0.9 1 1.1 1.2 1.3 1.4

ln(m )

ln((y-b)/m)

TEST RESULTS, MACROECONOMICS

2.3

2.4

2.5

2.6

2.7

2.8

2.9

0.6 0.7 0.8 0.9 1 1.1 1.2 1.3 1.4

ln(m )

ln((y-b)/m)

WHY THE DIFFERENCE?

• High scorers in macroeconomics exhibit disincentives but high scorers in microeconomics do not do so.

• Or, the significant results for macroeconomics are due to some other unidentified factor, perhaps related to the difference in difficulty of the material.

MAJORS OF HIGHEST SCORERS

Micro Highest Scorers Business Majors Non-Business or Unknown Total Two of Three Exams 3 0 6 One of Three Exams 6 4 10

Macro Highest Scorers Business Majors Non-Business or Unknown Total Two of Three Exams 2 2 8 One of Three Exams 8 7 15

WHAT TO DO ABOUT THE CURVE?

• allow the highest score to curve to a value greater than 100%.

• High scorers can “bank” points.

• High scorer moral hazard will be reduced.

• If moral hazard is present for macroeconomics, effect of change will be greater than that for microeconomics.

PRELIMINARY RESULTS OF CHANGE, MICROECONOMICS

2.3

2.4

2.5

2.6

2.7

2.8

2.9

0.6 0.8 1 1.2 1.4 1.6

ln(m )

ln((y-b)/m)

PRELIMINARY RESULTS OF CHANGE, MACROECONOMICS

2.3

2.4

2.5

2.6

2.7

2.8

2.9

0.6 0.8 1 1.2 1.4

ln(m )

ln((y-b)/m)