EFFORT AND INFORMATION:

EVIDENCE FROM MBA GRADE NON-DISCLOSURE*

Eric Floyd†

Rady School of Management

University of California at San Diego

Sorabh Tomar

University of Chicago, Booth School of Business

Daniel J. Lee

Rice University, Jesse H. Jones Graduate School of Business

This Draft: February, 2017

Abstract:

In this paper, we study the effect of grade non-disclosure (GND) on student and employer behavior

in the context of an MBA program at a highly-ranked business school. GND precludes students

from revealing their GPAs to prospective employers. Our results suggest students under GND

spend 55.4 hours (5.31%) less time preparing for classes during their MBA program while

weighting their enrollment towards more difficult subjects. We also document diminished GPA-

based matching of students with employers following GND. Overall, our study provides empirical

evidence on the relationship between information and effort that underpins a broad class of

principal-agent models.

JEL Codes: D70, D82, J24, M51

* The authors are grateful to Phil Berger, Matt Bernhardt, Kevin Boyd, Eric Budish, Hans Christensen, Amoray Cragun, Alan Crane, Kevin

Crotty, Eugene Fama, Christine Gramhofer, Yael Hochberg, Emir Kamenica, Christy Leak, Christian Leuz, John A. List, Michael Minnis, Kevin

Murphy, Patricia Naranjo,Valeri Nikolaev, Ray Pauliks, Canice Prendergast, Jonathan Rogers, Shiva Sivaramakrishnan Doug Skinner, Paul Steven, Kevin Troy, Sarah Zechman and seminar participants at the University of California Irvine and the University of Chicago Booth School

of Business for helpful comments and suggestions regarding this research. The authors acknowledge financial support from Rice University Jones

Graduate School of Business. † Corresponding Author: ejfloyd@ucsd.edu

1

1) Introduction

In 1994, MBA students at the Wharton School of Business of the University of

Pennsylvania implemented the first policy allowing students to voluntarily withhold grades

from recruiting employers. Since then, grade non-disclosure (GND) policies have become

common among leading business schools in the United States. In this paper, we explore the

implications of GND policies in MBA programs and labor markets. By limiting the impact

of academic transcripts in the recruiting process, GND can promote collaboration among

students by reducing grade-based competition in the classroom. Furthermore, GND can

encourage involvement in extracurricular activities and encourage students to enroll in

more demanding but potentially more valuable and specialized classes. If GND is effective

in changing student behavior, business schools can use such policies to influence the

skillsets of their graduates (for example, graduates may develop stronger soft skills, such as

negotiating ability). Changes to MBA human capital are likely meaningful because MBA

graduates are a significant proportion of top-level decision-makers within organizations.1 2

However, GND can also reduce student effort or alter labor market matching. Both

effects can plausibly pair employers with students that are less desirable than those under

grade disclosure. Guvenen, Kuruscu, Tanaka, and Wiczer (2015) analyze the importance of

skill mismatch between the requirements of an occupation and the skill set of an employee.

They show significant negative impacts of skill mismatch on both wages and the

probability of switching occupations. GND may exacerbate skill mismatch by making it

more difficult for employers to select a student that best fits their requirements. For these

reasons, GND’s desirability has been the subject of considerable debate between students,

1 Roughly 17,970 of 147,000 high ranking company employees (e.g., CEOs, CFOs, COOs, directors, etc.) in the BoardEx database for 2012

held MBAs from the twenty-five top business schools ordered by mean student GMAT score in 2011 (see Table 1). 2 A movement towards soft and social skills would be consistent with the evidence in Deming (2015), which shows the growing importance of social skills in the labor market.

2

faculty, and school policy makers.3 Providing causal evidence on GND’s effects is

imperative to moving this debate forward.

Our approach, which uses a standard difference-in-differences framework, allows us to

plausibly identify GND’s causal effects by comparing two student groups within the same

school that differ with respect to whether or not GND applies to them. The school we

investigate has both full-time, daytime MBA programs and part-time, evening MBA

programs, but only full-time MBA students are subject to GND. Evening MBA students

provide a useful comparison to those in the full-time program because students in both

programs typically take the same classes with the same professors, but in different course

sections. Evening and full-time MBA students work through the same syllabi and are

assessed on the same rubrics and criteria. In addition, professors at the school use a

common grading curve for their full-time and evening student class sections. These

similarities hold both before and after GND’s adoption.

We utilize two novel datasets that come directly from the school’s administrators.

These datasets provide information on classroom level averages from course evaluation

surveys for all students (including self-reported student effort), as well as individual GPAs

and post-graduation employment outcomes for full-time MBA students (evening students

typically have full-time employment during the program and rarely change employers

afterwards). We estimate that students reduce subject effort by 55.4 hours during their

MBA program following the adoption of GND. An estimate of a 5.31% effort reduction

over a two-year program results from utilizing a control sample to estimate the

counterfactual, and is smaller than estimates in prior literature (e.g. Jain (1997) reports a

32% reduction in student effort). We further show a greater effort reduction in core subjects

3 See Gottlieb and Smetters (2011), Financial Times (2013), and The Economist (2005) for some examples.

3

(which all students are required to take) than in non-core subjects. This finding aligns with

core subject grades carrying more signal value and/or provide timelier information than

non-core subject grades.

We then explore several alternative explanations for our findings. Two primary

considerations are whether the effect we observe is attributable to concurrent changes in

course offerings that differentially affect full time student effort, and whether effort-averse

students select into schools with GND. First, we find that the observed treatment effect

occurs sharply around the policy implementation date. This sharp change in behavior does

not correspond with observed patterns in new course offerings, indicating a concurrent,

asymmetric change in curriculum across the two student groups does not explain our

findings. We also find that the sequence of quarter-by-quarter effort-declines across first-

and second-year subjects supports neither a selection-based nor a macroeconomic (e.g.

financial recession) explanation for effort reduction.

We next examine student course selection. We find evidence that suggests students

enroll in more challenging classes following GND. However, weighting enrollments

towards more difficult classes is not the only avenue through which GND can affect

students’ time allocation. The possibility remains that students respond to GND by trading

off class preparation effort for leisure.

If students respond to GND by altering their allocation of time across human-capital

augmenting activities, then changes in labor market matching could result. Using MBA

career preference data from Universum, an employer-branding firm, we find that the

correlation between GPA and job quality declines after the adoption of GND. This suggests

that the information in GPA is not recoverable from other sources of information available

to employers. One implication of this result is that employers could begin placing more

4

weight on other measures of student quality, such as interview performance.4 Importantly,

data on hiring decisions is only available for those subject to GND so this part of our study

does not benchmark its findings against a control group.

Our study contributes to existing literature in several ways. First, we provide direct

evidence on the effects of implementing a grade non-disclosure policy. The results of our

study are informative about the effects of other policies that change the informativeness of

signals that result from grades, such as pass/fail systems. Thus, our study informs

academics and school administrators interested in understanding the effects of policies that

vary the information derived from grades.

Second, we utilize a natural experiment that isolates the disclosure of grades. This

helps us to understand the impact of educational institutions' measurement of student

ability, in addition to their role in augmenting student ability, on student investment

decisions and job market outcomes. The results of our paper inform the debate on the value

of grades in the labor market (e.g. Weiss, 1995). Our findings reinforce the role that grades

play in how students match with employers.

The contribution of our paper is not intended to be limited to MBA labor markets; we

anticipate our paper will contribute to the extensive literature on principal-agent models,

most of which is theoretical. As Prendergast (1999) points out, empirical confirmations of

agency theory are relatively sparse, despite a large literature that shows incentives can have

large effects for motivating agent’s effort. Our setting is consistent with agency theory

because we not only show that agent’s behavior changes in response to changes in

incentives that result from increased uncertainty, but also that the implicit contract between

employers and MBA students in hiring decisions changes in response to increases in

4 A number of firm MBA recruiters that we contacted suggested this possibility.

5

uncertainty. In addition, as Prendergast (1999) points out, non-contractible effort is

unlikely to be observed by researchers, and therefore existing research typically focuses on

output measures such as number of goods produced. Our ability to observe effort is a novel

feature of our data.5

Finally, we contribute to the subset of empirical economics literature that analyzes the

impact of transparency regulation. The recent increase in the use of transparency to solve

market failures has increased the need to better understand the effects of mandatory

information disclosure on the functioning of economic markets (Fung, Graham, and Weil,

2007). Empiricists now explore the implications of mandatory disclosure, notably real

effects (Kanodia and Sapra 2016), in markets such as accounting and finance, healthcare,

and restaurant hygiene (Dranove et al., 2003; Jin and Leslie, 2003; Christensen et al., 2015,

Christensen et al., 2016). Considerable uncertainty remains regarding the impact of

disclosure policies in a diverse range of settings, and whether the results of existing

research generalize given the complexities and intricacies of different markets. By

analyzing a disclosure policy in a relatively unexplored setting - labor markets - we also

add to this growing body of empirical research.

2) Setting and Theoretical Motivation

2.1 Grade Non-Disclosure

Our study takes place at a top-ranked, U.S. business school. Before beginning their

program in 2001, the incoming full-time MBA class voted to implement GND. The policy

asks full-time students to not disclose their grades and GPAs to prospective employers, for

both on- and off-campus recruiting, and for both internship and graduate employment

recruiting. Further, overturning the GND policy at this school is a two-step process. First,

5 Foster and Rosenzweig (1994) is a notable exception.

6

25% of the incoming class must sign a petition to force a vote. Then, if at least 75% of the

incoming full-time MBA students participate in the vote and two thirds of the votes are in

favor of overturning the status-quo policy, the policy is overturned.

Under the policy, students can release their grades and GPAs upon accepting an offer or

graduating. The policy's reach extends to the Dean's list and other academic honors for

internship recruiting, but not for graduate recruiting. The policy is not enforced by the

school but deviations are subject to reputation penalties for both students and recruiters

(discussed further below). The policy does not extend to evening, executive, or weekend

MBA students. As we later argue, this feature of the GND policy allows us to explore its

causal impact.

Ciolli (2006) provides descriptive evidence on the administration and enforcement of

GND policies at business schools across the United States. Table 1 (reproduced from

Gottlieb and Smetters, 2011) lists the top 25 business schools, ordered by average GMAT

score, and whether the school has an active GND policy. He suggests that the main penalty

for non-compliance with GND is reputation-based, much like that from breaking a social

norm. Many websites, forums, and blogs publish lists of reasons for why one should

embark on an MBA - networking and alumni relations feature prominently in these lists-

which suggests a reputation penalty is non-trivial. Deviating from GND may also indicate

to an employer that a candidate is dishonest or disloyal, potentially more so if the recruiter

is an alumnus of a business school that practiced GND when the recruiter was a student.

Furthermore, it is feasible that there are reputation consequences for recruiters as well. If it

were known that a recruiter solicited grades from students, the student representative board

could reduce their promotion of that recruiter to the student body.

7

We have discussed the issue of GND violations with staff who work closely with MBA

students and have detailed knowledge of the recruiting process as well as with faculty.

While some believe that deviations are commonplace in the upper tail of the grade

distribution, the consensus is that the majority of students comply with GND. That

complete unraveling does not occur is consistent with a reputation cost to disclosing grades

(Verrecchia, 1983). That is, the benefit of disclosing (separating from non-disclosers)

exceeds the expected reputation penalties for only a select subset of students with grades

above some threshold. To the extent the policy does unravel, it increases the likelihood of

observing no consequences stemming from GND.

The benefits and costs of GND policies are contentious issues themselves, as evidenced

by different schools taking different stances towards GND (see Table 1), with schools

sometimes alternating their stance towards GND. For example, in 2005, Harvard Business

School switched from actively enforcing GND to making adherence voluntary. Proponents

of GND claim that the policy allows for more academic freedom and collaboration, and

allows students to take on more difficult and specialized subjects without the fear of grade-

based competition. MBA internet discussion forums and blogs (e.g., GMAT Club, Poets &

Quants) echo these sentiments. For example, one participant notes, “Disclosing grades may

encourage competitive attitude at the cost of collaboration. Employers value collaboration

and hence schools want to promote it.” On the other hand, detractors of GND suggest that

it leads to reduced student effort. Another participant notes, “Given that there is not much

at stake, our effort and desire to learn naturally goes down.”

2.2 Theory

Principal-agent models have been used extensively to explain observed economic

phenomena ranging from employee compensation to education. Examples of these models

8

include signaling (e.g. Spence, 1973) and multitask principal-agent models (e.g.

Holmström and Milgrom, 1991). A common feature of principal-agent models is that the

precision of information available to the principal influences the magnitude of actions

taken by economic agents as well as the contract offered by the principal to the agent.

We model GND by reproducing a multitasking model that captures the basic elements

of our setting (Holmstrom and Milgrom 1991). We consider a vector of effort that contains

two elements, e=(eh,es), which represents effort made by students in hard and soft tasks (i.e.

subject effort produces hard skills and extracurricular activities produce soft skills). Effort

produces a benefit represented by the concave function B(eh,es) and the agent’s cost is

represented by the convex function C(eh,es). The agent is compensated based on signals,

x=f(e)+ε, available to the employer. ε is assumed to be normally distributed with

covariance matrix Σ.

Holmstrom and Milgrom (1991) show that under reasonable assumptions the optimal

performance incentive is linear of the form w(x)= αhxh+αsxs+β. Under the assumption that

agents choose an allocation of effort across hard and soft tasks to maximize their certainty

equivalent, and σhs=0, then αh satisfies the following condition:

𝛼ℎ = (𝐵ℎ −𝐵𝑠𝐶ℎ𝑠

𝐶𝑠𝑠)/[(1 + 𝑟𝜎ℎ

2 (𝐶ℎℎ −𝐶ℎ𝑠

2

𝐶𝑠𝑠)]

This simple condition is sufficient to motivate the effects of GND. Grades provide one

useful signal of hard ability. When grades are no longer available, the information

concerning eh becomes less informative, or σ2h increases. As σ2

h increases, the weight

placed on signals of hard tasks by employers decreases. The weight placed on soft tasks

after GND is less clear and depends crucially on the cost function. If hard and soft tasks are

complements (Chs is negative) then increasing the weight on soft skills after GND can

9

incentivize student’s to exert effort in hard skills even when performance on hard tasks is

difficult to measure. Alternatively, if hard and soft tasks are substitutes, then increasing the

weight on soft skills will reduce the agent’s incentive to invest in hard tasks. When GND

limits the ability of employers to reward effort in hard tasks, employers can best incentivize

student’s investment in hard tasks by reducing the weight placed on soft skills.

Ultimately, whether or not agents reduce effort in hard tasks following GND depends

on the extent in which 1) employers value hard tasks and 2) hard and soft tasks are

complementary in the agent’s cost function in such a way that rewarding soft tasks can

preserve effort in hard tasks. Alternatively, employers may be able to minimize the effort

loss in hard tasks by reducing the extent to which they reward soft tasks if the tasks are

substitutes. Finally, as the measurability of hard tasks decreases, students may begin

placing weight on the relatively better measured task, soft skills, if employers increase the

weight on soft skills in the wage function.

3) Empirical Predictions

Our primary empirical prediction is that GND reduces student effort in classes. All

MBA students at the business school we study are required to complete the following core

subjects, typically in their first year: Business Statistics (or Applied Regression Analysis),

Financial Accounting, and Microeconomics. Thus, these subjects' grades allow employers

to rank students across the full spectrum of student types. If grades have informational

value, grades in core subjects are likely the most informative. Therefore, one potential

consequence is that the effort drop following GND is larger for core subjects, relative to

non-core subjects.6

6 A greater effort reduction for core subjects has an alternative explanation. First year internship recruiting at the school we study typically

begins in January, while second year full-time recruiting typically begins the following October. In both cases, second year grades are not

available for evaluation to recruiters. Thus, a larger decrease in effort for core subjects following GND would also be consistent with first year subjects (both core and non-core) being the only ones that produce timely information. Unfortunately, we cannot identify non-core subjects

10

An effort decrease should occur during the first-year courses for the 2001-incoming

cohort (as compared to 2000-incoming). This is because the 2001-incoming cohort voted

in favor of GND before beginning their MBA program. When they did, GND applied not

only to their cohort, but to all subsequent cohorts as well. Accordingly, we expect to see

no drop in first year effort when comparing the second non-disclosing cohort to the first

non-disclosing cohort (2002-incoming vs. 2001-incoming respectively). We expect to see

no change in effort for second year subjects across any of the years since second year

subject grades lack generalizability to all students, and lack timeliness - second year

subjects are more specialized, and the bulk of recruiting occurs in October, before second

year grades are acquired. Table 2 summarizes these predictions across the two years.

A purported benefit of GND is that it allows students to move out of their academic

comfort zones. Under a grade disclosure regime, students may be compelled to take

subjects that they feel are easier or less adventurous, fearing competition based on grades.

Thus, it is possible that students enroll in more difficult or specialized subjects following

GND. Furthermore, students may use the academic freedom afforded to them under GND

in order to pursue activities that are not directly related to coursework, such as club

activities and networking events.

4) Data

We use an administrative dataset from the business school that reports the mean values

from course evaluation surveys completed by students at the end of each academic quarter.

The data cover classes from autumn 1997 to autumn 2013. Each observation corresponds to

an instance of a class taught by a specific instructor in a given quarter-year (instructor-

that predominantly first years take. Doing so would allow us to separate the effects of signal generalizability (i.e., how wide the support of

student types is) and signal timeliness.

11

subject-quarter-year). Henceforth, we refer to each observation as a “classroom”. We

observe course evaluation data for 5,867 classrooms, and do not observe evaluation data for

80 classrooms. Thus, our sample is relatively complete. The main variables we construct

from the database are as follows: HOURS is the classroom average response to the question

“Excluding class sessions, estimate the average number of hours per week spent in

preparation or review”; FULL is a dummy variable that takes the value 1 for full-time

MBA observations, and 0 otherwise.

FULL information comes from subject codes indicating which classrooms were held in

the downtown campus during weekdays. These classrooms are designated for evening

MBA students; though full-time MBA students may also enroll in them. Any enrollment in

evening classrooms by full-time MBA students (and vice versa) will bias our statistical

tests towards the null since it acts to homogenize the treatment and control groups.

5) Student Response to GND

5.1 Counterfactual

To make causal inferences about GND’s effects on student behavior using a difference-

in-differences design, a valid control group is needed. GND, at the school we study, applies

only to full-time MBA students. We can therefore consider evening MBA, weekend MBA,

executive MBA, PhD, and undergraduate students as potential samples to estimate the

counterfactual. To obtain unbiased estimators of GND’s effects, our estimation strategy

must ensure that by classifying our observations as either treatment or control we do not

sort on unobservable characteristics correlated with the dependent variables of interest.

Executive MBA students, PhD students and undergraduate students are inherently

different from full-time MBA students due to differing job market incentives, demographic

12

profiles, and/or course materials. Therefore, we do not consider them suitable control

groups because the likelihood of correlated omitted variables is high.

Evening MBA students and weekend MBA students are more likely to have

characteristics similar to those of full-time MBA students. In Table 3 we present

descriptive statistics on the school’s students. In 2011, the average ages of full-time MBA

students, evening MBA students, and weekend MBA students were 28, 29, and 30

respectively. The average lengths of prior full-time employment were 5 years, 5 years, and

6 years respectively. In addition, evening and weekend MBA students take many of the

same courses as full-time MBA students. These courses are taught by the same instructors,

use the same teaching assistants, follow the same syllabi, provide the same learning

materials, and involve the same assessment criteria. Instructors also use a common grading

curve across their evening and full-time sections.

Evening MBA students and weekend MBA students typically begin their studies with

secured employment. Therefore, at first glance, their job market incentives may seem

markedly different, i.e., evening and weekend students may appear to have fewer

incentives to focus on grades. However, anecdotal evidence suggests these students

typically aim for promotions, partake in recruiting events (including recruiting events

specifically for part-time students), and often have their studies funded by their employers,

conditional on a minimum academic performance level. Thus, the grades of evening and

weekend MBA students still yield informational value in the labor market. We focus on

evening MBA students since they live in the same city as full-time MBA students and are

exposed to the same local shocks, whereas weekend MBA students often fly in from

around the country.

13

A large concern regarding identification in our setting is that there exist changes in

course offerings over time that differentially impact the treatment and control groups. Our

analysis might be distorted if the school we study offers courses after GND that are both

more time intensive and only taught in the full-time program. For example, interactive lab

classes were introduced after GND; we discuss later how our analysis accounts for this

issue.

To motivate that our use of evening MBA students as a control group is a reasonable

empirical strategy, we plot full-time and evening students' average academic effort in all

subjects, core subjects, non-core subjects, a subset of non-core subjects, and a finer subset

of non-core subjects, in Figures 1a-1e, respectively.

First, in Figure 1a there is no sign of a drop in effort for full-time MBA students around

the policy date. In Figure 1b, however, there is a discontinuous comparative drop in full-

time MBA student core subject effort in 2001 that persists into successive years. This is

consistent with GND reducing incentives to exert effort in core subjects. The drop likely

occurs within subject since these subjects are compulsory and students cannot substitute

away from them.

Second, in Figure 1c, there are comparative increases in full-time MBA student non-

core effort in 2003 and 2005 to 2008. This can be driven by students exerting more effort

within non-core subjects or by a change in the course offering.7

To disentangle the above explanations, we examine non-core subjects that were

available to full-time MBA students pre-and post-GND, along with subjects that were

7 While students can substitute towards more difficult subjects, this would not explain the increase unless substitution was so prevalent that new class sections were added or removed for some subjects in response to changed demand. This is because Figures 1a-1e are based on

reported subject effort at the classroom level—each point is a mean of means. Thus, if only a few students move across classrooms, the mean

of means remains unchanged unless there is sufficient heterogeneity in student effort, and a selected sample of students substitute towards difficult subjects. Subject substitution is discussed later in Section 4.3.

14

available to evening MBA students pre-and post-GND. This involves dropping 677 of

5,063 non-core observations. The resulting effort plot, Figure 1d, eliminates changes in

effort driven by changes in the course offering, since the subsample focuses on a fixed

course offering. The increases in full-time non-core effort in 2003 and 2005 to 2008 are

somewhat reduced. Thus, changes in the course offering drive a portion of the effort

increases for full-time MBA students. Discussions with staff involved with course

planning indicated that changes in the course offering around GND were not spurred by

changes in student demand following GND.

We then focus on subjects that were available to both full-time and evening MBA

students, both pre and post-GND. This involves dropping a further 566 observations. The

resulting effort plot, Figure 1e, additionally eliminates subjects available only to full-time

or only to evening MBA students. In this sense, it is the most comparable set of non-core

subjects for full-time and evening MBA students. The increase in effort after GND for full-

time students is even further reduced. Overall, this highlights the potential need to control

for course offering changes.

5.2 Tests of Reduced Student Effort

To test our main prediction, a reduction in student effort following GND, we estimate

the following difference-in-differences model using OLS:

𝐻𝑂𝑈𝑅𝑆𝑛𝑖𝑠𝑔𝑦 = 𝛼 + 𝛽1𝐹𝑈𝐿𝐿𝑔 + 𝛽2𝐺𝑁𝐷𝑦 + 𝛽3𝐹𝑈𝐿𝐿𝑔 × 𝐺𝑁𝐷𝑦 + ∑𝑘𝐹. 𝐸 + 𝜀𝑛𝑖𝑠𝑔𝑦 (1)

g denotes MBA student type (evening or full-time), y denotes year, i denotes instructor,

s denotes subject, and n indexes the instances of isgy combinations (for example, if

Professor X teaches financial accounting to two different classrooms of full-time MBA

students each year, n = 1,2 for those classrooms). FULL is a dummy set to 1 for full-time

MBA classrooms, and GND is a dummy set to 1 when y ≥ 2001 (the year of GND

15

adoption). The coefficient of interest is β3. It is interpreted as the change in HOURS that

full-time MBA students record when moving to GND, incremental to the change in

HOURS that evening students record over the same period. Standard errors are clustered at

the instructor-subject level to account for serial correlation.

Many schools require faculty to teach in only the full-time, or evening, MBA programs.

The school in our analysis, however differs by often scheduling faculty to teach the same

subject in both programs. Faculty typically teach their full-time classes in the afternoon and

immediately travel to teach an evening MBA class (1,821 classrooms are taught under this

setup). This feature lets us exploit instructor-subject-year and instructor-subject-FULL

interaction fixed effects. These fixed effects let us abstract from effort variation caused by

changes in the instructor/subject menu offered to student types across years. They also

negate the effect of instructors and subjects offered to only one student group, or in only

one of the pre- or post-periods. Thus, our design provides estimates within instructor-

subject (the variation we exploit is between treatment and control group, within the same

instructor and subject pair, around the GND implementation date).8

Columns 1 to 4 of Table 4 show the results from estimating variants of Model 1. β3, the

coefficient estimate on FULL×GND, ranges from 0.513 to -0.398 across the different

specifications. The difference between columns 1 and 2, which is explained by the

inclusion of fixed effects, reinforces the need to account for changes in course offerings

around GND implementation. We supplement this finding with the specification in column

3, which requires courses to be available both before and after GND to ensure a

comparable sample. As in column 2, β3 is significantly negative.

8 As an example, full-time students, but not evening students, enroll in certain lab subjects. These subjects are effort intensive, and require

varied effort across years. From discussing these subjects with their instructors, the required effort variation is not due to GND related factors. Thus, including these subjects would contaminate our results.

16

The estimate of β3 in column 2, -0.277, implies a subject effort decrease of 16.6 minutes

per subject per week under GND. Given that the program requires students to complete

twenty 10-week courses, this translates to 55.4 fewer hours over the program. The mean

number of hours spent per week by full-time MBA students on subjects pre-GND is 5.22,

suggesting a 5.31% student effort reduction under GND. This is substantially below the

32% mentioned in Gottlieb and Smetters, who cite Jain (1997). Jain (1997) uses a pre-post

design to measure the effort reduction following GND at The Wharton School. However,

it is imperative to account for effort time trends, as demonstrated in figures 1b, 1d, and 1e.

To test our second prediction, that the effort reduction is greater for core or first year

classes, we re-estimate Model (1) after interacting the independent variables with dummies

indicating core subjects and dummies indicating non-core subjects. Column 5 of Table 4

supports that the impact of GND is greater in economic magnitude for core subjects than

for non-core subjects. This difference, however, is not statistically significant, (p-value of

0.529). A potential explanation for this is the small number of identifying core subject

observations (n=664) compared to the number of identifying non-core subject observations

(n=3,883). While these results are consistent with greater signal generalizability and/or

signal timeliness for core subjects, given the limited sample size, they should be interpreted

cautiously.

When testing our prediction that the effort drop should be isolated to first year students

we focus closely on GND’s implementation date; however, we have no indicator for

whether a classroom predominantly consists of second year MBA students. To circumvent

this, we test the predictions outlined in Table 2 using autumn quarter observations. First

year students are encouraged to complete their core subjects in their first autumn quarter.

Thus, we assume that mostly first year students enroll in core autumn classrooms

17

(Microeconomics, Financial Accounting, Applied Regression Analysis and Business

Statistics), and mostly second year students enroll in non-core autumn classrooms. We use

instructor-subject fixed effects, which have a coarser structure than the fixed effects used in

Table 4, since our focus on autumn classrooms drastically reduces our sample size.

The results are shown in Table 5. Consistent with the predictions outlined in Table 2,

column 1 shows a first year subject effort decrease when moving from the last disclosing

cohort (2000 incoming) to the first non-disclosing cohort (2001 incoming). Column 3 does

not show a first year subject effort decrease when moving from the first non-disclosing

cohort (2001 incoming) to the second non-disclosing cohort (2002 incoming). Consistent

with second year subjects not being timely signals for employers, columns 2 and 4 show no

statistically significant effort decreases for second year subjects.9

5.3 Assessing Identification Assumptions

Parallel Trends:

To ensure that our results are not driven by pre-existing trends in the treatment-control

group effort differential, in Figures 2a and 2b we map out the treatment effect by plotting

the coefficients on year x indicators (Dx) interacted with FULL, the treatment group

indicator, in the following regression:

𝐻𝑂𝑈𝑅𝑆𝑛𝑖𝑠𝑔𝑦 = 𝛽1𝐹𝑈𝐿𝐿𝑔 + ∑ (𝛽2,𝑥𝐷𝑥 + 𝛽3,𝑥 × 𝐹𝑈𝐿𝐿𝑔)

2004

𝑠=1998

+ 𝜀𝑛𝑖𝑠𝑔𝑦 (2)

Figures 2a and 2b provide support for a causal effect of GND on student effort. 2000,

the last pre-GND year, is the benchmark year. There are no indications of a differential

full-time and evening student trend in the years leading up to GND, and we observe a sharp

9 While the magnitude of the decrease in column 2 (second year subjects, moving from the last disclosing cohort to the first non-disclosing

cohort) stands out, this becomes expected when we consider that not all first year students complete their core subjects in their first quarter, i.e., some first year students take subjects that we label as second year subjects.

18

decrease in effort around the policy date. Interestingly, when we limit the sample to core

subjects there appears to be a reversal in the decrease in student effort. One interpretation

of this finding is that evening students begin to comply with GND over time even though it

does not explicitly apply to them.10

We plot the coefficient from 1998 to 2004 for two reasons. First, in this specification,

fixed effects will not allow us to account for changes in course offerings since the treatment

effect is measured year by year. Limiting the time period to a symmetric period around the

GND implementation date makes it more likely that course offerings are balanced

throughout the sample. Second, the school we study completed construction of a new

building in 2004 and full-time MBA students began taking classes there in the 2005

academic year. The increased resources and capabilities of the building may have led to

increased effort from full-time MBA students. It may also have allowed the administration

to offer new, or adjust existing, courses. In either case, we expect there to be a differential

shock to full-time MBA students in 2005 unrelated to GND, and thus, we narrow our

sample window.

To ensure that our inferences are not sensitive to the sample window choice, we re-

estimate Model (1) and limit our sample to 1998-2004. The results, shown in column 4 of

Table 4 are almost identical to those in column 2. As observations fall further in time

beyond 2001, it becomes less likely that the corresponding instructor-subject pairs also

have observations falling in the pre-period. This, however, is a requirement for

observations to contribute to our estimation of the GND treatment effect.

Selection:

10 University staff have informed us that many evening MBA students prefer to present themselves in a similar manner to full-time students, and thus also comply with GND.

19

Students voluntarily adopted GND. It may be that the 2001 incoming class was

unusually effort averse, driving the popular vote in favor of GND, and also the observed

reduction in effort. If GND was the result of an unusually effort averse incoming class,

then one wouldn’t expect the treatment effect to persist into subsequent years. The fact that

we observe a persistent treatment effect means that either GND is causing the reduction in

effort or GND causes subsequent classes to also be effort averse. In other words, low effort

individuals may begin to select into programs with GND.

Table 5 provides evidence against a selection of effort averse students. If the 2001

incoming class was simply more effort averse, we would expect to see a comparable

decline in second year subject effort in 2002. In other words, to explain our results, the

2001 class would have to have been unusually effort averse only towards first year classes.

In summary, our analysis supports the fact that GND causes a reduction in effort and this

reduction appears to operate through a reduction in incentives to exert effort as opposed to

selection.

Financial Recession:

The early 2000s recession affected economic activity during our sample period. If

recessions affect full-time MBA students and evening MBA students differentially, this

may drive our results. For instance, the 2001 incoming class may have been especially

wary of the economic climate, and instead focused on recruiting activities rather than

academics. To address this concern, we note that we begin observing the effort decline in

fall 2000 (which is the start of the 2001 academic year). This is months before March 2001,

when the stock market reached its peak before the dot-com bubble burst.11

Concurrent Change in Course Offerings:

11 The NBER indicates March 2001 to be the start of the recession.

20

Our main estimation relies on subject-instructor fixed effects that remove the average

effect for each subject-instructor pair. Given this structure, however, an alternative

explanation is a shift in effort towards classes not in the pre-period. This explanation

would be concerning if, after GND implementation, there was a large increase in new

courses offered that received a large amount of student effort. For example, consider two

courses: an existing class offered in both the pre and post-periods and a new class offered

only in the post period. Imagine that overall effort remains constant while students

substitute effort towards the new class. Our estimation would suggest a decrease in effort

because only classes offered in both the pre and post periods provide identifying variation

in effort changes. In this scenario, however, there is no causal effect of GND on effort.

We examine this possibility in Figure 3, where we define a new course as one that is

not offered in prior years (with 1998 being the “first” year). In contrast to the scenario

illustrated above, fewer new courses are developed from 1999-2003. Effort spent on these

new courses aligns generally with the effort expenditures throughout our dataset, and are

not high enough to invalidate our results. Furthermore, Figure 3 shows the trend in new

course offerings does not resemble the treatment effects in Figures 2a and 2b; we do not

see a sharp increase in course offerings that occurs only with GND’s implementation.

Taken together, changes in course offerings unlikely drive our main result.

Additionally, using fixed effects could induce a treatment effect if full-time students

substitute effort towards classes that are taught both before and after GND, but only for

full-time students. Interactive lab classes fit this description. Substitution of effort towards

these classes account for the large increase in effort in Figure 1 Panel A. Importantly, any

treatment effect induced by lab classes appears to occur in 2003, which is after the sharp

decrease in effort we observe around GND. Furthermore, the enrollment for lab classes is

21

weighted towards more advanced students, which is inconsistent with the treatment effect

in Table 5 being focused in first year classes.

5.4 Tests of a Shift Towards More Difficult Subjects

When grades are disclosed, students have incentives to engage in GPA-augmenting

activities such as enrolling in easier classes. As such, a potential effect of the GND setting

is a shift towards enrollment in more demanding but potentially more valuable and

specialized classes. The test for sensitivity to subject difficulty requires not only measures

of subject difficulty, but also subject popularity. Our measure of popularity for subject s

among student type g (e.g., full-time MBA) is the proportion (in basis points) of student

type g's non-core enrollments comprising of subject s enrollments in quarter q of year y,

i.e.:

𝑃𝑂𝑃𝑈𝐿𝐴𝑅𝐼𝑇𝑌𝑠𝑔𝑞𝑦 =type g enrollments in non-core subject s in quarter-year, qy

number of type g non-core enrollments in quarter-year, qy (3)

We focus only on non-core enrollments because core subjects are largely compulsory

for MBA students. Further, we exclude summer quarter classrooms, which account for less

than 10% of our sample, because summer course offerings are much more limited.

Subject difficulty, DIFFICULTY, is measured using mean HOURS calculated at the

subject level. Thus, we assume that difficult subjects require more time. We discuss this

assumption further below.

To formally test for a shift towards more difficult subjects following GND, we restrict

the sample to only full-time classrooms and estimate the following model:

𝑃𝑂𝑃𝑈𝐿𝐴𝑅𝐼𝑇𝑌𝑠𝑡𝑞𝑦 = 𝛼 + 𝛽1𝐷𝐼𝐹𝐹𝐼𝐶𝑈𝐿𝑇𝑌𝑠 + 𝛽2𝐺𝑁𝐷𝑞𝑦 + 𝛽3𝐺𝑁𝐷𝑞𝑦 × 𝐷𝐼𝐹𝐹𝐼𝐶𝑈𝐿𝑇𝑌𝑠 + ∑ 𝐹𝐸𝑘

+ 𝜀𝑠𝑡𝑞𝑦 (4)

We use subject area fixed effects to explore the within-subject area margin of

substitution (e.g., a student switching from an easier accounting class to a more difficult

22

accounting class). Subject areas are collected from yearly student handbooks. We expect β3

to be positive if students become less responsive to subject difficulty and the potential for

poorer grades after GND.

The results from estimating Equation (4) are provided in Table 6. Column 1 uses the

sample of full-time classrooms described above. We see a positive shift towards more

difficult courses after the implementation of GND. As a placebo, column 2 uses only

evening classrooms - no reduction in the sensitivity to subject difficulty is observed. Thus

within subject area, difficult subject areas tend to be less popular, though, exposure to GND

appears to induce a shift towards more difficult classrooms. Finally, we re-estimate

Equation (4) using a constant composition sample that requires subjects to be available for

full-time MBA students pre and post-GND, and also available for evening MBA students

pre and post-GND, and obtain similar results.

To sum, the results in this section suggest that under GND students tend to substitute

towards more difficult classes within subject area. An alternative explanation is that

HOURS measures content volume as opposed to difficulty. However, in a highly-ranked

business school, we expect these two concepts are closely related. Further, in the context

of a principal-agent model, GND reduces the costs of a lower GPA from both content

volume and difficulty. Thus both cases lead to the same directional prediction.

6) Labor Market Outcomes

6.1 Predictions and Data

Effort reallocation is consistent with a student response to a decreased sensitivity of

employment outcomes to GPA. Under this explanation, we expect the relation between

GPA and employer desirability to diminish following the adoption of GND. We focus on

employer desirability rather than salary because salaries are empirically sticky, but not

23

employee-employer match outcomes.12 The relation between GPA and employer

desirability might persist, however, if other information that employers use to select

students (e.g., GMAT scores, interview performance, undergraduate GPAs) preserve the

ordering of MBA GPA. In other words, following GND employers might use other signals

or indicators that substitute for GPA.

Our employer rankings are obtained from Universum, who collect career preference

data from students, including MBAs. Universum rankings are based on a two-step process

in which MBA students are first asked which employers they are considering, and then

which of those employers are ideal. The rankings are based on the number of respondents

selecting each employer as ideal. Our Universum data spans 2000 to 2014 - only 2000 falls

in our pre-period. To increase the length of our pre-period, we extrapolate the year 2000

rankings to prior years.13

We use a dataset from the business school we study that contains student GPAs, GMAT

scores, and employment outcomes at both the internship and graduate recruiting stages. In

our data, we observe these academic variables and internship (graduate) recruiting

outcomes only for full-time cohorts starting during the academic years 1999 (1998) to 2007

(2006), inclusive. Unfortunately, this means we cannot construct a counterfactual in these

analyses. Accordingly, we view this analysis as providing evidence consistent with the

results presented earlier in the paper where we are more comfortable drawing causal

inferences.

Finally, in concordance with confidentiality laws, the data are anonymized such that we

cannot link students in the data to their real life identities. Accordingly, our data only

12 In untabulated results, we find the salary and signing bonus differences between high and low GPA students does not change with respect to

GND 13In untabulated results, we use only the 2000 Universum rankings for all years, and find that our conclusions remain unchanged.

24

allow us to link a student's GPA/GMAT to their employer when that employer hired

multiple candidates for a particular position type (internship or graduate) in a given year.

6.2 Empirical Specification

We examine the relation between employer rank and GPA using yearly regressions of

employer rank on GPA, with GMAT included as a control. GPA and GMAT are interacted

with dummy variables denoting each year in our sample to allow a flexible relation

between these variables and employer rank over time;14

𝑅𝐴𝑁𝐾𝑖𝑗𝑦 = ∑ (𝛽1,𝑥𝐷𝑥 × 𝐺𝑃𝐴𝑖 + 𝛽2,𝑥𝐷𝑥 × 𝐺𝑀𝐴𝑇𝑖) + 𝐹𝐸 + 𝜀𝑖𝑗𝑦

2006

𝑥=1999

(5)

Since students can disclose their GMAT scores under GND, we are interested in the

portion of GPA orthogonal to GMAT score. When a student matches with the same

employer twice, i.e., once for an internship and once for a graduate position, we select the

internship match. GPAs are measured at the end of the first fall quarter for internship

recruiting, and at the end of the first summer quarter for graduate position recruiting. We

are thus using the GPAs that students disclosed, or were precluded from disclosing, when

they first engaged in recruiting activity with their matched employer.

Table 7 shows the results from Model (5). The coefficients on GPA become less

negative moving from 2000 to 2001 (the year of GND implementation). A negative

correlation indicates that better GPA students match with better-ranked employers). The

decrease is statistically significant with a p-value of 0.015. This aligns with GND

diminishing the GPA-employer rank relation by concealing useful data from recruiters.

The sharpness of the decrease, while not a substitute for a control group, gives us

confidence that GND is causing the change.

14 The GMAT (Graduate Management Admission Test) is an aptitude test and a key business school student admission criterion.

25

The coefficient estimates on GPA in 2000 are the most statistically significant. The

2000 incoming class could have been exposed to abnormally high GPA-based matching.

The 2001 incoming class may have then responded by implementing GND. This can occur

in both the case where students know their ranking and the case where students do not

know their ranking prior to joining the program (i.e., a ‘Rawlsian veil’ setting).15 It is then

unclear whether GND per se, or mean-reversion, diminishes the GPA-employer rank

relation. A mean-reversion explanation, however, does not explain reduced student effort

unless students anticipate the eventual extent of GPA-based matching and set their effort

levels accordingly. We view this as unlikely.

To examine further whether employers substitute towards other measures of academic

ability following GND, we compute correlations between GMAT and employer rank by

year. GMAT is potentially a measure of at least one dimension of intrinsic student ability.

Table 8 shows that with the exception of 1999 and 2003, none of the years exhibit

significant negative correlations between GMATs and employer rank, even after GND.

Thus, GND does not appear to induce employers to utilize GMAT scores in their hiring

decisions. Our interpretation is that GPAs are incrementally informative to GMAT scores.

7) Conclusion

Using the MBA student setting at a high-ranking business school, we provide empirical

evidence that changes in information precision through disclosure policy affect economic

actions, in particular human capital investment through academic effort. We find that

following the adoption of a grade non-disclosure (GND) policy intended to preclude the

release of student grades to prospective employers, students reduce academic effort by 55.4

15 If sufficiently risk-averse, students unaware of their ranking amongst their peers would prefer a policy that allows them to pool with others in the case that they turn out to be of low rank, despite being unable to distinguish themselves in the case they turn out to be of high rank.

26

hours over the two-year program. Supporting evidence is consistent with students reducing

effort as a response to employers’ declining use of GPA in hiring decisions. We find some

evidence of students weighting their enrollments towards difficult subjects in response to

GND, in support of the arguments of GND proponents.

To provide context to our analysis, we conducted interviews regarding the results of our

study. We contacted 20 employer-recruiting departments to learn about their policies

regarding GPA. Although many did not respond, the consensus among those that did was

that GPA is used primarily as a preliminary screening device. Therefore, the institutional

driver for our results may be that students put effort into achieving high GPAs to avoid

being ruled out for a particular job, not necessarily to out-compete other students that also

meet the screening requirements. When GPAs are not provided, employers tend to focus on

non-GPA elements of resumes and on-campus interactions with students.

It is important to know whether or not GND is welfare improving. Ultimately, this

analysis provides evidence on behavioral responses to GND but is unable to determine the

overall desirability of GND. There are likely many costs and benefits to GND that we do

not explore, but one important reason we cannot conjecture GND’s overall impact is our

inability to observe firm-level changes in productivity that result from employers hiring

new employees with new skillsets.

At first glance, the informativeness principle (Holmstrom 1979) would suggest that the

change in matches between employers and employees reduces productivity because

employers would rationally ignore GND if it was not a useful signal. However, agency

problems within the firm make this conclusion ambiguous. Hiring departments may hire

students with higher GPA not because they are the best fit for the employer, but because

they are the easiest candidates for hiring departments to justify to those they report to.

27

GND may finally give hiring departments the flexibility to hire without relying on GPA,

which could potentially improve firm-level productivity if the reliance on GPA was

causing firm’s to invest in suboptimal matches.

In addition to providing evidence on the behavioral effect of GND specifically, our

study is likely of interest to many audiences within economics that use the principal-agent

model as a workhorse for explaining economic phenomena. Despite the theorized

importance of information precision in markets, there is a paucity of empirical evidence

demonstrating the magnitude of its relation with economic outcomes (e.g. Hong, Hossain,

List, and Tanaka, 2013) because few settings exist in which the requisite counterfactual

actions are observed and information precision varies exogenously (Floyd and List, 2016).

Given the prevalence of principal-agent models in economic theory, confirming the central

implications of these models and establishing a causal relation between information

precision and economic outcomes has far reaching implications. The GND setting affords

us plausible exogeneity and allows us to provide causal evidence concerning the impact of

information precision on economic actions for both the principal and the agent.

28

References

Al- Ubaydli, Omar, et al. "Carrots that look like sticks: Toward an understanding of

multitasking incentive schemes." Southern Economic Journal 81.3 (2015): 538-561.

Christensen, Hans Bonde, et al. " The Real Effects of Mandated Information on Social

Responsibility in Financial Reports: Evidence from Mine-Safety Records." Available at

SSRN 2680296 (2016).

Christensen, Hans Bonde, Eric Floyd, and Mark G. Maffett. "The Effects of Charge-Price

Transparency Regulation on Prices in the Healthcare Industry." Chicago Booth

Research Paper 14-33 (2015).

Ciolli, Anthony. "Grade Non-Disclosure Policies: An Analysis of Restrictions on MBA

Student Speech to Employers." U. Pa. J. Lab. & Emp. L. 9 (2006): 709.

Daley, Brendan, and Brett Green. "Market signaling with grades." Journal of Economic

Theory 151 (2014): 114-145.

Deming, David J. The growing importance of social skills in the labor market. No. w21473.

National Bureau of Economic Research, 2015.

Dranove, David, et al. "Is More Information Better? The Effects of “Report Cards” on Health

Care Providers." The Journal of Political Economy 111.3 (2003): 555-588.

Floyd, Eric, and John A. List. "Using Field Experiments in Accounting and Finance." Journal

of Accounting Research 54.2 (2016): 437-475.

Foster, Andrew D., and Mark R. Rosenzweig. "A test for moral hazard in the labor market:

Contractual arrangements, effort, and health." The Review of Economics and

Statistics (1994): 213-227.

Fung, Archon, Mary Graham, and David Weil. Full disclosure: The perils and promise of

transparency. Cambridge University Press, 2007.

Gottlieb, Daniel, and Kent Smetters. Grade non-disclosure. No. w17465. National Bureau of

Economic Research, 2011.

Guvenen, Fatih, et al. Multidimensional Skill Mismatch. No. w21376. National Bureau of

Economic Research, 2015.

Holmstrom, Bengt, and Paul Milgrom. "Multitask principal-agent analyses: Incentive

contracts, asset ownership, and job design." Journal of Law, Economics, &

Organization 7 (1991): 24-52.

Hong, Fuhai, et al. Testing the theory of multitasking: Evidence from a natural field experiment

in Chinese factories. No. w19660. National Bureau of Economic Research, 2013.

Jain, Anjani. Rethinking grade nondisclosure. An address to faculty and students, University of

Pennsylvania, 1997.

Jin, Ginger Zhe, and Phillip Leslie. "The Effect of Information on Product Quality: Evidence

from Restaurant Hygiene Grade Cards." The Quarterly Journal of Economics (2003):

409-451.

Keser, Claudia, and Marc Willinger. "Principals’ principles when agents’ actions are

hidden." international Journal of industrial Organization 18.1 (2000): 163-185.

Lazear, Edward P. "Performance Pay and Productivity." The American Economic Review 90.0

(2000): 1346-1361.

"News from the schools: Grade concerns at Harvard." The Economist. The Economist

Newspaper, 16 Dec. 2005. Web. 18 Jan. 2017.

Prendergast, Canice. "The provision of incentives in firms." Journal of economic

literature 37.1 (1999): 7-63.

29

Prendergast, Canice. "Uncertainty and incentives." Journal of Labor Economics 20.S2 (2002):

S115-S137.

Singer, Johanna. "The Perks of GND." Financial Times. N.p., n.d. Web. 25 Jan. 2016.

Spence, Michael. "Job market signaling." The quarterly journal of Economics (1973): 355-374.

Verrecchia, Robert E. "Discretionary disclosure." Journal of accounting and economics 5

(1983): 179-194.

Weiss, Andrew. "Human capital vs. signaling explanations of wages." The Journal of

Economic Perspectives 9.4 (1995): 133-154.

30

Tables and Figures

Figure 1a: Hours Spent Preparing for Subjects

Figure 1b: Hours Spent Preparing for Core Subjects

31

Figure 1c: Hours Spent Preparing for Non-core Subjects

Figure 1d: Hours Spent Preparing for Non-core Subjects Offered Pre and Post-GND for Full-

time MBA Students, or Pre and Post-GND for Evening MBA Students

32

Figure 1e: Hours Spent Preparing for Non-core Subjects Offered Pre and Post-GND for Full-

time MBA Students, and Pre and Post-GND for Evening MBA Students

Notes: The data are from a highly-ranked business school’s course evaluation database, aggregated at the classroom level, spanning autumn 1997 to autumn 2013. A “classroom” is an instance of a subject, taught by an instructor, in one academic quarter. HOURS for a class is the average response to “Excluding class sessions, estimate the average number of hours per week spent in preparation or review.”

33

Figure 2a: Yearly Treatment Effects for All Subjects

Figure 2b: Yearly Treatment Effects for Core Subjects

Notes: The figures above plot estimates of β3,x from estimating the following model:

𝐻𝑂𝑈𝑅𝑆𝑛𝑖𝑠𝑔𝑦 = 𝛽1𝐹𝑈𝐿𝐿𝑔 + ∑ (𝛽2,𝑥𝐷𝑥 + 𝛽3,𝑥 × 𝐹𝑈𝐿𝐿𝑔)

2004

𝑠=1998

+ 𝜀𝑛𝑖𝑠𝑔𝑦

The data are from a highly-ranked business school’s course evaluations, aggregated at the classroom level, spanning autumn 1997 to autumn 2013. A “classroom” is an instance of a subject, taught by an instructor, in one academic quarter. g denotes MBA student type (evening or full-time), y denotes year, i denotes instructor, s denotes subject, and n indexes the instances of isgy combinations. HOURS for a given classroom is the average response to “Excluding class sessions, estimate the average number of hours per week spent in preparation or review.” FULL is a dummy denoting Full-time classroom observations. Dx is a dummy denoting classrooms held in academic year x. Confidence intervals are based on robust standard errors clustered at the instructor-subject level.

34

Figure 3: New Courses over Time by MBA Type

Notes: The data in this figure are from the course catalog of a highly-ranked business school. A new class is

defined as one that was not offered in prior years (with 1998 being the “first” year).

35

Table 1: Grade Non-disclosure (GND) in U.S. Business Schools

School GND? GMATa Ranka

Stanford Yes 728 1

Harvard (HBS) Mixedb 724 2

Yale Yes 722 11

Penn (Wharton) Yes 718 4

MIT (Sloan) No 718 3

UC Berkeley (Haas) Partialc 718 8

Dartmouth (Tuck) No 716 7

U Chicago (Booth) Yes 715 6

NYU (Stern) Yes 715 10

Northwestern (Kellogg) No 714 5

Columbia Yes 712 9

UCLA (Anderson) No 710 14

Michigan (Ross) Yes 704 15

UVA (Darden) No 699 13

Duke (Fuqua) No 697 12

Wash U St Louis (Olin) No 695 20

Carnegie Mellon (Tepper) Partialc 694 18

Minnesota (Carlson) No 694 21

U of Florida (Hough) No 694 47

UC Davis No 692 29

USC (Marshall) No 690 22

Cornell (Johnson) No 687 16

UNC (Kenan-Fiagler) No 686 19

Notre Dame (Mendoza) No 685 38

UT Austin (McCombs) No 684 17

Notes: This table describes the grade disclosure schemes at top business schools across the United states. The

table and accompanying footnotes are sourced from Gottlieb and Smetters (2011). aSource: U.S. News and World Report, 2011 bHarvard’s GND policy was officially enforced by the school until 2005, when a new dean terminated the

school’s support. Student support for GND remained high at 87% (Harvard Business School Alumni Bulletin,

2006). Harvard’s current 1-2-3 point grading system, however, effectively pools many students since most

(75%) receive grade 2. In effect, even after abandoning the official nondisclosure policy, the Harvard system is

very similar to a non-disclosure school with honors and a minimum grade requirement. cGND until second round interview

36

Table 2: Predictions for Tests based on Implementation Date

2001 compared to 2000 2002 compared to 2001

First year subjects Effort decreases Effort unchanged

Second year subjects Effort unchanged Effort unchanged

Notes: This table summarizes the predictions for treatment effects around the GND

implementation date. Grade non-disclosure is predicted to reduce effort exerted on first-year

subject coursework for the 2001-incoming MBA cohort (relative to the 2000-incoming cohort).

Table 3: Descriptive Statistics by MBA Type (2011)

Full-Time Evening Weekend

Mean Age 28 29 30

Mean Work Experience 5 years 5 years 6 years

Program Length 21 months 2.5 to 3 years 2.5 to 3 years

Notes: This table reports means across three MBA program types for age and work experience

upon entry as well as typical time to completion. These data are reported directly by the business

school under consideration.

37

Table 4: The Effect of GND on Student Effort

Variables (1) (2) (3)a (4)b (5)

FULL 0.0692 -273.1 0.0793 762.3

(0.118) (540.8) (0.140) (1,167)

GND -0.129 -0.0778

(0.13) (0.146)

FULL×GND 0.513 -0.277*** -0.398*** -0.263***

(0.507) (0.0621) (0.138) (0.0651)

FULL×GND×NONCORE -0.254***

(0.0626)

FULL×GND×CORE -0.524*

(0.274)

Fixed Effects

Ins-Sub-Year X X X

Ins-Sub-Full X X X

R-Squared 0.005 0.844 0.014 0.936 0.844

Observations 5,834 4,547 1,845 1,904 4,547

Notes: This table reports the results from our analysis on the effect of GND on self-reported student effort

using a Difference-in-Differences OLS framework. The data are from a highly-ranked business school’s

course evaluations, aggregated at the classroom level, spanning autumn 1998 to autumn 2013 unless

otherwise noted. A “classroom” is an instance of a subject, taught by an instructor, in one academic quarter.

The dependent variable, HOURS is the classroom-average response to “Excluding class sessions, estimate

the average number of hours per week spent in preparation or review.” FULL and GND are dummy

variables denoting full-time classroom observations and classrooms held during or after academic year

2001, respectively. Estimated interactions of FULL and GND with CORE and NONCORE are not reported

for brevity. A comparable sample requires instructor-subject to be offered at least once pre and post-GND

for full-time MBA students, and pre and post-GND for evening MBA students. Robust standard errors are

reported in parentheses. ***, **, and * indicate statistical significance at the 1%, 5%, and 10% levels (two-

tailed), respectively. aThis model uses the comparable sample, which requires instructor-subject to be offered at least once pre

and post-GND for full-time MBA students, and pre and post-GND for evening MBA students. bThis model uses the academic window spanning 1998-2004.

38

Table 5: Tests of Reduced Effort Investment around GND Implementation

Variables (1) (2) (3) (4)

FULL -0.195 0.191 -1.318*** -0.165

(0.230) (0.181) (0.315) (0.213)

D2001 0.648 -0.073

(0.426) (0.173)

FULL×D2001 -1.024*** -0.253

(0.339) (0.221)

D2002 -0.935** 0.315

(0.320) (0.227)

FULL×D2002 0.437 0.115

(0.409) (0.274)

Subject Level First Year Second Year First Year Second Year

Comparison 2000 vs. 2001 2000 vs. 2001 2001 vs. 2002 2001 vs. 2002

R-Squared 0.912 0.919 0.898 0.904

Observations 58 129 56 132

Notes: This table reports the results from our analysis on the incoming class of 2001’s effect on self-

reported effort. The data are from a highly-ranked business school’s course evaluations, aggregated at the

classroom level, spanning autumn 1998 to autumn 2013. A “classroom” is an instance of a subject, taught

by an instructor, in one academic quarter. The dependent variable, HOURS is the classroom average

response to “Excluding class sessions, estimate the average number of hours per week spent in preparation

or review.” FULL is a dummy denoting Full-time classrooms. Dx is a dummy denoting classrooms held in

academic year x. All estimations include Instructor-Subject fixed effects. Robust standard errors are

reported in parentheses. ***, **, and * indicate statistical significance at the 1%, 5%, and 10% levels (two-

tailed), respectively.

39

Table 6: Tests of Reduced Sensitivity to Subject Difficulty

Variables (1) (2)

GND -22.15 18.81

(16.90) (24.45)

DIFFICULTY -14.59*** -8.057*

(2.80) (4.715)

GND×DIFFICULTY 4.329*** -2.037

(1.873) (4.759)

Sample Full-time Evening

R-Squared 0.137 0.113

Observations 1,864 1,632

Notes: This table reports the results from our analysis on the effect of GND on the popularity of difficult

classes using a Difference-in-Differences OLS framework. The data are from a highly-ranked business

school’s course evaluations, aggregated at the classroom level, spanning autumn 1998 to autumn 2013. A

“classroom” is an instance of a subject, taught by an instructor, in one academic quarter. The dependent

variable, POPULARITY is the the proportion (in basis points) of student type g's non-core enrollments

comprising of subject s enrollments in quarter q of year y. GND is a dummy variable indicating classrooms

held during or after academic year 2001, DIFFICULTY is the classroom average response to “Excluding

class sessions, estimate the average number of hours per week spent in preparation or review.” All

estimations include Subject Area fixed effects. Robust standard errors are reported in parentheses. ***, **,

and * indicate statistical significance at the 1%, 5%, and 10% levels (two-tailed), respectively.

40

Table 7: The Effect of GND on GPA-Based Student-Employer Matching

Variables (RANK)

D1999×GPA -9.681

(-1.489)

D2000×GPA -8.749***

(-4.810)

D2001×GPA 0.217

(0.0694)

D2002×GPA -5.244

(-1.435)

D2003×GPA 5.200*

(1.917)

D2004×GPA -5.121**

(-2.155)

D2005×GPA -2.254

(-0.468)

D2006×GPA 4.628***

(2.832)

R-Squared 0.312

Observations 2,656

Notes: The data come from a highly-ranked business school’s data archives. Each observation corresponds

to a student-employer match. When a student matches with the same employer twice, i.e., once for an

internship and once for a graduate position, we select the internship match. RANK is the employer rank that

comes from Universum, who survey MBA students about the desirability of MBA employers. Universum

data spans 2000 to 2014. Thus, to increase the length of our pre-period, we extrapolate the year 2000

rankings to prior years. GPA is student GPA measured as at the end of the first fall quarter for internship

recruiting, and as at the end of the first summer quarter for graduate position recruiting. GMAT score is the

maximum GMAT score obtained by a student and listed on their MBA program application. Dx is a dummy

denoting students from cohorts commencing their studies in academic year x. All estimations include

Industry-Cohort fixed effects. Robust t-statistics are reported in parentheses. ***, **, and * indicate

statistical significance at the 1%, 5%, and 10% levels (two-tailed), respectively.

41

Table 8: Correlation between Student GMAT Scores and Employer Rank

1999 2000 2001a 2002 2003 2004 2005 2006

Correlation -0.11 -0.01 -0.08 -0.01 -0.12 -0.06 -0.04 0.00

t -1.81 -0.10 -1.23 -0.08 -1.85 -1.08 -0.88 -0.01

Observations 269 274 265 206 255 340 401 376

Notes: The data come from the data archives of a highly-ranked business school. Each observation

corresponds to a student-employer match. When a student matches with the same employer twice, i.e., once

for an internship and once for a graduate position, we select the internship match. GPAs are measured as at

the end of the first fall quarter for internship recruiting, and as at the end of the first summer quarter for

graduate position recruiting. Employer ranks come from Universum, who survey MBA students about the

desirability of MBA employers. Universum data spans 2000 to 2014. Thus, to increase the length of our

pre-period, we extrapolate the year 2000 rankings to prior years. aFirst year that GND was implemented