Empirical Methods in Applied Economics

Jörn-Ste¤en PischkeLSE

October 2005

1 Instrumental Variables

1.1 Basics

A good baseline for thinking about the estimation of causal e¤ects is oftenthe randomized experiment, like a clinical trial. However, for many treat-ments, even in an experiment it may not be possible to assign the treatmenton a random basis. For example, in a drug trial it may only be possible too¤er a drug but not to enforce that the patient actually takes the drug (aproblem called non-compliance). When we study job training, we may onlybe able to randomly assign the o¤er of training. Individuals will then decidewhether to participate in training or not. Even those not assigned trainingin the program under study may obtain the training elsewhere (a problemcalled non-embargo, training cannot be e¤ectively witheld from the controlgroup). Hence, treatment itself is still a behavioral variable, and only theintention to treat has been randomly assigned. The instrumental variables(IV) estimator is a useful tool to evaluate treatment in such a setup.

In order to see how this works, we will start by reviewing the assump-tions necessary for the IV estimator to be valid in this case. We need someadditional notation for this. Let z = f0; 1g the intention to treat variable.D = f0; 1g is the treatment again. We can now talk about counterfactualtreatments: D(z) is the treatment for the (counterfactual) value of z. I.e.D(0) is the treatment if there was no intention to treat, and D(1) is thetreatment if there was an intention to treat. In the job training example,D(0) denotes the training decision of an individual not assigned to the train-ing program, and D(1) denotes the training decision of someone assigned tothe program. As before, Y is the outcome. The counterfactual outcome isnow Y (z;D) because it may depend on both the treatment choice and thetreatment assignment. So there are four counterfactuals for Y .

1

We can now state three assumptions:

Assumption 1 z is as good as randomly assigned

Assumption 2 Y (z;D) = Y (z0; D) 8z; z0; DThis assumption says that the counterfactual outcome only dependson D, and once you know D you do not need to know z. This is theexclusion restriction, and it implies that we can write Y (z;D) = Y (D).

There are three causal e¤ects we can de�ne now:

1. The causal e¤ect of z on D is D(1)�D(0).

2. The causal e¤ect of z on Y is Y (1; D(1))�Y (0; D(0)). Given Assump-tion 2, we can write this as Y (D(1))� Y (D(0)).

3. The causal e¤ect of D on Y is Y (1)� Y (0).

E¤ects 1. and 2. are reduced form e¤ects. 1. is the �rst stage relation-ship, and 2. is the reduced form for the outcome. 3. is the treatment e¤ectof ultimate interest. Without Assumption 2, it is not clear how to de�nethis e¤ect, beause the causal e¤ect of D on Y would depend on z.

Assumption 3 E(D(1)�D(0)) 6= 0.This assumption says that the variable z has some power to in�uencethe treatment. Without it, z would be of no use to help us learnsomething about D. It is the existence of a signi�cant �rst stage.

Assumption 1 is su¢ cient to estimate the reduced form causal e¤ects1. and 2. The exclusion restriction and the existence of a �rst stage areonly necessary in order to give these reduced form e¤ects an instrumentalvariables interpretation. Because of this, it is often useful to see estimatesof the reduced forms as well as the IV results in an application.

In order to get some further insights into the workings of the IV esti-mator, start with the case where the instrument is binary and there are noother covariates. In this case, the IV estimator takes a particularly simpleform. Notice that the IV estimator is given by

b�IV = cov(yi; zi)

cov(Di; zi):

2

Since zi is a dummy variable, the �rst covariance can be written as

cov(yi; zi) = E(yi � y)(zi � z)= Eyizi � yz= E(yijzi = 1)E(zi = 1)� yE(zi = 1)= fE(yijzi = 1)� ygE(zi = 1)= fE(yijzi = 1)� [E(yijzi = 1)E(zi = 1) + E(yijzi = 0)E(zi = 0)]gE(zi = 1)= fE(yijzi = 1)E(zi = 0)� E(yijzi = 0)E(zi = 0)gE(zi = 1)= fE(yijzi = 1)� E(yijzi = 0)gE(zi = 1)E(zi = 0):

A similar derivation for the denominator leads to

b�IV =cov(yi; Zi)

cov(Di; Zi)

=fE(yijzi = 1)� E(yijzi = 0)gE(zi = 1)E(zi = 0)fE(Dijzi = 1)� E(Dijzi = 0)gE(zi = 1)E(zi = 0)

=E(yijzi = 1)� E(yijzi = 0)E(Dijzi = 1)� E(Dijzi = 0)

=E(yijzi = 1)� E(yijzi = 0)

P (Di = 1jzi = 1)� P (Di = 1jzi = 0):

This formulation of the IV estimator is often referred to as the Wald-estimator. It says that the estimate is given by the di¤erence in outcomesfor the groups intended and not intended for treatment divided by the dif-ference in actual treatment for these groups. It is also easy to see that thenumerator is the reduced form estimate, also frequently called the intentionto treat estimate. The denominator is the �rst stage estimate. Hence, theWald-estimator is also the indirect least squares estimator, dividing the re-duced form estimate by the �rst-stage estimate. This has to be true in thejust identi�ced case.

The IV methodology is often useful in actual randomized experimentswhen the treatment itself cannot be randomly assigned because of the non-compliance and the lack of embargo problems. For example, in the Mov-ing to Opportunity exeriment (Kling et al., 2004), poor households weregiven housing vouchers to move out of high poverty neighborhood. Whilethe voucher receipt was randomly assigned, whether the household actuallyended up moving is not under the control of the experimenter: some house-holds assigned a voucher do not move, but some not assigned a vouchermove on their own. Kling et al. (2004) therefore report both estimates of

3

the reduced form e¤ect (intention to treat estimates or ITT) and IV esti-mates (treatment on the treated or TOT). Since about half the householdswith vouchers actually moved, TOT estimates are about twice the size ofthe ITT estimates. Notice that the ITT estimates are of independent in-terest. They estimate directly the actual e¤ect of the policy. Nevertheless,the TOT estimates are often of more interest in terms of their economicinterpretation.

If an instrument is truely as good as randomly assigned, then IV esti-mation of the binary model

yi = �+ �Di + "i

will be su¢ cient. Often, this assumption is not going to be satis�ed. How-ever, an instrument may be as good as randomly assigned conditional onsome covariates, so that we can estimate instead

yi = �+ �Di +Xi� + "i;

instrumenting Di by zi. The role of the covariates here is to ensure thevalidity of the IV assumptions. Of course, covariates orthogonal to zi mayalso be included simply to reduce the variance of the estimate.

An interesting and controversial example of an IV study is the paper byAngrist and Krueger (1991). They try to estimate the returns to education.The concern is that there may be an omitted variable (�ability�) whichconfounds the OLS estimates. The Angrist and Krueger insight is that UScompulsory schooling laws can be used to construct an instrument for thenumber of years of schooling. US laws specify an age an individual has toreach before being able to drop out of school. This feature, together withthe fact that there is only one date of school entry per year means thatseason of birth a¤ects the length of schooling for dropouts.

Suppose, for example, that there are two individuals, Bob and Ron. Bobis born in January, and Bob is born in December of the previous year. Sothey are almost equal in age. School entry rules typically specify that youare allowed to enter school in summer, if you turned 6 during the previouscalendar year. This means that Ron, who turns 6 in December, is allowed toenter school at age 6. Bob will not satisfy this rule and therefore has to waitan additional year and enter when he is 7. At the time of school entry, Bobis 11 months older than Ron. Both can drop out when they reach age 16.Of course, at that age, Bob will have completed 11 months less schoolingthan Ron, who entered earlier. The situation is illustrated in the following�gure.

4

Bob

RonDec

Jan

turn 6

turn 6

enter at 6

enter at 7 age 16

age 16

Figure 1: Schooling for Bob and Ron

5

The idea of the Angrist and Krueger (1991) paper is to use season ofbirth as an instrument for schooling. So in this application, we have

y = log earnings

D = years of schooling

z = born in the 1st quarter.

The �rst thing to do is to check the three IV assumptions. Assumption1 (random assignment) is probably close to satis�ed. Births are almostuniformly spaced over the year. There is relatively little parental choiceover season of birth although there is clearly some. There is some evidenceof small di¤erences in the socioeconomic status of parents by season of birthof the child. Assumption 2, the exclusion restriction says that season of birthdoes not a¤ect earnings directly, only through its e¤ect on schooling. If youare born early in the year you enter school later (like Bob in the example).Hence, age at school entry should not be correlated with earnings. There issome psychology evidence that those who start later do better in school. Ifthis translates into unobserved factors (i.e. anything other than how longyou stay in school) which are correlated with earnings, then this will leadto a downward bias in the estimates. Assumption 3, the existence of a �rststage is an empirical matter, which we can check in the data.

Figures 1 and 2 in Angrist and Krueger plot years of completed educationagainst cohort of birth (in quarters) for those born between 1930 and 1950.The �gures reveal that average education tends to be higher for those bornlater in the year (quarters 3 and 4) and relatively low for those born inthe �rst quarter, as we would expect. The pattern is more pronouncedfor the early cohorts, and starts to vanish for the cohorts born in the late1940s, when average education is higher. This is consisent with the ideathat compulsory schooling laws are at the root of this pattern, since fewerand fewer students drop out at the earliest possible date over time.

Table 1 presents numerical estimates of this relationship. It reveals thatthose born in the �rst quarter have about 0.1 years less schooling than thoseborn in the fourth quarter, with a slightly weaker relationship for the cohortsborn in the 1940s. There is a small e¤ect on high school graduation ratesbut basically no e¤ect on college graduation or post-graduate education.1

This pattern suggests that schooling is a¤ected basically only for those withvery little schooling. Table 2 compares the quarter of birth e¤ects on school

1They also check for the e¤ect on years of education for those with at least a highschool degree. This is not really valid, because the conditioning is on an outcome variable(graduating from high school), which they showed to be a¤ected by the instrument.

6

enrollment at age 16 for those in states with a compulsory schooling age of 16with states with a higher compolsory schooling age. The enrollment e¤ect isclearly visible for the age 16 states, but not for the states with later dropoutages. The enrollment e¤ect is also concentrated on earlier cohorts when age16 enrollment rates were lower. This is suggestive that compulsory schoolinglaws are indeed at the root of the quarter of birth-schooling relationship.

Figure 5 in the paper shows a similar picture to �gures 1 and 2, butfor earnings. There is again a saw-tooth pattern in quarter of birth, withearnings being higher for those born later in the year. This is consistent withthe pattern in schooling and a postive return to schooling. One problemapparent in �gure 5 is that age is clearly related to earnings beyond thequarter of birth e¤ect, particularly for those in later cohorts (i.e. for thosewho are younger at the time of the survey in 1980). It is therefore importantto control for this age earnings pro�le in the estimation, or to restrict theestimates to the cohorts on the relatively �at portion of the age-earningspro�le in order to avoid confounding the quarter of birth e¤ect with earningsgrowth with age. Hence, the exclusion restriction only holds conditional onother covariates in this case, namely adequate controls for the age-earningspro�le.

Table 3 presents simple Wald estimates of the returns to schooling es-timate for the cohorts in the relatively �at part of their life-cycle pro�le.Those born in the 1st quarter have about 0.1 fewer years of schooling com-pared to those born in the remaining quarters of the year. They also haveabout 1 percent lower earnings. This translates into a Wald estimate ofabout 0.07 for the 1920s cohort, and 0.10 for the 1930s cohort, compared toOLS estimates of around 0.07 to 0.08.

Tables 4 to 6 present instrumental variables estimates using a full set ofquarter of birth times year of birth interactions as instruments. These spec-i�cations also control for additional covariates. It can be seen by comparingcolumns (2) and (4) in the tables that controlling for age is quite impor-tant. The IV returns to schooling are either similar to the OLS estimates orabove. Finally, table 7 exploits the fact that di¤erent states have di¤erentcompulsory schooling laws and uses 50 state of birth times quarter of birthinteractions as instruments in addition to the year of birth times quarter ofbirth interactions, also controlling for state of birth now. The resulting IVestimates tend to be a bit higher than those in table 5, and the estimatesare much more precise.

7

1.2 Weak Instruments

Starting in the late 1980s, applied researchers became concerned with theperformance of instrumental variables estimators when the instruments areonly poorly correlated with the endogenous regressor. It had long beenknown in the econometrics literature that the two stage least squares es-timator is not unbiased in small samples, but the implications of this hadnot generally been considered by applied researchers. Bound, Jaeger, andBaker (1995) made the case that weak instruments might produce spuriousconclusions in the Angrist and Krueger (1991) study. They make two pointsabout weak instruments: one on the potential problem of inconsistency, andone on small sample bias. Both problems are related to the predictive powerof the instruments, but they are conceptually very distinct.

1.2.1 Inconsistency

Consider the simultaneous equations model

yi = �xi + "i

xi = �zi + vi:

The OLS and IV estimators are given by

b�OLS =cov(yi; xi)

var(xi)b�IV =cov(yi; zi)

cov(xi; zi)

and the plim�s of the estimators are

plimb�OLS = � +�x"�2x

plimb�IV = � +�bx"�2bx:

This yieldsplimb�IV � �plimb�OLS � � = �bx"=�x"

�2bx=�2x=�bx"=�x"R2xz

:

This expression shows that the inconsistency of the IV estimator relative tothe OLS estimator is related to the relative �endogeneity�of z and x. Theinstrument z may be �almost�as good as randomly assigned but not quite.Hence, �bx" may be small but not quite zero. However, even if �bx" is small

8

compared to �x", the relative inconsistency of the IV estimator may still beimportant as long as R2xz is also small, i.e. as long as the correlation of zand x is low.

Notice that R2xz, is the R2 from the �rst stage regression. In the Angrist

and Krueger (1991) application, the �rst stage R2s are only in the order of0.0001 to 0.0002. So even very small deviations from random assignmentin z, and hence very small values of �bx" may imply that the inconsistencyin the IV estimator is worse than that of the OLS estimator. This is thesame type of concern we discussed above in the context of �xed e¤ectsestimators. Instrumental variables, like �xed e¤ects, removes some, andpotentially much of the variation in x. It may well remove �good�as well as�bad�variation. If, after instrumenting, we are left with much less variationwe are not any better o¤ than in the OLS case if we have removed as much�good�as �bad�variation.

Bound, Jaeger, and Baker (1995) point out that the individuals bornearly in the year are (very slightly) more likely to be mentally retarded andhave lower IQ scores. These di¤erences are very small, but we had seen intable 3 that those born in the �rst quarter of the year earn only 1% lessthan those born later. Even if there is only a very small group of individualsamong the �rst quarter births whose earnings are substantially lower, thismay impart a substantial bias on the estimates. Hence, this is clearly a realconcern. It is very di¢ cult to know how to deal with this problem, except incases where there is actual random assignment. Good �natural experimentsmay rely on tiny amounts of variation in the endogenous regressor, andhence be very succeptible to this problem.

1.2.2 Small Sample Bias

The second point made by Bound, Jaeger, and Baker (1995) is about thesmall sample bias of two stage least squares etimators. There are two sourcesof the problem, and it is straightforward to see the intuition for each:

1. Suppose the instrument is completely uncorrelated with the endoge-nous regressor. With an in�nite amount of data, cov(xi; zi) is going tobe exactly zero, and the IV estimator cannot be computed anymore.In a small sample, cov(xi; zi) is not going to be literally zero, justsmall. In any particular sample, z and x will have some slight randomcorrelation. This means bx picks up some variation that is just like theoriginal variation in x, so it is not in any way purged of the endogenousvariation in x. As a consequence, the instrumental variables estimator

9

will be biased towards OLS.

2. Suppose you start with a valid instrument (i.e. a z which is as goodas randomly assigned and which obeys the exclusion restriction). Nowyou add more and more instruments to the (overidenti�ed) IV model.As the number of instruments goes to n, the sample size, the �rst stageR2 becomes 1, and hence b�IV ! b�OLS . In a small sample, even a validinstrument will pick up some small amounts of endogenous variationin x (just as for the random instrument in 1.). Adding more and moreinstruments, the amount of random, and hence endogenous, variationin x will become more and more important. So IV will be more andmore biased towards OLS.

Return to the simultaneous equation model, but with multiple instru-ments as in

y = �x+ "

x = �0Z + v:

The bias in the IV estimator can be approximated by

E�b�IV � ' � +

�"v�0Z 0Z�

(p� 2)

= � +�"v�2v

�2v�0Z 0Z�

(p� 2)

where p is the number of instruments. Notice

1. The second term is related to the F-statistic for the �rst stage regres-sion, which is

F =�0Z 0Z�=p

�2vwhen there is no constant. The lower the F-statistic, the worse thebias will be. If there are no other covariates, the F-statistic is, ofcourse, directly related to the �rst stage R2, which we have seen to beimportant above. If there are other covariates, it is the F-statistic onthe excluded instruments which matters.

2. The �rst term �"v=�2v is similar to the bias in the OLS estimator,

which is �x"=�2x. If the instruments were truely uncorrelated with x,i.e. � = 0, then �2x = �

2v and �x" = �"v. However, in that case, the IV

estimator has no moments, so the mean bias does not exist. However,it can be shown that the median bias of the IV estimator is that ofthe OLS estimator again.

10

Bound, Jaeger, and Baker (1995) show that these small sample issuesare a real concern in the Angrist and Krueger case, despite the fact that theregressions are being run with 300,000 or more observations. �Small sample�is always a relative concept. Bound et al. show that the IV estimates in theAngrist and Krueger speci�cation move closer to the OLS estimates as morecontrol variables are included, and hence as the �rst stage F-statistic shrinks(tables 1 and 2). They then go on and completely make up quarters of birth,using a random number generator. Their table 3 shows that the results fromthis exercise are not very di¤erent from their IV estimates reported before.Maybe the most worrying fact is that the standard errors from the randominstruments are not much higher than those in the real IV regressions.

In order to illuminate these problems further, I produced some MonteCarlo results using the following design:

y = 1 + �x+ " � = 1:

The instrument vector used includes one valid instrument with various cor-relations with x. In addition, it includes k random (�garbage�) instrumentswith no correlation with x. " and v are correlated so that the OLS estimateis about 1.7. The sample size is 100.

The results are produced at the end of the notes. Each panel is fora di¤erent number of k. These columns in each panel refer to di¤erentestimators, the OLS estimator, and �ve IV or 2SLS estimators, where the�rst instrument has a correlation with x of 0, 0.025, 0.05. 0.1, and 0.2.The rows in each panel display the mean estimate of b� across the MonteCarlo experiments, �ve percentiles of the distribution of b�, the median ofthe standard error estimates for each experiments, the standard deviationof the b� estimates across experiments (this is the true sampling error, whichthe standard error should estimate), and a standard error on the estimateof the mean of b�.

In the �rst panel, there is only one instrument, so the models are justidenti�ed. If there is only one random instrument, the IV estimator has thesame median bias as the OLS estimator. You can also see that the absenceof a (theoretical) mean for this estimator results in the mean of b� movingaround a lot from column to column. As soon as the single instrumenthas a slight correlation with x, the IV estimator becomes median unbiased.However, the estimated standard errors do not re�ect the true samplingvariation very well until the correlation of the instrument with x becomessu¢ ciently high. The true sampling distribution of the IV estimator isextremely spread out for the random instrument case, as well as for weakly

11

correlated instruments. Only once the correlation of the instrument withthe endogenous regressor reaches a value of about 0.2 does the standarderror re�ect the sampling variation roughly correctly in this example.

As soon as a second random instrument is added, things change dra-matically. The sampling distribution of the (seemingly overidenti�ed) 2SLSestimator becomes much tighter, and the estimated standard errors are muchmore accurate, although they are still too small. Even when our �rst in-strument has some, albeit low, correlation with x, the 2SLS estimator isnow biased towards OLS. Only once the valid instrument has a correlationof 0.1 or 0.2 with x does the bias disappear. Adding additional randominstruments ampli�es these features. With as few as four random instru-ments in addition to the �rst instrument, the estimated standard errors arenow almost all correct, but the true sampling distribution is quite tight (thedispersion is not terribly di¤erent from the sampling distribution of the OLSestimator). Even if the valid instrument has a correlation of 0.1 or 0.2 withx, the 2SLS estimator is still clearly biased now. With 32 random instru-ments things are even worse, and the valid instrument seems to make littledi¤erence now. Most disturbing is maybe that the sampling distribution isextremely tight: even with only random instruments, the dispersion is notmore than twice that of the OLS estimator. Hence, these results can easilytrick the researcher into believing that the OLS result is correct, while thevalue of 1 for � could easily be rejected.

This exercise illustrates a number of points. If there is no valid instru-ment, any estimator has the same median bias as OLS. In the just identi�edcase, there is no median bias as soon as the instrument has some correlationwith x. Standard errors for the just identi�ed IV estimator get larger as theinstrument gets weaker but they do not re�ect the true sampling variationwell. Things are much worse in the overidenti�ed 2SLS case. As soon asthere are some random instruments present in the instrument set, there willbe a median bias towards OLS. Standard errors tend to be rather small,and hence do not send any warning signs: these could equally well be resultsfrom a well identi�ed model.

There are a variety of estimators, which have more desirable propertiesin small samples. The small sample problem derives from the fact that wedo not know the �rst stage �. Since Z comes from the same sample fromwhich b� is computed, there will tend to be a spurious correlation betweenZ and b�. This problem can be solved, by using a leave-out or jackknifeestimator of the �rst stage

bxJIV E;i = Zie�(i)12

where e�(i) is calculated from the sample with the i-th observation omitted.It turns out that

Zie�(i) = Zi(Z 0Z)�1

1� Z 0i (Z 0Z)�1 Zi

(Z 0x� Z 0xi)

=Zib� � hixi1� hi

where hi = Z 0i (Z0Z)�1 Zi, a quantity called leverage, which many regression

packages compute. Hence, the jackknifed IV estimator, or JIVE (Angrist,Imbens, and Krueger, 1999) can be easily computed with two passes throughthe data:

1. compute b�2. subtract out the in�uence of the i-th observation.

Then b�JIV E = � bX 0JIV EX

��1 bX 0JIV Ey:

Similarly, the LIML estimator, which is implemented in many regressionpackages, also has better �nite sample properties (e.g. it is median unbiasedeven in the overidenti�ed case with weak instruments). Moreover, standarderrors for both the JIVE and LIML estimators become large when the in-struments are truely random (see �gures 1 to 4 in the working paper versionof Angrist, Imbens, and Krueger, NBER TWP 172, 1995). Angrist, Imbens,and Krueger (1999) show that these estimators give very similar results tothe 2SLS estimator in Angrist and Krueger (1991) for the quarter of birthinstruments. This indicates that small sample bias may not be a problemin these estimates after all.

Hence, in terms of prescriptions for the applied researcher, the literatureon the small sample bias of IV yields the following results:

1. Report the �rst stage of your model. The �rst check is whether theinstruments predict the endogenous regressor in the hypothesized way(sometimes the �rst stage results are of independent interest). Reportthe F-statistic on the excluded instruments. Stock, Wright, and Yogo(2002), in an excellent survey of these issues, suggest that F-statisticsabove 10 or 20 are necessary to rule out weak instruments. The p-valueon the F-statistic is rather meaningless in this context.

13

2. In the just identi�ed case, the IV, JIVE and LIML estimators arethe same. Carefully watch the standard errors. IV standard errorswill always be higher than OLS standard errors but if they go uptoo much, this may indicate weak instruments. If you can constructslightly di¤erent versions of your instrument, this may be a usefulspeci�cation check. If the results vary a lot across spec�cations, thismay indicate weak instruments.

3. In the overidenti�ed case, check the standard 2SLS results with analternative estimator like JIVE or LIML. If the results are di¤erent,and if JIVE or LIML standard errors are higher than those of 2SLS,this may indicate weak instruments. Experiment with di¤erent subsetsof the instruments. Are the results stable?

In some applications there is more than one endogenous variable, andhence a set of instruments has to predict these multiple endogenous vari-ables. The weak instruments problem can no longer be assessed simply bylooking at the F-statistic for each �rst stage equation alone. For example,consider the case of two endogenous variables and two instruments. Sup-pose instrument 1 is strong and predicts both endogenous variables well.This will yield high F-statistics in each of the two �rst stage equations.Nevertheless, the model is underidenti�ed because bx1 and bx2 will be closelycorrelated now. With two instruments it is necessary for one to predict the�rst endogenous variable, and the second the second. In order to assesswhether the instruments are weak or strong, it is necessary to look at amatrix version of the F-statistic, which assesses all the �rst stage equationsat once. This is called the Cragg-Donald statistic. References can be foundin Stock, Wright, and Yogo (2002).

14

Angrist and Krueger 1991: Figures 1 and 2

Angrist and Krueger 1991: Table 1

Angrist and Krueger 1991: Table 2

Angrist and Krueger 1991: Figure 5

Angrist and Krueger 1991: Table 3

Angrist and Krueger 1991: Table 4

Angrist and Krueger 1991: Table 5

Angrist and Krueger 1991: Table 6

Angrist and Krueger 1991: Table 7

Bound et al. 1995: Table 1

Bound et al. 1995: Table 2

Bound et al. 1995: Table 3

Monte Carlo Design y = 1 + x*b + e b = 1 Instrument vector z includes one instrument with various correlations with x and k garbage instruments (with no correlation with x). All experiments use samples with 100 observations and no other regressors. 10000 replications 0 garbage instruments ols 2sls 2sls 2sls 2sls 2sls corr of z and x 0.000 0.025 0.050 0.100 0.200 mean of b 1.707 6.803 -0.822 0.631 0.919 0.957 10th perc. of b 1.549 -1.938 -0.777 -0.281 0.227 0.509 25th perc. of b 1.623 0.497 0.405 0.512 0.658 0.763 median of b 1.708 1.735 1.117 1.020 0.999 0.997 75th perc. of b 1.791 2.959 1.634 1.400 1.286 1.202 90th perc. of b 1.866 5.597 2.288 1.722 1.516 1.363 median of std err 0.123 2.630 0.935 0.652 0.451 0.320 std dev of b 0.124 371.588 152.595 7.684 2.734 0.366 se of mean of b 0.001 3.716 1.526 0.077 0.027 0.004 10000 replications 1 garbage instruments ols 2sls 2sls 2sls 2sls 2sls corr of z and x 0.000 0.025 0.050 0.100 0.200 mean of b 1.707 1.719 1.184 1.057 1.010 0.999 10th perc. of b 1.550 0.101 0.089 0.217 0.402 0.565 25th perc. of b 1.625 1.024 0.758 0.729 0.763 0.810 median of b 1.707 1.722 1.266 1.141 1.069 1.037 75th perc. of b 1.790 2.411 1.710 1.490 1.332 1.237 90th perc. of b 1.865 3.315 2.185 1.811 1.549 1.393 median of std err 0.123 1.232 0.746 0.574 0.424 0.307 std dev of b 0.123 2.938 1.631 1.104 0.685 0.354 se of mean of b 0.001 0.029 0.016 0.011 0.007 0.004 10000 replications 2 garbage instruments ols 2sls 2sls 2sls 2sls 2sls corr of z and x 0.000 0.025 0.050 0.100 0.200 mean of b 1.707 1.730 1.334 1.197 1.081 1.038 10th perc. of b 1.553 0.561 0.434 0.483 0.528 0.625 25th perc. of b 1.626 1.169 0.936 0.879 0.842 0.856 median of b 1.707 1.712 1.368 1.242 1.129 1.066 75th perc. of b 1.788 2.253 1.757 1.553 1.378 1.258 90th perc. of b 1.864 2.889 2.175 1.868 1.591 1.417 median of std err 0.123 0.895 0.644 0.517 0.398 0.299 std dev of b 0.123 1.475 0.885 0.693 0.485 0.323 se of mean of b 0.001 0.015 0.009 0.007 0.005 0.003

5000 replications 4 garbage instruments ols 2sls 2sls 2sls 2sls 2sls corr of z and x 0.000 0.025 0.050 0.100 0.200 mean of b 1.707 1.700 1.467 1.336 1.201 1.103 10th perc. of b 1.549 0.874 0.782 0.743 0.712 0.723 25th perc. of b 1.624 1.317 1.144 1.049 0.980 0.928 median of b 1.707 1.709 1.470 1.352 1.228 1.128 75th perc. of b 1.790 2.089 1.788 1.638 1.452 1.308 90th perc. of b 1.864 2.510 2.133 1.897 1.652 1.454 median of std err 0.123 0.628 0.511 0.445 0.362 0.283 std dev of b 0.123 0.718 0.589 0.487 0.393 0.299 se of mean of b 0.002 0.010 0.008 0.007 0.006 0.004 5000 replications 8 garbage instruments ols 2sls 2sls 2sls 2sls 2sls corr of z and x 0.000 0.025 0.050 0.100 0.200 mean of b 1.706 1.700 1.561 1.458 1.327 1.209 10th perc. of b 1.543 1.116 1.042 0.978 0.908 0.863 25th perc. of b 1.621 1.407 1.291 1.212 1.118 1.040 median of b 1.707 1.699 1.568 1.454 1.346 1.223 75th perc. of b 1.790 1.993 1.822 1.710 1.552 1.391 90th perc. of b 1.868 2.270 2.069 1.943 1.731 1.534 median of std err 0.123 0.444 0.395 0.362 0.311 0.258 std dev of b 0.128 0.472 0.422 0.395 0.336 0.264 se of mean of b 0.002 0.007 0.006 0.006 0.005 0.004 2500 replications 16 garbage instruments ols 2sls 2sls 2sls 2sls 2sls corr of z and x 0.000 0.025 0.050 0.100 0.200 mean of b 1.705 1.705 1.630 1.556 1.462 1.347 10th perc. of b 1.549 1.314 1.255 1.190 1.129 1.057 25th perc. of b 1.623 1.491 1.426 1.362 1.299 1.199 median of b 1.704 1.715 1.633 1.554 1.469 1.353 75th perc. of b 1.785 1.910 1.838 1.749 1.633 1.502 90th perc. of b 1.860 2.112 2.016 1.926 1.792 1.622 median of std err 0.123 0.310 0.292 0.278 0.253 0.224 std dev of b 0.124 0.317 0.306 0.292 0.264 0.223 se of mean of b 0.002 0.006 0.006 0.006 0.005 0.004 2500 replications 32 garbage instruments ols 2sls 2sls 2sls 2sls 2sls corr of z and x 0.000 0.025 0.050 0.100 0.200 mean of b 1.701 1.703 1.665 1.630 1.583 1.496 10th perc. of b 1.539 1.422 1.387 1.365 1.316 1.245 25th perc. of b 1.618 1.555 1.518 1.483 1.444 1.370 median of b 1.702 1.705 1.668 1.626 1.585 1.501 75th perc. of b 1.787 1.855 1.808 1.770 1.718 1.629 90th perc. of b 1.859 1.978 1.938 1.900 1.845 1.735 median of std err 0.123 0.217 0.211 0.207 0.198 0.184 std dev of b 0.126 0.220 0.216 0.213 0.208 0.193 se of mean of b 0.003 0.004 0.004 0.004 0.004 0.004

Angrist, Imbens, and Krueger (NBER TWP 172): Homoskedastic model, one good, one garbage instrument

Add 18 more garbage instruments

Heteroskedastic, 1 good, 19 garbage instruments

Only garbage instruments

Angrist et al. 1999: Table 2