RD Lecture

8/10/2019 RD Lecture

1/60

EITM2011

Chris Berr


2/60

.

II. DiscussionofExemplaryPapers

III. Practitioner sGui e

IV. Stataexamples


3/60


4/60

Im r n R f r n RDmethodswereintroducedbyThistlewaite andCampbell

. .

Recentapplicationsinpoliticsincludeanalysesoftheincumbencyeffect(Lee,2008), electoralcompetitionon

, , ,onlegislatorbehavior(Rehavi nd),thevalueofaseatinthe

legisalture (EggersandHainmueller 2009),theeffectof

Recentimportanttheoreticalworkhasdealtwithidentificationissues(Hahn,Todd,andVanDer Klaauw,2001),

design(McCrary,2008),bandwidthselection(Imbens andKalyanaraman 2011).

,Klaauw(2008),andImbensandLemieux(2008). Todaysdiscussionborrowsfromallofthem


5/60

Re ressionDiscontinuit Basics ThebasicideabehindtheRDdesignisthatassignment

tothetreatmentisdetermined,eithercom letel or

partly,bythevalueofanassignment(orforcing)variable(thecovariateX)beingoneithersideofafixedthreshold. Assignmenttotreatmentbycovariatevalue,assignallunits

withXictotreatment

ofthetreatmentatX= c

RDislikearandomizedexperimentatthecutpointX= c

TheRDdesignisgenerallyregardedashavingthegreatestinternalvalidityofallquasiexperimentalmethods.Itsexternalvalidityismorelimited,sincetheestimatedtreatmenteffectislocaltothediscontinuity.


6/60


7/60

RDScatterplot:NoTreatmentEffect

(Y)

Assignment Variable (T)Cutting Point


8/60


9/60

Underfairlyweakassumptions,theeffectofthe

cXYEcXYE iiii |lim|lim00

Whichis:RD=E[Yi(1) Yi(0)|Xi=c].

cc iiii mm00

Thisistheaveragetreatmenteffectatthecutoffpoint,aparticularLATE.

andE[Y(0)|X]are(assumedtobe)continuousinx.

SomeextrapolationisrequiredbecausebydesigntherearenounitswithXi=cforwhomweobserveYi(0).


10/60

RDComparedtoOtherObservationalMethods

Remem ert etwoassumptionsnee e toi enti ytreatmente ectsfromobservationaldatausingregression/matching(Kosukes slide#31):overlapandunconfoundedness. .

Ingeneral,unconfoundednessisnotconsideredaparticularlycredibleassumption,andtheothermethodswerestudyingthisweekcanbethoughtofaswaysformakingitmoreplausible.

RDisspecialinthefollowingways: Unconfoundedness issatisfiedbydefinition.WhenXc,Tisalways1;when

X


11/60

RDComparedtoanExperiment RDisoftendescribedasaclosecousinofarandomizedexperimentorasa

localrandomizedexperiment.

Coughey andSekhon argueagainstthisconceptualization,forreasonswewillsee a er, u swor un ers an ngw y eana ogy sma e

Consideranexperimentinwhicheachparticipantisassignedarandomlygeneratednumber,v,fromauniformdistributionovertherange[0,1]. Unitswithv 0.5assi nedtotreatment unitswithv


12/60

ExperimentasRD

(Y)

Assignment Variable (T)

Cutting Point

0.50


13/60

EstimationBasics1

Wehavenowdefinedacausaleffectasthedifferenceoftwofunctionsatapoint.Howdowe

.

Approach#1:Comparemeans = , ,

arenounitsatthecutoffthatdontgetthetreatment,

butinprincipleitcanbeapproximatedarbitrarilywellby=

Thereforeweestimate:

cXYEcXYE ||

Thisisthedifferenceinmeansforthosejustaboveandbelowthecutoff.

Thisisanon arametrica roach.A reatvirtueisthatitdoesnotdependoncorrectspecificationoffunctional

forms.


14/60

NotethatIsaidinprinciplewecanestimatemeans

arbitraril closetothecutoff.In ractice,thisde endsonhavinglotsofdatawithinofthecutoff.Supposeyoudont.

TheoriginalRDdesign(Thistlewaite andCampbell1960)was

implementedbyOLS.

whereisthecausaleffectofinterestandisanerrorterm.

XTY

Thisregressiondistinguishesthenonlinearanddiscontinuousjumpfromthesmoothlinearfunction.

OLSwithonelinearterminXisseldomusedanymoreecauset e unctiona ormassumptionsareverystrong.

Whatarethey?


15/60

Supposetheunderlyingfunctionsarenonlinearandmaybeunknown.In

particular,supposeyouwanttoestimate

wheref(X)isasmoothnonlinearfunctionofX.

)(XfTY

.CommonpracticeistofitdifferentpolynomialfunctionsoneachsideofthecutoffbyincludinginteractionsbetweenTandX.

Modelingf(X)withapthorderpolynomialinthiswayleadsto

p

p

p

p

TXTXTXT

XXXY

...

...

221

02

0201

coefficientonTisthetreatmenteffect.

Commonpractice,forwhateverreason,seemstousea4th orderpolynomial,thoughyoushouldbesurethatyourresultsarerobusttootherspecificationsmoreont is e ow .


16/60

EstimationBasics4 OLSwithpolynomialsisaparticularlysimplewayofallowinga

flexiblefunctionalforminX.Adrawbackisthatitprovidesglobalestimatesoftheregressionfunctionthatusedatafarfromthe

cutoff. Theremanyareotherways,buttheRDsetupposesacoupleof

. Weareinterestedintheestimateofafunctionataboundarypoint.

(Forwhythisisaproblem,seeHTVorImbens andLemieux.)

Standardnon arametrickernelre ressiondoesnotworkwellhere

ThisleadstoApproach#3:LocalLinearRegression Insteadoflocallyfittingaconstantfunction(e.g.,themean),fitlinear

regressionstoobservationswithinsomebandwidthofthecutoff

Arectangularkernelseemstoworkbest(seeImbens andLemieux),butoptimalbandwidthselectionisanopenquestion

Aseriousdiscussionoflocallinearregressionisbeyondthescopeofthislecture.See forexam le FanandGi bels 1996

But,really,werejusttalkingaboutrunningregressionsondatanear

thecutoff.


17/60

RDPitfall:MistakingNonlinearity

forDiscontinuity

formarepotentiallymoresevereforRDthanforothermethodswearestudyingthisweek

Misspecificationofthefunctionalformmay

generateabiasinthetreatmenteffect emos commons u a o no s ype sw enanunaccountedfornonlinearityintheconditionalmeanfunctionismistakenfora

discontinuity Eachofthe3estimationmethodsdealswiththis

ssue na eren way


18/60

(Y)

False discontinuity



19/60

(Y)



20/60

LocalLinearRegression

(Y)



21/60


22/60

CompareMeans:SmallerBandwidth

(Y

)



23/60

Manipulation Ifindividualshavecontrolovertheassignmentvariable,thenwe

shouldexpectthemtosortinto(outof)treatmentiftreatmentisdesirable(undesirable) Thinkofameanstestedincomesupportprogram,oranelection

Thosejustabovethethresholdwillbeamixtureofthosewhowouldpassedandthosewhobarelyfailedwithoutmanipulation.

I in ivi ua s aveprecisecontro overt eassignmentvaria e,wewouldexpectthedensityofXtobezerojustbelowthethresholdbutpositivejustabovethethreshold(assumingthetreatmentisdesirable . McCrary(2008)providesaformaltestformaniupulaiton ofthe

assignmentvariableinanRD.TheideaisthatthemarginaldensityofXshouldbecontinuouswithoutmanipulationandhencewelookfor

.

HowprecisemustthemanipulationmustbeinordertothreatentheRDdesign?SeeLeeandLemieux(2010).

Thismeansthatwhen ourunanRD oumustknowsomethin aboutthemechanismgeneratingtheassignmentvariableandhowsusceptibleitcouldbetomanipulation.


24/60

ExampleofManipulation

n ncomesupportprogram nw c t oseearn ngun er , qua y orsupport

SimulateddatafromMcCrary2008


25/60


26/60

In rinci le,covariatesarenotneededforidentificationinRD,buttheycanhelpreducesamplingvariabilityintheestimatorandimprove

Thisisastandardargumentwhichalsosupports

inclusionofcovariatesinanalysesofrandomizedtrials Addingcovariatesshouldnotaffectthepoint

estimateoftheeffect(verymuch). Ifitdoes,

Thewiderthebandwidththemoreimportantitmaybetoincludecovariates.


27/60

GraphicalinspectionisanintegralpartofanyRDanalysis.

3typesofgraphsshouldalwaysbeproduced,where

1:theoutcome

2:othercovariates : ens yo cases

1shouldshowadiscontinuity;2and3shouldshownodiscontinuity

Ifyoucan'tseethemainresultwithsuchasimplegraph,it'sprobablynotthere

,


28/60

BandwidthSelection

ForLocalLinearRegression Bandwidthselectionrepresentsthefamiliartradeoffbetweenbiasandprecision

Whenthelocalregressionfunctionismoreorlesslinear,thereisntmuchofatradeoffsobandwidthcanbelar er.

Therearetwogeneralmethodsforselectingbandwidth

Adhoc,orsubstantivelyderived(e.g.,electionsbetween4852%areclose) Datadriven

Optimalbandwidthmethods(Imbens andKalyanaraman)

Crossvalidationmethods(LudwigandMiller;Imbens andLemieux)

ForPolynomialRegression Choosingtheorderofthepolynomialisanalogtothechoiceofbandwidth

Twoapproaches UsetheAkaike informationcriterion(AIC)formodelselection:AIC=Nln(2)+2p,where2 (shouldhavea

a s emeansquare erroro eregress onan p s enum ero mo e parame ers

Selectanaturalsetofbins(asyouwouldforanRDgraph)andaddbindummiestothemodelandtesttheirjointsignificance.Addhigherordertermstothepolynomialuntilthebindummiesarenolonger

jointlysignificant. Thisalsoturnsouttobeatestforthepresenceofdiscontinuitiesintheregressionfunctionatpointsotherthenthe

cutoff,whichyoullwanttodoanyway

Inbothcases Inpractice,youmaywanttofocusonresultsfortheoptimalbandwidth,butit'simportanttotestforlotsofdifferentbandwidths.Thinkoftheoptimalbandwidthonlyasastartingpoint.

Ifresultscriticallydependonaparticularbandwidth,theyarelesscredibleandchoiceof.

Inprinciple,theoptimalbandwidthfortestingdiscontinuitiesincovariatesmaynotbethesameastheoptimalbandwidthforthetreatment. Again,followthepracticeoftestingrobustnessto

variationsinbandwidth.


29/60

FuzzyRD Supposetheprobabilityoftreatmentchangesdiscontinuously

atathreshold,butnotfrom0to1.ThisisasituationforapplyingFRD. Notet att e uzziness inFRDcomes romt ec angein

probabilityoftreatment,notfuzzinessaboutthethreshold InsharpRDdesigns,thejumpintheoutcomeatthecutoffisthe

es ma eo ecausa mpac o e rea men . na es gn,thejumpintheoutcomeisdividedbythejumpinthe

probabilityoftreatmentatthecutofftoproducethelocalWald.(InSRD,thejumpisone,sothedivisionisinconsequential,butthisdemonstratestherelationshipbetweenSRDandFRD).

equivalent(andconceptuallysimilar)toIV(seeMostlyHarmlesssec.6.2)

treatmentbyassignmentvariableshouldshowadiscontinuous

probabilityatthethreshold


30/60

RegressionDiscontinuity


31/60

DoVotersAffectorElectPolicies?

byLee,Moretti,andButler(LMB)


32/60

Motivation:2fundamentallydifferentviewsoftheroleofelections

Convergence:Competitionforvotesdrivescandidatestoseekmiddlegroundpolicies,compromise(medianvotertheorem).

o ersa ec po cyc o ceso po c ans.

Divergence:Votersselectcandidates,whothenenacttheirownpreferredpolicies.Voterselectpolicies.

makecrediblepromisestoimplementpoliciesthatarenotattheirownblisspoint(crediblecommitmentsarefacilitatedbyrepeatinteractions)

ThegoalofthepaperistoexaminewhichphenomenonismoreempiricallyrelevantforUSpolitics,specificallyvotingintheHouse


33/60

Consider2parties,DandR

Rsblisspointis0,Dsblisspointisc(>0)

TheprobabilitythatDwinstheelectionisP

IfDwinselection,policyxisimplemented;ifRwins,yis

P*representstheunderlyingpopularityofpartyD,ortheprobabilitythatDwouldwinifx=candy=0.Anincreasein

*

Whendx*/dP*anddy*/dP*>0,wesaythatvotersaffectcandidatespolicychoices

*

Whendx*/dP*anddy*/dP*=0,wesaythatvotersmerelyelectpoliticianswithfixedpolicies.Thatis,anincreasein

* .


34/60


35/60


36/60

WhyThisWorks

Theelectcomponentis

1

records between the parties at time t Thefractionofdistrictswonb Democratsint+1

isanestimateof

Because we can estimate the total effect, ,,net out the elect component to implicitly getthe affect component

RandomassignmentofDtiscrucial.Without it,equation (5) would reflect 1 and that Dem


37/60

GraphicalEstimateofEquation4

20


38/60


39/60

0.50


40/60


41/60

atthediscontinuity

representativevotingrecords

Resu tsro usttoa ow ng orvar oussortso

districtheterogeneity

Results(smallaffectcomponent)stableovertime


42/60

RegressionDiscontinuityDesigns

ofProIncumbentBiasinCloseU.S.

HouseRaces

byCaughey andSekhon (CS)


43/60


44/60

TheLee(&McCrary)TestsforManipulation

A graph like (A) led Lee, and separately McCrary, to conclude that there is no manipulation.

However, (B) and (C) begin to suggest another story. Remember, the concern is with theincumbent partys vote share, not the Democratic vote share.

Density of the Assignment Variable


45/60

DensityoftheAssignmentVariable

Key Takeaway: The candidate of the incumbent party is about three times more likely to winelection by half a percentage point or less than to lose by a similar margin. The density of thisvariable appears to diverge rather than converge in the neighborhood of the cut-point.


46/60

BasedoncorrectingsomeofLeesdataandaddingsome

newvariables,CSfindimbalanceatthecutoffinthe

following: Democraticmargininthepreviouselection

thepartiesrelativecampaignexpenditures

1stdimensionNOMINATEscoreofthecurrentincumbent

whethertheDemocratic(Republican)candidateisthecurrentincumbent

numberoftermstheDemocrat(Republican)hasservedintheU.S.HouseofRepresentatives

w e er e emocra epu can asmorepo caexperiencethantheRepublican(Democrat)

CongressionalQuarterlysOctoberpredictionofwhich

Covariate Imbalance Graph


47/60

CovariateImbalanceGraph

S iti it t B d idth S l ti


48/60

SensitivitytoBandwidthSelection

P t ti l M h i


49/60

PotentialMechanisms

Notlikelybeoutrightfraud,becausesignificanceoflaggedvoteshareisincreasingovertimeandwebelievepotential

Controloverrecountsdoesnotappeartobethekeybecausetheyrarelyhappenandevenmorerarelychange.

Butwedontneedanexplanationbasedonvotecounting.Differencesbetweenwinnersandlosersinincumbency,

, , resourcesareevidentfarbeforeanyvotesarecast,counted,ormanipulated.

closeexanteandinthosethatwereinfactdecidedbyanarrowmargin.

,expectations,andallelseshouldbebalancedintheclosest

elections.


50/60

Disturbingly,themostseverelyimbalancedcovariatesarethoseclosely

thatis,thelaggedvalueoftreatmentandthesecondorderlagofthe

outcome.Onesuchcovariate,notshowninthebalancetable,isthegeneralpartisanswingamongallHouseracesinagivenyearrelativeto

. , , , ,1974,and2006,closeelectionsareoverwhelminglyconcentratedinnormallyRepublicanseats.Conversely,Democratheldseatstendtobe

closelycontestedinbadDemocraticyears,like1946,1966,1980,and.races,closeDemocraticvictoriesaremuchmorelikelytooccurinbadDemocraticyears,andcloseRepublicanvictoriestooccurinbadyearsforRepublicans.Totheextentthatbadelectionsforonepartytendtobe

,

incumbentpartyadvantagemaybecontaminatedbyregressiontothemeaneffects.


51/60

Thisisacautionarytale

LMBareverygoodscholars.

Theydidalmosteverythingright Theydonotdoalottojustifyfunctionalformorshowrobustnesstodifferentbandwidths

Remember,theLMBresultsforDemocraticvote(eqn.5)arenotimplicatedinthiscritique.Thisisthebestpartofthepaperanyway,IMHO.

Whatcanyoulearnfromthisexchange: Trytofindproblemsinyourdesignbeforesomeoneelsedoesitforyou

Identifyandcollectaccuratedataontheobservablecovariatesmostlikelytoreveal .

yourdataset. Laggedvaluesofthetreatmentvariablearealwaysagoodidea.Inelections,thepartythat

currentlycontrolstheoffice.

Automatedbandwidthselectionalgorithmsdonotguaranteegoodresults.Theyareustastartingpoint.

ForRDpurposes,whatconstitutesacloseelectionappearstobecloserthanthe4852%bandwidthwidelyuseduptonow.CSgetmostoftheirresultsusing49.550.5%.

GiventhecurrentfetishwithRDinpoliticalscience,understandthatitisnotafactofnaturethatcloseelectionsarerandom.Rememberthiswhenyousee(orsetout

towrite)thenextRDpaperoncloseelections.


52/60



53/60

Thetreatmentisdeterminedatleastin artb theassignmentvariable

Thereisadiscontinuityintheleveloftreatmentatsomecutoffvalueoftheassignmentvariable(selectiononobservablesatthecutpoint)

Unitscannotpreciselymanipulatetheassignmentvariabletoinfluencewhethertheyreceivethe

Othervariablesthataffectthetreatmentdonot


54/60

The strength of the RD design is its internal validity,arguably the strongest of any quasi-experimental design

External validity may be limited Shar RD SRD rovides estimates for the sub o ulation

with X=c, that is those right at the cutoff of the assignmentvariable.

The discontinuit is a wei hted avera e treatment effectwhere weights are proportional to the ex ante likelihoodthat an individuals realization of X will be close to thethreshold.

Fuzzy RD (FRD) restricts the estimates further to compliersat the cutoff (more on this below)

(e.g., treatment homogeneity)


55/60

Therearethree eneralt esofthreatstoanRD

design

1. Othervariableschangediscontinuouslyatthe

Testforjumpsincovariates,includingpretreatment

valuesoftheoutcomeandthetreatment2. Therearediscontinuitiesatothervaluesofthe

assignmentvariable

. an pu a ono eass gnmen var a e Testforcontinuityinthedensityoftheassignment

variableatthecutoff


56/60


57/60

1. Graphtheaverageoutcomesoverasetofbinsasinthe

caseofSRD,butalsographtheprobabilityoftreatment.

2. Estimatethetreatmenteffectusing2SLS,whichisnumericallyequivalenttocomputingtheratiointheestimateofthejump(atthecutoffpoint)intheoutcomevariableoverthejumpinthetreatmentvariable.

Standarderrorscanbecomputedusingtheusual(robust)stan ar errors

Theoptimalbandwidthcanagainbechosenusingoneofthemethodsdiscussedabove.

. ero us nesso eresu scan eassesse us ng evariousspecificationtestsmentionedinthecaseofSRDdesigns.

EvaluatinganRDPaper


58/60

g p

oss y our wn Doestheauthorshowconvincinglythat

Treatmentchan esdiscontinuousl atthecut oint

Outcomeschangediscontinuouslyatthecutpoint

Othercovariatesdonotchangediscontinuouslyatthecutpoint Pretreatmentoutcomesdonotchangeatthecutpoint

ere snoman pu a ono eass gnmen var a e unc ngnear ecutpoint)

Arethebasicresultsevidentfromasimplegraph?

Aretheresultsrobusttodifferentfunctionalformassumptionsabouttheassignmentvariable Forexample,parametricandnonparametricfits,differentbandwidths,etc.

Couldotherpossiblyunobservedtreatmentschangediscontinuouslyat

Forexample,18th

birthdaymarksadiscontinuouschangeineligibilitytovote,butalsoeligibilityfordraft,sentencingasanadult,andlotsofotherthings,whichmayormaynotberelevantdependingontheoutcomeinquestion

Arecasesnearthecutpoint differentfromcasesfarfromthecutpoint inotherways?Dothesedifferencesmakethemmoreorlessrelevantfroma

theoreticalorpolicyperspective?


59/60



60/60

Documents

RD Lecture