- 1 -

DesigningSocialProtectionPrograms:

UsingTheoryandExperimentationtoUnderstandhowtoHelpCombatPoverty

RemaHannaHarvardUniversity

DeanKarlanYaleUniversity

March2016

Abstract

“Anti-poverty”programscomeinmanyvarieties,rangingfrommulti-faceted,complexprogramstomore simple cash transfers. Articulating and understanding the root problem motivatinggovernmentandnongovernmentalorganizationinterventioniscriticalforchoosingamongstmanyanti-poverty policies, or combinations thereof. Policies should differ depending on whether theunderlyingproblemisaboutuninsuredshocks,liquidityconstraints,informationfailures,orsomecombination of all of the above. Experimental designs and thoughtful data collection can helpdiagnose the root problems better, thus providing better predictions for what anti-povertyprogramstoemploy inspecificconditionsandcontexts.However, themorecomplex theoriesarelikewisemorechallengingtotest,requiringlargersamples,andoftenmorenuancedexperimentaldesigns, as well as detailed data on many aspects of household and community behavior andoutcomes.Weprovideguidanceonthesedesignandtestingissuesforsocialprotectionprograms,from how to target programs, to who should implement the program, to whether and whatconditionstorequireforprogramparticipation.Inshort,carefulexperimentationdesignedtestingcanhelpprovidea stronger conceptualunderstandingofwhyprogramsdoornotwork, therebyallowing one to ultimately make stronger policy prescriptions that further the goal of povertyreduction.

- 2 -

I. IntroductionIn low-income countries, more than one billion individuals are enrolled in at least one

safetynetprogram(Gentilinietal.2014).1Theseprogramscomeinavarietyofformsand

sizes. Some aim to simply supplement consumption in hard times. Other newer, more

nuanced, social protection programs aim to address the underlyingmarket failures that

mayhavecontributedtoahousehold’spersistentstateofpovertyinthefirstplace,driven

byabeliefthatdirectlyaddressingthesefailuresmayhelpfamiliesbreakoutofapoverty

trap.Theultimate choiceofprogram—or combination therein—that countries choose to

implementwilldependgreatlyontheirsocialgoals,institutionalcapabilitiesandresources.

However, evenwithin eachbroad categoryof program, the specific design choicesmade

andmethodsofimplementationmayaffectwhethertheseprogramsactuallyachievetheir

statedgoals.

Inordertostartthinkingabouthowtodesign—andthentesttheimpactsof—safety

net programs, we begin by classifying them into four main categories based on the

underlying motivation for intervening. The first and simplest category is comprised of

programsdesignedforredistributivepurposes,e.g.,recognizingthatthemarginalutilityof

consumptionishigherforthepoorthanfortherich,andthustransfersaresociallyoptimal

fromautilitarianperspective(assumingnaturallythatthetaxationprocessdoesnotcreate

considerable deadweight losses).While these programs can differ in actual design, they

sharethecommonfeatureoffirstidentifyingtheneediestfamiliesalongaparticularmetric

and then providing themwith cash transfers. Of course, these programs could still have

long-rungrowthimpacts,e.g.iftheyarelargeenoughtoprovidesufficientcapitaltostart

new agricultural, migratory, or business activities (e.g., A. Banerjee and Newman 1993;

McKenzie and Woodruff 2006) or if they persuade risk-averse households to invest in

riskier, butmoreprofitable endeavors (Chetty andLooney2006) and so forth.2 But, the

1Throughoutthischapterwereferto“socialprotection”and“safetynet”programsasoneandthesame.2Notable examples that have found changes increases in investment as a result of cash transfer programsincludeCovarrubias,etal(2012)whofindthatMalawi’sSocialCashTransferledtoanincreaseinagriculturalinvestments; and Gertler, et al (2012) who find that Mexico’s CCT (Progresa) led to higher levels ofagriculturalincome,ashouseholdspartiallyinvestedaportionofthecashinproductiveassets.

- 3 -

primarygoalofthesetypesofprogramsissimplytolimitpovertyandhungerbyensuring

thathouseholdsattainaminimumlivingstandard.

Second,amissinginsurancemarketmaymotivateasocialprotectionprogram.The

poor facemanyrisks, suchasunexpectedhealthcosts,agriculturaldamageor job losses.

Without fully functioning financial markets, households may be unable to borrow to

smoothconsumption.Informalcreditandinsurancemarketsprovideanotheravenuetodo

so,buttheyoftenunderperformandhavebecomeevenlesseffectiveascountriesgrowand

urbanizationbreaksdowntraditionalsocialnetworks(Coady2004).Furthermore,evenif

householdssmoothconsumptioninthefaceofincomeshocks,theymaybedoingsointhe

short run by making long-term sacrifices, such as pulling children out of school. The

insurancemarketmayexistbutneedsasubsidytoincreasequantitydemanded,oritmay

notexistatallandthesafetynetprogramdirectlyprovidesprotectioninbadtimes.Social

protection from natural disasters is an extreme example of the insurance motivation;

however,economic theoryandempiricalworkhashad less tosayabout thestructureof

suchpolicies.Lastly,unemploymentandhealthinsuranceprogramsalsofitintothisoften-

(atleastpartially)missinginsurancemarketcategory.

Third, theremaybebehavioralorhouseholdbargainingconstraints that influence

the choices of low income households, and perpetuate poverty. For example, difficulty

resisting immediate temptations can lead to undersaving and thus underinvestment in

lumpygoods. Intra-householdbargaining issuescan lead tosuboptimaloutcomes for the

underpowered,typicallywomenandthuschildren.Manytransferprogramsaimtoaccount

forthesebehavioralfactors,byprovidingin-kindtransfers(suchasfood)ratherthancash

to prevent temptation or by providing transfers towomen rather thanmen to increase

female bargainingpowering in thehousehold. Importantly,manyprograms—conditional

cashorin-kindprograms—evendirectlyconditionassistancetopoorfamiliesonbehaviors

thatsocietywouldliketoaddress,e.g.childrenattendingschoolandreceivingvaccinations

orotherpreventivehealthmeasures.Finally,“workfare”programsalsosharethisaim,by

providingtransfersconditionalonlaborforcetrainingand/orparticipation.

Fourth,andfinally,theremaybemarketfailurespreventingassetaccumulation.We

focusontwodomainswhereassetaccumulationcouldbeimportantforsocialprotection.

- 4 -

First, short andmediumrunproductiveassetbuilding could increasehousehold income,

thus allowing individuals to no longer need consumption transfers from government.

Rather than cash transfers, productive asset transfers as well as training, coaching and

informational programs may be critical to motivate and improve investment choices to

makesuchagoalattainable.Second, long termfinancialasset-buildingmaybedifficult if

savingsmarketsaremissing,or ifbehavioral andhousehold constraintsdiscussedabove

bind. The current savings market infrastructure does not facilitate pension savings for

individuals,andbuildingsuchmarketscouldbecriticalforimprovingconsumptionforthe

elderlyindevelopingcountries,justasindevelopedcountries.

Naturallymanyprogramscanandoftendo targetmultiple issuesatonce, for two

reasons. First, individuals may face multiple market failures, thus motivating a more

complexprogram.Forexample,recentlymulti-facetedprogramshavearisenthatexplicitly

aim to “graduate” house holds out of extreme poverty by providing households with a

varietyofinputs,includingworkingcapital,assets,andjobstraining.Theseprogramsaim

to increase household earnings capacity by concurrently relieving a number of different

barriers to economic growth. Second, low income households even within a particular

settingmayfacedifferentmarketfailuresandthustheoptimalpolicymaynotbethesame

for all. This difference across households may motivate targeting different aspects of a

programtodifferentpeople,ordesigningprogramsthatmanagetoaddressdifferentissues

fordifferentpeople.

Inshort,awiderangeoftoolsareavailabletopolicy-makersinthegoalofpoverty

alleviation.Thisvarietynaturallycausesustoquestion:whataretherightprograms,who

should they be targeted to, and how do we know that they are working? Randomized

evaluations can answer this question by providing clear answers to whether or not a

program “works.” However, importantly, a randomized evaluation can go even further,

providing insights into why it works, i.e. the underlying mechanisms that drive the

observed treatment effects. By doing so, the evaluation can offer greater insight into

whetherasimilarprogramwouldworkelsewhereorhowtheprogramshouldchangeas

circumstanceschangeevenwithinthesamecontext.Inordertogeneratetheseinsights,a

well-designed evaluation must pay careful attention to the theory behind the different

- 5 -

forms of social protection programs, consider multiple treatment variations to isolate

theoreticalchannels,andbecreativeindatacollection.

Wewill firstdiscuss issues toconsider in testingprograms that fallundereachof

the four categories we lay about above, weaving in examples and knowledge from the

currentstateof the literature.3We thenaddress issuespertaining to implementation,as

success not only relies on whether the proposed program theoretically can achieve the

socialgoalitwasdesignedtoaddress,butalsoreliesonhowitisimplementedinpractice.

Finally, we offer advice on further research needed in this space, including a better

understandingofboththegeneralequilibriumandlong-runimpactsoftheseprograms.

II. RedistributiveProgramsMotivationsforredistribution,absentspecificmarketfailures,inevitablycomefromsocial

preferences for lower levels of inequality or from philosophical tenets such as

utilitarianism. In the simplest utilitarian form, elegantly put forward by Peter Singer

(1997),redistributionismotivatedbyanawarenessofthestarktradeoffsbetweenmoreof

one’s own consumption (for the wealthy) versus more consumption for the poor.

Philosophicalmotivations abound, naturally: for example, JohnRawls’ATheoryofJustice

(1971)argues for a veil of ignorance inwhich onemakesmoral and policy judgements

withoutknowledgeof one’sownposition in society.This hypothetical construct leads to

argumentsforredistributiontotheworst-offmembersofsociety.

With evenminimalweighting on other people’s utility in one’s own utility, some

redistribution typically becomes the individually optimal policy for all. This naturally

motivateswhyanindividualmayredistributetheirwealthtoothers(i.e.throughcharitable

actions),butnotnecessarilywhyagovernment,throughtaxation,mayeffectivelymandate

such redistribution. However, there aremany reasonswhy a governmentmaymandate

such redistribution, rather than leave it to the voluntary actions of individuals. First,

3Summarizing the broad and sizable literature on poverty alleviation programs poses a unique set ofchallenges. We have tried to focus on key ideas and use the literature to help provide examples whenpossible,aswellastogiveinsightintowhereopenquestionspersist. Wehavetriedtocovertheimportantpapers in the literature, but cannot cover all of themwhile keeping this chapter a reasonable length.Weapologizeinadvancetothosewhosepapersthatwedonotcoverindetail.

- 6 -

transaction costsmaybe high for individuals to target the poorest effectively. Second, it

maybecostlyforwealthyindividualstotransferwealthtothepoorest.Third,highlevelsof

inequalitycancausesocialstrife,orevenviolentuprising,andcollectiveactionproblems

make it difficult for a purely voluntary system to generate sufficient redistribution to

achieve the social optimal. Fourth, behavioral theories could explain why individuals

under-redistribute if left to their own device, but do have a stated preference formore

redistribution. This is akin to models of quasi-hyperbolic preferences or the dual-self

(FudenbergandLevine2006)appliedtoredistribution:ifoneismaximizingthewelfareof

themoredeliberativeandsharingself(asopposedtotheimpulsiveandselfishself),then

one may want a commitment device to help stay in line with their more deliberative

preferences.Agovernmentprogram for redistribution thusbecomessuchadevice.Fifth,

individualsmaybemisinformedaboutthecurrentlevelofinequalityandpoverty,andtheir

relative wealth, whereas policymakers may be more informed. In the United States, for

example,recentworkshowsthatmostpeoplevastlyunderestimatethelevelofinequality

in the United States, and state a preference for more equality, even though they are

simultaneouslyopposedtomoretaxes.Thisfindingdemonstratesaclearinformationgap

(NortonandAriely2011;Kuziemkoetal.2015).

In short, there are a number of rationales for governments to engage in

redistributiveactivities.Akeyempiricalquestionishowtobestdesigntheseprogramsand

test if they accomplish their goals. In doing so, the first aspect to consider is how to

identify the poor to direct resources towards them (“targeting”). In Section A, we first

discuss the potential strengths and weaknesses of common targeting methodologies, as

wellasdescribethekeyfactorsthatonemustconsiderwhenplanningrandomizedcontrol

trials(RCTs)ontargetingmethodologies.InSectionB,wethendiscusshowtoredistribute:

forexample,howbigshouldthetransfersbe?Howlongshouldthetransfers last? Inthe

processof thisdiscussion,werecountwhatweknowanddonotknow from thecurrent

experimental literature.We also provide a guide for the design of RCTs to evaluate the

typesofsocialprotectionprogramsoutlinedabove.

A.HowtoTargetthePoor?

- 7 -

Inhigh-incomecountries,targetingisoftenachievedbymeanstesting:householdsbringa

proofofincomeorunemploymenttoabenefitsoffice,ortheyreceivetransfersthroughtax

systems, such as through theUnited States Earned IncomeTax Credit. However, in low-

income countries, a lack of formal labor markets with a paper trail of income and

employment status, coupledwith underdeveloped tax systems, results in limited data to

verifyincome.

Inordertofillthisdatagap,low-incomecountry’sgovernmentscanconductincome

orconsumptioncensuses.However,suchcensusesalsopresenttheirownsetofchallenges,

asanyonewhohasevertriedtoconductasurveymoduletoelicitthesekindsofdatacan

attest:themodulestakeaninordinateamountoftimeandrequirecertainskills,sinceone

needstomapoutallthedifferentcomponentsofincome(e.g.farmingyourownlandone

day, casual labor the next) or consumption (e.g. items purchased, the crops that one

grows).Plus,withouta formalmechanismwithwhich to cross-checkdata, there isoften

nothingstoppingsurveyedpopulationsfromlyingiftheyknowthatacashprizeisattached

totheiranswers.

As such, low-incomecountries tend todevelopalternativemethodswithwhich to

identifythepoor.Themethodchosenwilldependontheprioritiesofthegovernment:for

example,istheaimtotargetbasedonaparticularpovertyline?4Isitpreferabletotarget

thepoorbasedon income,consumption,orsomeothermetricofpoverty?Howspatially

dispersedare thepoor? Itwill alsodependon the institutions inplaceandcontext:how

good is the implementing agency’s ability to conduct surveys?How responsive are local

leaders to citizens?We first outline key categories of targetingmethodologies and then

discusshowexperimentalmethodscanhelpdistinguishbetweenvaryingfeaturesofthese

methods.

Finally,note thatwe focusontargetingpoorhouseholds in thissection,giventhat

the goal of redistributive programs is to provide impoverished householdswith a basic

standardof living. Transferprograms thathope toenact longer runchangesmay target

4Note that in addition to targeting poverty status, some programs also target particular demographiccharacteristics, suchaswhetherawoman ispregnantorhaschildren.As targetingon thesecharacteristicstendstobeeasier—duetotheirverifiability—wewillnotdiscussthemindetailhereforconciseness.

- 8 -

differently,dependingagainontheirgoals.Forexample,Barrera-OsorioandFilmer(2013)

comparetheeffectivenessofscholarshipswhentheyaretargetedtothepoorversuswhen

they are merit-based: while both increase school enrollment, only merit scholarships

increasetestscores.Thus,themethodthatyouwouldchoosewoulddependuponwhether

youwould liketoredistributescholarshipstothepoor,ortoredistributescholarshipsto

thosewhowillhavethehighestmarginalreturn fromthemgivenametricof testscores.

We will revisit the idea of differences in targeting methods and goals below, when we

discussprogramsthataimtochangelong-runoutcomes.

1.TargetingMethodsWhile there are many variations in practice, there are four primary categories that

encompassmosttargetingmethodologies:

• Geographic:Ifthepoorareconcentratedinparticularvillages,districts,orregions,

givingeveryonewithin thoseareasaccess tosocialprotectionmaybeaneffective

methodtotransferresourcestothepoor(see,forexample,BakerandGrosh1994;

Elbersetal.2007).Moreover, this formof targetingmaybeparticularlyattractive

when the institutional capacityneeded to collect individual information is low, as

onlyaggregateinformationisneeded,e.g.povertymappings,rainfalldata,etc.Note,

however,thatthismethodmayalsobepoliticallysensitiveasitdisbursesbenefitsto

someareas,butnotothers.

• Proxy-means testing (PMT): In thismethod, thegovernment collectsdemographic

andassetdatafromhouseholdsandusesthesedatatopredictor“proxy”incomeor

consumption.5 Sometimes this method involves a quick-and-dirty poverty score

card with just a few questions. Other times, a longer, more detailed asset and

demographic survey is conducted. However, in either case, the key is to choose

5Thisistypicallydonewithanationallyrepresentativedatasetthatincludesthevariableonwhichtotarget(e.g. income),aswellasnumerousdemographicandassetvariables.Next, income isregressedondifferenthousehold characteristics, looping through different combinations and permutations of the variables, untilthe set of characteristics that best predicts income is identified you find (often regional fixed effects orregional variables are also included for better precision). After conducting a census to obtain the chosenhousehold characteristics, it is then possible to compute predicted income for each household using theformula.Householdsbelowachosencutoffofpredictedincomewouldthusbeeligiblefortheprogram.

- 9 -

variables that are simple to collect, relatively easy to verify (e.g. whether the

householdhasadirtorconcretefloor,oriftheyhavetelephoneline),andthatare

less likely tobedistortionary (e.g. school enrollmentmaypredictpoverty,butwe

may not want to incentivize households to keep their kids out of school).

Householdsthatpasstheproxymeanstest—i.e.arebelowacertainpovertyline—

are then automatically enrolled. The effectiveness of the PMT will depend upon

differentfactors:theformula’spredictivepower,thequalityofsurveyteam,etc.

• Self-targeting: Self-targeted programs are those in which everyone is allowed to

apply,but inwhich somesortof “barrier” isput intoplace inorder to reduce the

probabilitythatrichhouseholdstrytoaccesstheprogram.Theoretically,thereare

differentbarriersthatcangeneratethis formofselection(NicholsandZeckhauser

1982), ranging from time costs to apply, means testing on arrival (Alatas et al.

2015), and work requirements (Besley and Coate 1992; Ravallion 1991).6 When

done right, this can effectively screen out the rich (see, for example Alatas et al.

2015;Christian2014).Giventhathouseholdsmayfallintoandoutofpoverty,these

programs also have the potential advantage of flexibility, allowing households to

accesstheprogramswhentheyhaveabadshock.However,theseprogramsalsorun

a substantial risk: itmaybehard for governments to initiallypredict, in advance,

howmanypeoplewillactuallyenrollorparticipate,providingadditionalchallenges

tobudgetingandimplementation.

• Community basedmethods: In this method, community members choose who in

their locality are needy. Theoretically, thismethod could not only bring in better

local information on who is poor, but it could also incorporate the community’s

perceptions as to what determines poverty in their location (Seabright 1996). A

6Forcertaintypesofin-kindgoods—subsidizedhealthproducts,insuranceproducts—thepricechargedmayalsobeusedtoselectaparticularly typeofpersonwhomayvaluetheparticularproduct(see forexample,CohenandDupas,2010;Beaman,etal,2014).WediscussthisinmoredetailinSectionIV,whenwediscussprogramsthataregearedatchanginglonger-runincomeorbehavior.

- 10 -

potential benefit is that this may make the programmore politically popular, as

peoplemay feel that the list ismore in-linewith their vision ofwho is needy or

deserving.Akeyworryisthatbyallowingforlocaldiscretioninchoosingprogram

recipients,elitesmaypossiblycapturetheprocess(BardhanandMookherjee2000).

Moreover,anotherpotentialdownside is thatwhile thismethodelicitsbetterdata

onrelativepovertywithinanarea, itdoesnotprovideinformationacrossvillages.

Forexample,Alatasetal. (2012)showthat thedifferencebetween thePMT’sand

community’sabilitytotargetbasedonconsumptionnearlydoubleswhenthePMT

isallowedtouseitscross-villageinformation.

Ultimately, themethod anddesignwill depend on a number of factors: the goals,

context,institutionalcapacity,targetingbudget,etc.Forexample,Alatasetal.(2012)shows

that thechoiceofcommunitymethodsversusPMTcandependonwhetheronewants to

specificallytargetahardmeasureofpoverty(e.gincome)orasoftone(e.g.perceptions).

Similarly,choosingtherightdesignofthePMT—e.g.thenumberofquestionsthatgoesinto

the formula—will also depend on the institutional capacity to administer the PMT.

Moreover,Beathetal.(2013)showsthatwhetheraidallocationsreachtheneediestornot

candependonthetypeofcommunityinstitutionsthatparticipateintargeting.

Inpractice,while therearedistinctcategoriesofmethods,governmentsoftenmix

andmatch themethods depending on circumstance. For example, they often try to save

money by conducting a PMT on a selected sample of peoplewho are likely poor, rather

than conducting a full PMT census. For example, in its earlier form, Mexico’s Progresa

programconductedaPMT todetermineeligibilityonly in the areas thatwere chosenas

likely poor based on geographic targeting (Schultz 2004). Similarly, Indonesia’s Data

CollectiononSocialProtectionProgramme(PPLS)usescommunity-basedmethodstohelp

determine the list of households that will receive the PMT survey (Alatas et al. 2012).

Kenya’sCashTransfer forOrphansandVulnerableChildrenactuallyuses threemethods:

first,geographictargetingisusedtodeterminelocations,thencommunitytargetingisused

in the selectedareas todeterminea list ofhouseholds, and finally, thosehouseholds are

givenaPMTtodetermineactualeligibility(TheKenyaCT-OVCEvaluationTeam2012).

- 11 -

2.ExperimentallyTestingBetweenTargetingMethods

Are experimental methods important for testing across targeting approaches? Not

necessarily.Forexample,onesimpleresearchdesignwouldjustbetotryouttwodifferent

methodologies in the same areas and then compare the income levels of those selected

underbothmethods.

Whilethisresearchstrategymayproveattractiveinsomeways,itmaymissouton

the nuances of targeting that may ultimately be quite important in understanding the

relative effectiveness of the differing methods. First, many of these targeting methods

require considerable effort on the part of citizens and the program staff.7Simulating

conditionstobeasrealaspossible,withatleastasmallamountofcashonthelineforthe

households that are to be selected, is important to understand how themethodswould

actuallyworkinpractice.8Intermsofcitizens,peoplemightbehavedifferentlyduringthe

process if they do not believe that the targeting list will actually be used to distribute

resources.Forexample,theymaynotexertanyeffortindiscussingandrankinghouseholds

incommunitytargeting,orindividualsunderself-targetingmayjustnotbothertoshowup

eventhoughtheywouldiftherewererealcashinvolved.Moreover,peoplemaygivemore

truthfulanswers in thePMT,ornot claimagreaterpoverty status than their realityata

communitymeeting, if they believe that their answers do not have any consequence. In

termsofstaff,theresultsmaynoticeablydifferifyoutestoutmethodswithoutstakeswith

ahighlytrainedsetofstaffthaniftryingoutthemethodswiththetypicaltypeofstaffthat

wouldbehiredandtrained(Alatasetal.2012).

Second,thechoiceoftargetingmethodmayaffectboththeprogramandhousehold

outcomes. For example, the targeting method chosen may help determine the ultimate

7One obvious exception to this is geographic targeting, which does not rely on field operations, such ashouseholdandcommunity leader interactions. Ifadifferentmethodwasconductedfor theactualprogram,then simulating geographic targeting over the same areas based on administrative data can provide anaccurate comparison of the type of person selected under both methods. However, by not conductinggeographic targeting in practice, it will not be feasible to test program outcomes other than just who isselected(e.g.politicalacceptability,leakages)thatresultfromusingdifferenttargetingmethods.8You could, for example, try out two methods in the same area and offer a transfer to everyone who isselectedbyeitherofthedifferentmethods,butthenstafforvillagesmaycoordinatesothatdifferentpeopleareoneachlist.And,itrunstheriskofconfusingpeople,sothattheydonottaketheexerciseveryseriously.

- 12 -

satisfaction of the program, which in turn may affect program acceptance and the

government’sability to implementtheprogram.Targetedprogramscanbecontroversial,

withsomehouseholdsreceivingatransfer,whiletheirneighborsdonot.Ifcitizensbelieve

that a certainmethod isunfair and that itwouldproducea flawed list, theymaybe less

likely to support the overall social protection program and potentially block the

distribution of benefits.9 Moreover, if people believe that the wrong individuals were

chosenduetothefactthatacertainmethodwasused,itmaypossiblyleadtodistortionsin

howinformalinsuranceorlendingoperateswithinacommunity.

Finally, who is chosen may be different than who actually receives the transfer

(Alatasetal.2013)andthismayalsovarybythetargetingmethodthatwasemployed.For

example, suppose that thePMTbetter selected thepoor than a communitymethod.But,

that thePMThad less legitimacy than thecommunitymethod, so thatvillage leadersdid

notadheretothePMTinpracticewhendistributingthetransfers,buttheydidadhereto

thecommunitylist.Inthiscase,simplysimulatingbothmethodsinthesameareatoelicita

beneficiary listwouldpossiblywronglysuggestthatoneshouldselect thePMTsinceyou

wouldonlybeable tostudywhowaschosen,butnotbeable tomeasurewhoultimately

wouldgetthebenefits.

Inshort,forallofthesereasons,wewouldwanttorandomizethetargetingmethods

todifferentareastoseehowthemethodaffectswhoischosenandwhoultimatelyreceives

the transfers. The optimal design for a field experiment in the domain of targeting will

dependonanumberoffactors,includingwhichmethodisbeingstudied,butalsowhatthe

particularcontextlookslike.However,thereareanumberofkeyquestionstokeepinmind

regardlessofthegivendesign.

The firstquestion iswhetherornot it isnecessary tohaveacontrolgroup, in the

traditional sense. RCTs generally compare the outcomes from a treatment group that

receivesaninterventionwiththoseofacontrolgroupthatdoesnot.Inthiscase,sincethe

9Forexample,Alatasetal. (2012)showthat community targeting led tomuchhigher levelsof satisfactionthan the PMT,with village leaders feeling less comfortablemaking the transfers under the guise of publicscrutinywhen thePMThadbeenused.Theyprovide suggestiveevidence that thisdifference isdue to theperceptionsofthemethods,andnottheultimateliststhatthedifferentmethodsproduced.

- 13 -

outcomeof interest iswhoisselectedunderdifferenttargetingmethodswithinthesame

program, it may be viable to simply have multiple treatment groups where each is

randomlyassignedadifferenttargetingmethod,buteveryonereceivestheprogram.

Second, at what level should the randomization take place? Randomizing at the

individualleveloffersthemoststatisticalpower,butinthiscase,thetargetingtreatments

often involve some sort of group participation (e.g. community, self-targeting) or group

data (e.g. geographic).Moreover, evenwith a PMT,where it is possible to vary how the

survey is conducted across individuals, a question of interestmight also bewhether the

programadministratorschangehowtheyrespondtothetargetinglistintheirareabased

onwhichmethod generated the list. Thus, inmost cases, it is appropriate to randomize

across a sensible geographic unit; as such, power calculations to determine sample size

shouldaccountforthegroupstructureofthedata.

Third, is a baseline survey needed? With most experiments, the answer is not

necessarily:theanalysiswillconsistofcomparingoutcomesofthoseinthetreatmentand

the control group. A baseline might help for power if the outcome measures within a

personarehighlycorrelatedacrosstimeanditmayallowyoutotestfortheheterogeneous

treatment effects by various baseline characteristics (Duflo et al. 2006). But, it is not

necessaryperseinatypicalrandomizedexperiment.Inthiscase,abaselineisessential:a

key outcome of any targeting experiment will be the baseline income or consumption

levels—priortothetargeting—ofthosewhoareactuallychosen.

Fourth,whatkindsofdatashouldbecollected?Theexactvariableswould,ofcourse,

dependonwhatmethodsarebeingtestedandwhatoutcomesareexpected.But,typically,

inthebaseline,itisessentialthatdatabecollectedonthevariablesbeingtargetedon(e.g.

income,consumption,etc.), so that inclusionandexclusionerrorscanbecomputed.10To

measuredistortions inwho is chosenalong certaindimensions, itmaybeworthwhile to

collectbaselinedataonpoliticalaffiliationsandrelationstopoliticalleaders.Intheendline

surveys, valuable data to collect could include who actually received the program,

satisfactionlevels,andmetricsongeneralprogramfunctioning.

10Alatas et al. (2012) also ask villagers to rank one another to gain the “average” person’s belief aboutanotherhousehold’spovertystatusinavillage,aswellasaskhouseholdtoassesstheirownpovertystatus.

- 14 -

Finally,will theexperiment just aim tomeasure the reduced formeffectorwill it

also attempt to understand why it is working (or not)? If the only relevant question is

comparingmethod one versusmethod two (i.e. the reduced form difference of the two

programs),onlytwotreatmentarmsareneeded.But,weknowthattheeffectivenessofa

methodmayvarybasedon itsowndesign, and therefore, itmaybeworthwhile to learn

more about amethod’s effectiveness if certain details of the implementation are varied.

Here,theorycanhelpguidetheappropriatesub-randomizations:Forexample,Alatasetal.

(2015) experimentally compare the outcomes of a PMTwith a self-targetingmechanism

within Indonesia’s conditional cash transfer program. Importantly, they experimentally

varythedistanceoftheapplicationsiteunderself-targetinginordertogenerateexogenous

variationinthecostoftheapplication“barrier.”Theythenusethisvariationtoestimatea

modelofthedecisiontoapplyfortheprogramandsimulateself-targetingoutcomesunder

differentlevelsandtypesofapplicationcosts.

B.EvaluatingtheImpactsofRedistributivePrograms

Once the poor have been identified, the next task at hand is to think aboutwhether the

redistributiveprogramsareachievingtheirgoals inpractice. Thesimplestredistributive

programsarethosethatentitletheidentifiedpoortosomeformofcashstipendinorder

provide a certain standard of living and are unconditional on any behaviors (an

unconditionalcashtransfer,orUCT).TheChineseDi-BaoProgramisthelargestUCTinthe

developingworld,reaching78millionhouseholds(seeChenetal.(2006)foradescription

of the program). Other prominent examples include South Africa’s Child Support Grant,

India’sNationalOldAgePensionScheme,andKenya’sHungerSafetyNetProgram.

Evaluating theseprogramsusually followsa typicaldesign.First,poorhouseholds

aretargetedinthesamplearea.Then,potentialbeneficiariesarerandomlyassignedtothe

treatmentgroup(“receivestransfers”)andthecontrolgroup(“doesnotreceivetransfers”)

toassess theprogram impacts.Examplesofunconditionalcashprograms thathavebeen

evaluatedinthisfashionincludetheZambiaChildgrantprogram(Jesseeetal.2013),the

Kenya Hunger Safety Net program (Merttens et al. 2013), Kenya's Cash Transfer for

- 15 -

Orphans andVulnerable Children (see for example, TheKenya CT-OVCEvaluationTeam

(2012)andCovarrubiasetal.(2012),ontheMalawiSocialCashTransferScheme).11

Eventhoughtheoverallevaluationstrategyisrelativelystraightforward,thereare

stillanumberofdecisionstobemade—concerningthelevelofrandomization,thetypeof

datacollected,thetimingofdatacollection,etc.—thatareimportantinassessingimpacts.

Thelevelandformofrandomization: transfers are generallyprovided to an individual or

household,andsoitistemptingtorandomizeattheindividual-leveltomaximizestatistical

power.Forexample,Schadyetal.(2008)doexactlythis.However,AngeluccianddeGiorgi

(2009), among others, show that there may be spillovers of transfers from eligible to

ineligiblehouseholdswithinavillage.12Thisimpliesthattherandomizationlikelyneedsto

be done at a higher level, at the village-level or sub-district level depending uponwhat

typesofspilloversonemightexpect.13

Ifrandomizationisatagrouplevel, it isvitaltohaveenough“units”torandomize

overoritwillbechallengingtomeasureimpacts.Forexample,theKenya'sCashTransfer

forOrphansandVulnerableChildren(TheKenyaCT-OVCEvaluationTeam2012)hadonly

28 units of randomization, which might account for their difficulty in detecting any

programimpacts;theMalawiSocialCashTransferSchemehadonly8clustersanddidnot

fullyaccountforthegroupednatureofthedataintheanalysis(Covarrubiasetal.2012).14

Thus, one shoulddetermine in advancewhat thedesired sizeof treatment effects is (i.e.

11ThereweretwoRCTsconductedonBonodeDesarrolloHumano–oneonchildhealth(PaxsonandSchady2010;FernaldandHidrobo2011)andoneoneducation(Schadyetal.2008;EdmondsandSchady2012).Theprogram was initially supposed to be conducted as a conditional cash transfer program and someannouncementsweremade to this effect (Paxson and Schady (2010) hadmultiple treatment arms to testbetweenpure cash and cashwith conditions), but the conditionswere never enforced. Given that just theframingoftheprogramasaCCTcouldstillhaveimpacts,wedonotincludethisinourdiscussionofUCTs,butinsteaddiscussitbelow.12Aswediscussbelow,AngeluccianddeGiorgi(2009)evaluateaconditionalcashtransferprogram,butthebasicideasonspilloversholdforunconditionalcashtransfersaswell.13Aswediscussbelow,onemightevenwanttodesignthestudytocapturedifferenttypesofspilloversandgeneralequilibriumeffects.14In the analysis of these cases, it would be necessary to adjust standard errors to account for both thegroupednatureoftherandomizationandthesmallnumberofgroupsinkeepingwiththeprocedureoutlinedbyCameronandMiller(2010).

- 16 -

largeenoughfortheprogramtobecost-effective,etc.)andusethistoassessthestatistical

poweroftheproposeddesign.

Importantly,itisalsokeytoidentifythebeneficiariesinthecontrolgroup,notjust

thetreatmentgroup.SupposearandomizationdeterminedwhichvillagesobtainedaUCT

andwhichwereinthecontrolgroup.IntheUCTvillages,targetingwouldhavefirstbeen

conducted to choose the beneficiaries within the village. However, unless a similar

targetingstrategywasalsoconductedinadvanceforthecontrolgroup,itwouldbedifficult

know who were the hypothetical beneficiaries in the control group (see, for example,

Covarrubiasetal.(2012)).Inthiscase,itwouldbepossibletoestimatetheimpactonthe

entire village as awhole, but not easy to estimate the impact on just the beneficiaries.15

Thus,itisessential,whenpossible,tousethesametargetingmethodsinthetreatmentand

control group, even if the control group is not receiving the program at the time of the

study.

Data Collection: What data should be collected and when? If the goal is simply

redistributive,wemaysimplycarewhetherpoorhouseholdswereactuallyidentifiedand

whetherornottheywerereceivingtheirentitlements.Inmanydevelopingcountries,due

to weaker institutional structures, corruption, or imperfect information on their

entitlements, households do not receive the full transfer. So, an important set of survey

questions should be focused on whether households actually received the transfer,

whether they received the full transfer, whether the village elites held them up for a

portion of their transfer, and so forth. Next, one may want to administer a household

incomeorconsumptionmodeltomeasurehousehold’sincomestatus.

Ifwe simply care about redistribution,wewould not necessarily care about how

households spend the transfers—we would just care about whether they receive the

money. Thus, if motivated by such a philosophy, one may argue to only measure the

15Ofcourse,thisisnotaproblemwithinthevillageiftheprogramisgeographicallytargetedandeveryoneinallsamplevillagesiseligible.However,thismayalsogeneratespilloversoverlargergeographicregionsthatonemaywanttobeawareof,e.g.ifeveryoneinadistrictofprovincereceivesadditionalincome,wouldthisincreasedemandforfood,raisingfoodprocess?

- 17 -

transfers, and not bother examiningwhat happenswith themoney transferred. But, for

many reasons researchers do collect more data. For example, one political rationale of

manyredistributiveprogramsistoprovideabasicstandardof livingforchildren,soone

maywanttocollectmeasuresonchildhealth,nutrition,andeducation.Thereareanumber

ofissuestoconsiderindoingthistypeoflargerdatacollection,butwewillrevisititbelow

whenwediscussbroaderissuessurroundingbehavioralchangeforprogrambeneficiaries.

Next,akeyquestioniswhetherornotabaselinesurveyisneeded.Inassessingthe

impactofatransferprograms,abaselinesurveyisnotnecessarilyneeded,sincetreatment

wasassignedrandomly,meaningthatapost-interventionofthetwogroupsisanunbiased

estimateofthetrueimpact.However,therearetwokeyexceptionstothis.First,ifwewant

toassesswhetherthepoorwereindeedtargetedandwhetherthepoorestofthepoorwere

able to receive their entitlements, thenwewould need baseline consumption or income

data. Second, if particularly vulnerable sub-groups areof importance (e.g. the verypoor,

familieswith childrenwhoare less likely toattendschool, etc.), onemaywant to collect

data on these groups—if administrative data of this sort is unavailable—to be able to

stratifytherandomization.

Finally,whendoyouconductthefollow-upsurveys?Again,itdependsabitonthe

researchaims.Afollow-upsurveyshouldgenerallybeconductedwithinthedurationofthe

program to understand whether households are receiving the transfer. However, as we

discussbelow,ifthelonger-runimpactsoftransferprogramsareofinterest,onemayalso

want to do additional surveys sometime after a household is no longer enrolled in the

program.

III. MissingInsuranceMarketsThe poor face many risks, such as unexpected health costs, agricultural damage or job

losses.Withoutfullyfunctioningfinancialmarkets,householdsmaybeunabletoborrowto

smoothconsumption.Furthermore,evenifhouseholdsdosmoothconsumptionintheface

ofshocks,theymaybedoingsointheshortrunbymakinglonger-termsacrifices,suchas

pullingchildrenoutofschoolornotinvestinginhealth.Attheextreme,naturaldisasters—

- 18 -

e.g. earthquakes, flooding, famine—may cause group shocks that further limit the

functioningabilityofinformalinsurancemechanismswithinaregion.

Thus,an importantroleofsocialprotectionmaybetohelpalleviate the impactof

such shocks by providing social insurance of various forms. Two prominent forms of

insuranceareagriculturalinsuranceandhealthinsurance,butwewillnotcoverthemhere

sincetheyarediscussedelsewhereinthisHandbook. Ratherwewillbrieflydiscusstwo

otherformsofinsurance:disasterreliefandunemploymentinsurance.

Disaster relief is an important form of social insurance. While some quasi-

experimentalworkhasbeendone to studyboth the consequences of a disaster, and the

distribution of aid, to our knowledge, there has been less experimental evidence in

evaluating social protection inhumanitarian settings.Notable exceptions includedeMel,

McKenzie, and Woodruff (2008), who studied the effect of cash transfers to

microenterprises in post-tsunami Sri Lanka, as well as Aker (2014) and Hidrobo et al.

(2014) that examined cash versus other types of transfer programs in informal refugee

campsintheDemocraticRepublicofCongoandNorthernEcuador,respectively.Partofthe

reasonforthelackofexperimentalevidencecomesfromethicalconcerns,withaneedto

provide immediate assistance trumping research and evaluation. Related is a logistical

challenge:thechaosandhighly-transientnatureofrefugeecampscanmakerandomization

particularly challenging to implement.However, given theabout60million refugeesand

internally displaced people worldwide (UNHCR 2014), understanding how safety net

programscanbebettertailoredtoprovideassistanceinhumanitariansettingshasbeenan

increasingly urgent need, particularly as many refugee camps persist for long after the

immediacyofthedisaster.

A second important form of insurance is unemployment insurance (UI), the

provisionoftemporaryassistancetothoseoutofwork.Thisformofsocialprotectionhas

been traditionallymissing from poor countries, as the data-poor environments preclude

easy identification of those who are working in full-time positions in order to then

determinewhoistemporarilyoutofwork.But,itisbecomingmorecommoninrelatively

more developed, low-income countries with formal labor markets (e.g. Brazil, Egypt).

WhiletherehavebeennumerousexperimentsthathavetriedtounderstandaspectsofUI

- 19 -

in developed countries, to our knowledge, there is little experimental evidence on UI in

lowertomiddleincomesettings.16OnenotableexceptionisMickelwrightandNagy(2010),

whichrandomizedunemploymentbeneficiaries,inHungarytovaryinglevelsofmonitoring

(i.e.visitingtheemploymentofficeeverythreemonthswithnojobsearchquestionsversus

visitingevery threeweeks toanswerquestionson jobsearchbehavior). However,asUI

spreads and becomes a more important component of social protection in developing

countries, empirical evidencewillbeneeded tounderstand its impactson labormarkets

and household outcomes. Empirical evidencewill also be needed to understand how to

betterdesigntheseprograms—addressinghowtobestverifyemploymentstatus,howto

best distribute benefits, andhow to ensure that theprogramsprovide incentives to find

work.

IV. BehavioralConstraintsItisoftenarguedthatbehavioralorhouseholdbargainingconstraintsinfluencethechoices

of low-income households, thereby perpetuating poverty. Regardless ofwhether or not

thesebehavioralconstraintsarepresent,safetynetprogramsareoftendesignedtocorrect

thesetypesofconstraintsandencouragesociallyoptimalbehavior.

There are two key forms that behaviorally-focused social protection programs

usuallytake. Thefirstisprovidingin-kindtransfersratherthancash,underassumptions

suchaspeoplewillbetootemptedtospendmoneyon“bad”goods(suchastobaccoand

alcohol)or thathouseholdbargainingconstraintswould imply that cash transferswould

cause socially undesirable outcomes, such as an underinvestment in children or an exit

fromthelabormarket.

The second is to directly condition the receipt of transfers to households on a

household’scompliancewithcertainlong-terminvestments,suchasthechildrenattending

schoolorvisitinghealthclinics. Theseconditionalcashtransferprograms,orCCTs,have

16Forexample,Meyer(1995)summarizesanumberofearlyexperimentsonUIintheUnitedStates,focusingonfourcashbonusexperimentsandsixjobsearchexperiments.MorerecentexamplesincludevandenBergandvanderKlaauw(2006),whoexploredtheeffectofadditionalmonitoringandcounselingonUIrecipientsintheNetherlands;GrenierandPattanayak(2011),whomeasuredtheimpactoflegalassistanceonUIintheUnitedStates.

- 20 -

becomeincreasinglycommoninLatinAmericaandhavebeguntospreadtootherpartsof

theworld,existinginmorethan52developingcountriesasof2013(FiszbeinandSchady

2009; Saavedra and Garcia 2012; Gentilini et al. 2014). And, finally, note that some

programsdoacombinationofbothin-kindandconditions,with130countriesdoingsome

sortofconditional,in-kindprogram.

Carefulexperimentationthathelpstesttheunderlyingrationalefortheseprograms

canhelpdetermine if behavioral constraints exist, and if so,what form they take. Thus,

they can provide insights into whether these kind of constraints on behavior actually

improvewelfareorsimplymakeredistributiveprogramsmoreexpensive.

A.ProvidingIn-kindTransferstoConstrainSpendingChoices

1.TheRationaleforIn-KindPrograms

Unconditional, in-kindredistributiveprogramstypicallyprovidefreeorhighlysubsidized

goods,toprogramrecipients.Theycanentailadirectprovisionofthegoods,suchasafood

orfueltransferprogram;thesubsidyofaproductdistributedthroughlocalgovernments,

NGOs,ordesignatedshops;orasystemofvouchersthatareconstrainedtoparticulartypes

ofgoods,suchasafoodstampprogram.17Gentilinietal.(2014)notesthat89low-income

countries have unconditional in-kind transfer programs, with examples including

Indonesia’sRiceSubsidyProgram(“Raskin”)and thePublicDistributionSystems inboth

BangladeshandIndia.

There’smuchdebateaboutwhethertransfersshouldbein-kind.Ifyouaskatypical

economist,mostwill favor cashprograms,under the idea thathouseholdswillmaximize

utility if theyhavechoiceoverwhattheypurchase,ratherthanreceivingagoodofequal

monetary worth that they may not value as much. However, there are a number of

argumentsproposed in favorof in-kindsubsidies (see CurrieandGahvari (2008) foran

excellentreview),whichmayexplaintheirgeneralpersistenceworldwide.

17Itisimportanttonotethatthereareothertypesofin-kindtransfers,suchasonesthatprovidefreehealthproducts(CohenandDupas2010;Dupasetal.2013;Maetal.2013;Glewwe,Park,andZhao2014),prizesorscholarshipsforschool(Berry2014;Kremer,Miguel,andThornton2009),schoolmealsprograms(Kazianga,DeWalque, andAlderman2012;Vermeersch andKremer2004), public housing (Kling et al. 2004). Theseprogramsmay alsohavedifferent aims, such as solving the externality issue inhealthproduct take-up.AstheyarediscussedinotherchaptersinthisHandbook,werefrainfromdiscussingthemhere.

- 21 -

Themostcitedexplanationisthatofpaternalism:peopleoftenhaveanimageofthe

lazy,out-of-workhusbandco-optingthefamilycashandwastingitonalcohol,tobaccoand

otherformsofentertainment,ratherthanmakingspendingdecisionsthatcanimprovethe

family’slivingsituationorinvestinginchildren.Thus,theargumentfollowsthatanin-kind

subsidy could reduce the husband’s ability to do so, forcing redistribution within the

householdtothewomenandchildrenwhotypicallyhavelesshouseholdbargainingpower.

However,othersarguethatifthein-kindsubsidyisinfra-marginal—orifitiseasytoresell

goods—thenin-kindsubsidieswillnotalterthehousehold’sconsumptionbundle,anditis

simplyamorecostlymechanismto redistribute to thepoor thancash.Theexperimental

evidence thus far suggests that cash programs do not generate more spending on

temptationgoods, suchasalcohol and tobacco. Nonetheless, in-kindprogramsareoften

“sold” as targeted towomenand children and thus tend tobemorepopular among tax-

payerswhowanttoensurethattheirtaxdollarsarenotwasted,butratherusedto“feed

children” and reduce social ills such as school dropouts and crime (for example, see de

Janvryetal.1991;EppleandRomano2008).

Asecondpotentialreasontofavorinkindtransfersovercashtransfersisthatthey

might have lower de-incentive effects on work, and may in fact spur work. A common

worrywithcashtransfersisthattheycouldprovideadisincentivetowork,particularlyif

householdsworryaboutlosingtheirbenefitsastheirincomerisesabovetheeligibilityline.

Ithasbeenarguedthatin-kindtransfergeneratefewerlabormarketdistortions,andmay

in fact be labor market enhancing if the provided good is a complement to work. For

example,inareaswhereproductivityislowduetonutritionalconstraints,afoodtransfer

programcouldeasethisconstraint.However,theexistingevidencethusfardoesnotimply

thatcashtransfersgreatlyreducelabormarketparticipation(forexample,seeAlzúaetal.

(2013);Banerjeeetal.(2015)),perhapsduetothelongdurationofbenefitsanduncertain

processes for re-certification observed in many developing countries. Furthermore, the

cash transfersmayalsohelpeasecreditconstraints for thoseengaged in theagricultural

sector,increasingtheproductivityofagriculturallabor(Gertleretal.2012).

A third reason in support of in-kind programs is their self-targeting properties

(BesleyandCoate1991;Christian2014):byprovidingagoodthatthepoordifferentially

- 22 -

valuerelativetotherich, thepoorwillapplyandtherichwilloptout.Again, ifmoney is

fullyfungible,richerhouseholdscouldsimplyopt-inandsellthesegoodsforthecash,etc.

However,wemightexpectthattransitioncostsandotherconstraintswouldimplythatin-

kind goods are viewed differently than just pure cash. Note that despite this rationale,

manyin-kindtransferprogramsareindependentlytargetedpriortodistribution,shutting

offachannelthroughwhichanin-kindprogrammaygeneratethesetypesofeffects.And,

whilethenon-experimentalevidencesuggeststhatin-kindprogramsarebetteratselecting

thepoor(seeforexampleJacoby(1997)),thereislittleexperimentalworkthatcompares

the magnitude of its targeting properties against different forms of means-tested cash

programs.

A final argument in favor of in-kind transfers comes from the idea of missing

markets, i.e. if for some reason the market does not provide the good on its own, just

providingcashmaynotbeenough.Thismaybeaparticular issueinremoteareaswhere

hightransportcostsdiscouragethespreadofproductsorindisasterorwarzones,where

foodandotherproductsmaybeinshortsupply.

2.EvaluatingIn-KindPrograms

One can evaluate an in-kind transfer program in a manner similar to evaluating a cash

program, i.e. randomize some areas to receive in-kind transfers (treatment group) and

otherstonot(controlgroup).Thenonecancollectdatatounderstandifpoorhouseholds

were indeed properly targeted and whether or not they were able to access their

entitlementorsubsidizedproducts.

However, unlike pure redistributive programs, onemay also want to understand

whether the theoretical behavioral constraint actually exists, as well as whether the

behavioral change thatoneaimed to inducehasoccurred. As such, thereare twoother

designfeaturestoconsider.First,whileanevaluationofaredistributiveprogramwouldbe

focused on collecting variables to determine whether or not the redistribution has

successfully occurred, an evaluation of an in-kind program would additionally require

collectingvariablestotestforthehypothesizedbehavioralchanges.

- 23 -

Forexample, in evaluatinga food transferprogram, if thegoal is to increase food

consumption forchildren,onewouldcareaboutcollectingdetailedmoduleson foodand

caloriesconsumed,aswellashealthindicatorsforchildren.However,notetwoimportant

caveatsofthisdatacollectionprocess.First,asSchadyetal.(2008)pointout,peoplemay

hesitate to provide accurate information on surveys if they believe that the surveys are

connected to re-verification for the program. In which case, one may want to collect

variables that are easier to verify: assets that one can observe, vaccination records on a

card,bodymassindex(BMI)andotheranthropometricvariablestoassessthehealthstatus

ofchildren,etc.Second,asthetransferprogrammaybedesignedtoaddressanumberof

behavioralchanges,oneworryisthatincollectingmanydifferentvariables,wewouldfind

somesignificant impacts justbychance.Thus,onemightwanttopre-specifysomeofthe

keyhypothesesandoutcomevariablesinadvance(Migueletal.2014;Olken2015).

Second, and perhaps more important, one may also want to consider evaluation

strategiesthatisolatethebehavioralconstraint.ThebasicRCTdescribedabovecomparing

theeffectofaspecificprogramagainstcontrolareasthatdonotreceiveanyassistanceis

importantforunderstandingtotalprogrameffects,butitdoesnottellushowtheparticular

constraint(e.g.providingriceversuscash)affectstheobservedoutcomes.Understanding

the relevance of specific behavioral constraints can be important, if they imply specific

changestoprogramdesign.

Therefore, comparing in-kind transfer programs to cash transfer programs can

provide useful insights into whether or not constraining behavior is welfare improving.

For example,Aker (2014) explores the effect of cash versus food vouchers for displaced

households living in an informal camp in the Congo, and shows that the level of food

consumptionwasthesameregardlessofthemechanismsincevoucherhouseholdssimply

boughtfoodthatisrelativelyeasytosell(e.g.salt).Similarly,theMexicangovernmenttook

advantageofmultiple treatment arms tomeasureboth the effect of in-kindprograms to

cash,andtodoingnothingatall. Specifically,theycomparedthreetreatments:(1)anin-

kindtransferprogramthatgavehouseholds10differentitemsoffood(2)acashtransfer

- 24 -

intended to be of equivalent value (3) a control group that did not receive transfers.18

Whilethein-kindprogramdistortedconsumptionofsomeindividualtypesoffood(Cunha,

2012),overall foodconsumptionwassimilaracrossbothprograms(Skoufiasetal.2013)

andtherewasnodifferenceinobservedweightamongwomen(Leroyetal.2013).

Otherstudieshavealsocollectedvaluabledatathathighlightthetradeoffsbetween

programsthatmaysatisfyasociety’sspecificgoalthatanin-kindprogrammaybedesigned

to address (e.g. greater food consumption for kids) versus other important social goals.

Hidroboet al. (2014) compare cash, food, foodvouchersanda control group inEcuador

andfindthatallthreetypesofprogramsimprovepercapitafoodconsumptionandcaloric

intake.However,while food and food vouchers increase calories and diet diversity a bit

morerelativetocash, thecashprogramismucheasierandcheaperto implement,which

maybeofrealconcernforcountrieswithweakerinstitutionalquality.19

Similarly,Hoddinottetal. (2014)comparecashversusfoodtransfers inNigerand

find that the food transfer program had a larger effect on food consumption and diet

variety thancash.However,householdswerenotwastingthe fundsontemptationgoods

such as alcohol; theyused cash to invest in greater agricultural inputs.Thus, using their

estimates, one can think about how tomodel the tradeoff between the additional utility

householdsreceivefromspendingastheychoose,relativetosociety’sutilityfromtheshift

infoodconsumption.

In short, the theory suggested that food, cash and vouchers could have different

effects onhousehold outcomes, but the existing evidencemostly shows that this didnot

materialize in practice—likely because the particular items of food distributed in these

contextswereinfra-marginal.

B.AddingConditionstoIncentivizeBehavior

18Notethataparticularchallengeisensuringthatthein-kindtransferisequivalenttothecashtransfer.Forexample, in thePALprogram, thevalueof the food transferwas30percentmore than thecash treatment,sincethefoodbasketwasbasedonwholesalepricestothegovernmentratherthanthepricesthatconsumerpay.SeeCunha,2012foradiscussionofhowtomakethemex-postequivalent.19 In an interesting follow-up paper to this experiment, Hidrobo, Peterman and Heise (2013) shows that transfers (irrespective of whether food, cash or vouchers) reduce intimate partner violence.

- 25 -

Since the 1990s, it has been increasingly common formanydeveloping country transfer

programs to layer a level of “conditionality” onto redistributive transfer programs, e.g.

requiringchildrentogotoschoolandgethealthcheckupsforthefamilytoreceiveitsfull

transfer. The conditions usually aim to correct an underinvestment in the family’s

wellbeingthatstemsfromsomesortofmarketfailurewithinthehousehold—parentsnot

internalizingtheirchildren’sfullreturnstoschool,thefamilynotinternalizingthebenefits

ofvaccinationforsociety,etc.

Conditional cash transfer programs (CCT) worldwide have been studied by

randomized control trials—from Mexico’s Progressa/Oportunidades to the Philippines’

Pantawid Pamilyang Pilipino Program (PPPP)—showing important effects. Aswith the

otherprogramswehavediscussed,onecanevaluateCCTsbysimplyrandomizingareasto

receiveornot receive theCCTand thenstudying theoutcomes; this tellsus the reduced

formeffectofhavingtheprogramrelativetonoprogram.Indoingso,manyoftheissues

thatwehaveraisedabove—whatdatatocollectandwhen,howtotarget,etc.—wouldthus

beimportanttothinkaboutinthiscontextaswell.

However,giventheuniquenatureoftheCCTinattemptingtocorrectaninvestment

failure within the family—in addition to redistributing to poor households—more

sophisticated analysis can be done to better understand the role of the conditions on

householdbehavior.Below,wediscussthreekeyquestionsthatexperimentationcanhelp

shed light on: (1) What is the impact of the conditions relative to basic redistributive

programs?;(2)Whatshouldtheconditionsbe?;andfinally,(3)Howdoweenforcethem?

1.EvaluatingConditionsrelativetoBasicRedistributivePrograms

Evaluationsthattestaconditionaltransferversusnoprogramhavedifficultyteasingapart

the conditionality mechanism from the liquidity or income effect of the transfer itself.

Whether to includeconditions inacash transferprogram isanessentialdesigndecision.

Conditions require a budget to fund them and staff to enforce them. They may also

generateselectioneffectsonwhochoosestoparticipate.

Toisolatetheeffectsoftheconditionality,asimpleexperimentaldesignwouldhave

onetreatmentwithaconditionalcashtransfer,asecondtreatmentwithunconditionalcash

- 26 -

transfers,andacontrol.Thusthesecondtreatmentgroupgeneratesapureincomeeffect,

whichallowsonetolearnwhethertheconditionalitychangesbehavior,orwhethertheCCT

changedbehaviorsimplythroughitsincomeeffect.

ThisistheapproachtakenbyBaird,McIntoshandÖzler(2011)intheZombaCash

TransferPrograminMalawi.TheCCTsperformedbetteratinducingthedesiredbehavior:

schooldropoutrateswerelowerintheCCTarmthantheUCTarmandpersistedbeyond

the program’s end. Moreover, attendance rates improved and cognitive ability,

mathematics, andEnglish reading comprehension test scores increased for theCCTarm,

butnottheUCTarm.Thus,theconditionshadeffectsonhouseholdsaboveandbeyondjust

providing cash. However, the UCT arm had significantly lowermarriage and pregnancy

rates. This initially may feel like a counter-intuitive result since schooling is thought to

postpone marriage and lower pregnancy rates. However, the UCT is provided to

householdseveniftheteenagegirlsdonotattendschool,whereasonlythosewhoattend

schoolreceivetheCCT.Thus,theeffectonmarriagewilldependlargelyontherelativesize

oftwogroups,thegroupofhouseholdsthatdoesnotattendschoolundereithertheUCTor

CCT,comparedtothegroupofhouseholdsthatattendsschoolundertheCCTbutnotthe

UCT. If the income effect onmarriage and pregnancy in the first group is large enough

(Duflo,Dupas,andKremer2010;Ferre2009;Osili andLong2008;Ozier2011), theUCT

willleadto,onnet,lowermarriageandpregnancyrates.TheauthorsoftheMalawistudy

found this tobe true.Thisdynamic illustrates that theconditions themselvescanreduce

redistributive aid to thepoor—andanyassociatedbenefits of the aid—bychangingwho

participates in the program. And, it reinforces the need for careful data collection on

ancillaryoutcomes—e.g.inthiscase,marriageandpregnancy—inordertounderstandthe

full setofmechanisms throughwhichaprogramworks so thatpolicy-makers canbetter

understand the tradeoffs that they would make by implementing one program over

another.

Further insights on the tradeoff between a UCT and a CCT can come through

examiningwhat beneficiarieswould choose, given a choice.Most programs naturally do

notprovidesuchachoice,butprovidingachoiceinanexperimentalcontextcanhelpoffer

insightsintowhichprogramswouldyieldhigherutilitytocitizens.Forexample,individuals

- 27 -

couldusetheconditionsintheCCTtogenerateapersonalorfamilycommitmentdeviceto

engageinfuturebehavior.Ifrespondentsweretoopt-intosuchaCCToveraUCT,thisis

strongevidenceofeitherindividualdemandforacommitmentdevice(Ashraf,Karlan,and

Yin 2006; Bryan, Karlan, and Nelson 2010) or of demand due to family conflict over

educationor healthdecisions. Similarly, inBrazil, researchers examinedhouseholds that

weregivenachoicebetweenaUCTandaCCTonschoolattendance,andalsotestedasub-

treatment within the UCT in which households were informed of whether the children

wereattendingschoolornot(BursztynandCoffman2012).Theparentsexhibitedastrong

preference for the CCT, unless the UCT included monitoring of their children’s school

attendance, in which case they were content with the UCT. These preferences lend

important insight into the underlying mechanisms of the CCT, and suggest that the

conditionalityinthiscontextwassimplyatoolforparentstobettermonitorchildren.Thus,

interventionsthatimprovemonitoringandcommunicationsbetweenschoolsandparents

maybeabettersolutionthanthecomplicatednatureofCCTsoverUCTs.

2.WhichConditionsshouldbeImposed?

Policy-makershavetomakechoicesaboutwhatconditionstoimpose: addingconditions

adds additional costs of monitoring and as we discuss above can have important

implications onwho participates in programs, potentially screening out the very people

whom you want to reach. Thus, experimentation can also help along this dimension,

helpingtodeterminewhichconditionsaremoreimpactfulandworthfocusingon.

Indoingso,itismorecomplicatedthanjustchoosingtoconditiononasinglefactor,

say“schooling.” Thestructureisalsoimportant:conditionscanbeimposedoninputsor

activities(e.g.attendanceofchildren)oroutcomes(e.g.testscores)orboth.Andthen,the

payment structuremust alsobedefined.Forexample,Barrera-Osorioet al. (2011) show

thatwhetheryousimplyconditionamonthlypaymentbyattendance,orholdbackpartof

the payment and provide it only if the child re-enrolls in school, can affect schooling

outcomes.

Another type of structure provides conditions for participation, but does not

financiallypenalizehouseholds if theydonotmeetthem. Thus, thisstructureessentially

- 28 -

provideshouseholdswitha“nudge.” Forexample,comparingaCCTwithacashtransfer

program thatwas “labeled” for schooling (LCT),Benhassineetal. (2015) show thatboth

types of programs improved school participation relative to the control group, but they

were not statistically different from each other. The conditionality costs more to

implement,andsowasmoreexpensiverelativetotheLCT.

Aswediscussedabove,applyingconditionsmayleadtoatradeoffbetweenthegoals

oftheconditionsandtheinitialgoalofredistribution.Onelegitimateworryisthatcertain

typesofconditionsmaydiscouragepoorhouseholdsfromapplyingsincetheyfindsomeof

conditionstoooneroustocomplywith,thusdiminishingtheultimateredistributivegoalof

theseprograms.Forexample,inareaswithstrongculturalbeliefsagainstvaccines,would

requiring vaccines for children reduce the probability that the poorest households

participate in the program? One can imagine extensions of the CCT literature with

randomizationnotonlyforwhethertheprogramhasconditions,butalsoforthetypesof

conditions in different areas—measuring not only the effect on recipients, but also the

effect onwho applies for the programs and howwell the implementers can realistically

enforcethem.

3.EnforcingtheConditions

Theenforcementoftheconditionalityiscriticaltoexamine.Withouttheenforcement,the

program isperhaps theoretically equivalent to anunconditional cash transfer. Politics at

the policy level, and corruption at the implementation level, both can lead to de facto

removal of the conditionality. This occurred, for example, in Ecuador with the Bonode

DesarrolloHumano(BDH)program.Anevaluationoftheprogramthusanalyzestheresults

asiftheprogramisanunconditionalcashtransfer(FernaldandHidrobo2011;Paxsonand

Schady 2010). However, a CCT program that fails to enforcemay ultimately generate a

differentbehavioralresponsethanjustapureUCT.Forexample,intheEcuadorcase,many

households believed that the fundswere indeed conditional, even though in reality they

werenot.

Enforcement of conditions is an area worthy of further research, but it is not

obvious this is appropriate randomized trial territory. While one could randomize the

- 29 -

enforcementof theprogram, in an approach similar towhatOlken (2007)uses for road

building,thepoliticalenvironmentthatledagovernmenttofailtoenforcetheconditions

may matter, and it may be that merely randomizing enforcement in one setting does

provide insights to settings in which enforcement is not viable for political or social

reasons.Thereasonforlackofenforcementisimportant,andcannotbesimulatedmerely

through randomization. For example, lack of viability of enforcement couldbedrivenby

constituentexpectations,localsocialnorms(whichbothdrivethelevelofenforcementand

the treatment effect of the program), or simultaneous policies that interact with the

treatmenteffectsofthetransferprogram.

However,anunenforcedconditionmaybesimilartoasuggestionora“nudge.”For

example, an unenforced condition of school attendance may serve a similar role as the

governmentmerelylabellingatransferasan“educationsupportprogram.”Benhassineet

al.(2015)examinesthisquestionbydesigningatwo-prongedexperiment:conditionalcash

transfer versus labelled cash transfer (those two prongswere crossedwith a household

structure test,providing theprogramtomothersversus fathers). Inorder tounderstand

theunderlyingmechanisms, thestudycollecteddataonprocesschangesandalsouseda

multi-armed experimental design. For example, to understand if the conditionality (or

labelling) leads to changes by signaling information about returns to education,

researcherscollecteddataonparentalbeliefsaboutreturnstoeducation(nochangewas

found).Attendance(conditionalonenrollment)increasedafterCCTsandLCTs, leadingto

an increase in time spent studying, at school, and traveling to school and a decrease in

leisure and productive labor (but not a decrease in chores); this suggests the barrier to

schoolingwasduemoretoalackofstudentinterest(thusdrawingasimilarconclusionon

mechanisms as the Brazilian study above (Bursztyn and Coffman 2012). The multi-arm

experimentaldesignthentestedexplicitly themarginalbenefitof thecondition,overand

beyondamerelylabeledtransfer,andthestudyfoundnoadditiveeffectbeyondthelabel.

V. MarketFailuresPreventingAssetAccumulationMost of the issues we discussed thus far focus on providing short-term relief through

redistributive aid or insurance to households, with some programs paying attention to

- 30 -

ensuring that themoney is spentongoodsand services thatbenefithouseholds inways

beyondanincreaseincashorliquidity.However,certainformsofanti-povertyprograms

alsoaimtoaddressunderlyingmechanismsthatmaybecreatingpovertytrapsinorderto

improvelong-termincomeforthepoor,ortoprovidemechanismsforindividualstobuild

long-termfinancialassetsforwhentheyareelderly.Theseare, indeed,morecomplicated

challenges: if the underlying market frictions are beyond credit and savings market

constraints,thenthesolutionwillrequiremorethanredistribution.Ifredistributionalone

isemployed,itmayprovideimportantshort-runbenefits,butmayultimatelyactasmoreof

a band aid on immediate symptoms without helping individuals achieve a sustained

increaseinincome.

A.BuildingProductiveAssets

Forthepastthirtyyears,microcredithasbeenaleadingdevelopmentpolicyinthefightto

reduce poverty. Unfortunately, seven recent randomized trials have shown that

microcredit,20while it does provide important benefits, does not improve long term

income,onaverage,forparticipantsinitscurrentform(forareview,seeBanerjee,Karlan,

and Zinman2015; the seven randomized trials areAngelucci, Karlan, and Zinman2015;

Attanasioetal.2015;Augsburgetal.2015;Banerjeeetal.2015;Créponetal.2015;Karlan

and Zinman 2011; Tarozzi, Desai, and Johnson 2015). In addition, Meager (2015)

aggregates themicrodata across these studies, andusesBayesianhierarchicalmodels to

show that theeffectofmicrocreditonhouseholdprofits is likelyverysmall and that the

effects from each individual study site are reasonably informative for each other. This

suggests thateithercredit constraintsarenotdrivingstagnatedgrowth for thepoor, the

current designs of microcredit programs do not fully address credit constraints, or

microcreditmaynotworkwithoutchangesinotherconditionsthatalsogeneratemarket

failuresforthepoor.Multi-sitestudiesprovidetremendousopportunitiesforsuchanalysis,

e.g. through building more robust theories and then examining, using appropriate

statisticaltools,howwellresultsfrommultiplesitesfitbroadtheoreticalframeworks.

20 Note that this lesson was only learned by running similar experiments across different project sites and countries, showing how valuable replication studies can be in changing perspective.

- 31 -

Indeed,thepoordofacemultiplefailuresatthesametimethatmayhinderlong-run

investmentandincomegrowth.But,wetypicallyobserveprogramstacklingoneproblem

at a time, rather than a coordinated system. Poverty alleviation policy, as with many

government programs, often operates in silos. One silo, described above and typically

managed under the umbrella of “social protection,” focuses on redistribution policies

through either conditional or unconditional cash transfers.A second silo, oftenmanaged

under the ministries of trade or agriculture, focuses on livelihood support, such as the

transferoraproductiveassetoragricultural input, alongside some training.A third silo,

financial inclusion, is often administered through for-profit or nonprofit (and sometimes

subsidized)financialinstitutions.But,naturally,thecausesofpovertymaybemultifaceted.

Thus, uncoordinated programs across different ministries may fail to provide the right

bundleofinterventionsthatahouseholdwouldneedtoimprovetheirlivingstandard.This

lack of coordination in itself poses an interesting research agenda to understand the

impactsofthesemulti-approachprograms.

Onerecentexampleofsuchaprogramisthe“Graduation”approach—anintegrated,

multi-faceted program with livelihood promotion at its core that aims to “graduate”

individualsoutofextremepovertyandontoalong-term,sustainablehigherconsumption

path.BRAC,theworld’slargestnongovernmentalorganization,hasscaled-upthisprogram

inBangladesh(Bandieraetal.2016),whileNGOsaroundtheworldhaveengagedinsimilar

livelihood-based efforts. Six randomized trials across the world (Ethiopia, Ghana,

Honduras,India,Pakistan,andPeru)foundthattheintegratedmulti-facetedprogramwas

“sufficient”to increase long-termincome,where long-termisdefinedasthreeyearsafter

theproductiveasset transferBanerjeeetal. (2015).Theresults fromthepooledanalysis

acrossallsixcountriesfoundthattheprogramledtosustainableandsignificantimpactsin

10 out of 10 categories of impact. Using an index approach to account for multiple

hypotheses testing, positive impacts were found for consumption, income and revenue,

assetwealth,foodsecurity,financialinclusion,physicalhealth,mentalhealth,laborsupply,

politicalinvolvementandwomen’sdecision-makingaftertwoyears.Afterathirdyear,the

resultsremainedthesamein8outof10outcomecategories(withpointestimatesfalling

to below statistical significance for physical health and women’s empowerment).

- 32 -

Furthermore,theWestBengalsitefoundevenlargertreatmenteffectsaftersevenyears(A.

Banerjeeetal.2016).ThepatternofresultsarestrikinglysimilartotheBangladeshstudy

(Bandieraetal.2016).

Theseresultsarepromisinginthattheyshowthatasufficientsetofinterventionsis

capableofalleviatingpovertysustainablyandare thus important forpolicy.Theyshould

whettheappetite,bothforamoretheoreticallygroundedunderstandingofexactlywhich

marketfailuresledtoapovertytrap,aswellasamorepracticallygroundedunderstanding

ofwhetheralloftheinterventionsweretrulynecessaryorifcertaincomponentscouldbe

removed. In the event that some components are unnecessary, costs could be lowered

considerably,allowingtheprogramtoreachmorepeopleusingthesamebudget.Returning

to the theme of this paper, there are two complementarymethods to tackle testing the

importantmechanismsbehindthetheory,andsuccessor failure,of theseprograms:data

andexperimentaldesign.

The idealmethod, if unconstrained by budget and organizational constraints, is a

complex experimental design that randomizes all permutations of each component. The

productive asset transfer, if the only issuewere a creditmarket failure,may have been

sufficient to generate these results, and if no other component enabled an individual to

accumulate sufficient capital to acquire the asset, the transfer alone may have been a

necessary component. The savings component on the other hand may have been a

substitute for the productive asset transfer, by lowering transaction costs to save and

serving as a behavioral intervention which facilitated staying on task to accumulate

savings.Clearly it isnotrealistic inonesettingtotestthenecessityorsufficiencyofeach

component, and interaction across components: Even if treated simplistically with each

componenteitherpresentornot,thiswouldimply2x2x2x2=16experimentalgroups.

Data can also provide important insights, even absent experimental design

variation. Take the savings component, for example. For the savings component to be

eitheranecessaryorsufficientcomponent,presumablyanincreaseintheflowofsavings

must be observed (but not necessarily the stock, since withdrawals for investment

purposesmay bring the stock back down). The evidence from the Graduation programs

showswidelyvaryingimpactsonsavings, farmorethantheresultsoftheprogramitself.

- 33 -

Forexample,inthemostextremecase,savingsincreasedinEthiopiabyPurchasingPower

Parity(PPP)US$70721, comparedtoonlyPPPUS$17 inGhana.Thissuggests thatsavings

maybean importantcomponent,but isneitheranecessarynorsufficientcomponent for

somelevelofsuccess.

Several studies have tackled pieces of the puzzle, and theway forward is clearly

going to be the development of amosaic, rather than any onedefinitive study that both

testseachcomponentandalsoincludessufficientcontextualandmarketvariationsthatit

can help set policy for a myriad of countries and populations. For example, in a post-

conflict setting in Uganda, a NGO-led program provided youth groupswith training and

cash (US$150) towards non-agricultural self-employment activity and found a 57%

increaseinbusinessassets,17%increaseinworkhours,and38%increaseinearningsfour

yearsafterthecashgrants(Blattman,Fiala,andMartinez2014).Thisprogramdiffersfrom

theabovementionedGraduationprogramsinthreepotentiallyimportantandilluminating

dimensions:post-conflictversusnonpost-conflict,youthversusgeneralpopulationofthe

extreme poor, group-level intervention versus household-level, and no inclusion of

ancillary components such as life coaching, savings, and health care. The first two

differences speak to the applicability of the program to alternative sample frames and

settings,whereasthethirdandfourthprogramvariations,suggestthateitherthedriverof

theimpactoftheprogramlieswiththecashgrantsandtrainingnottheothercomponents,

orthatthegroup-levelaspectimprovestheimpactandeffectivelysubstitutesfortheother

components.

A second study in Uganda sheds insight into the value of the group-level

intervention, as it randomly varies the group aspect of the intervention, as well as the

intensityofsupervision(Blattmanetal.2014).Theseprograms,aswiththeaboveUganda

program,differfromtheGraduationstudiesinthattheydonotincludesavings,healthand

lifecoachingcomponents,andarefocusedonenterprisedevelopment(ratherthananimal

husbandry,thedominantlivelihoodintheGraduationstudies).

21AlthoughnotethattheEthiopiaprogramdesignincludedamuchstrongerpushforsavingsthantheotherprograms, with savings put forward as almost a “mandatory” component, even though there was noconsequenceifhouseholdsdidnotsave.

- 34 -

Thus, the initial studies above have established a base case, that there exists a

sufficientinterventionpackagethatincreaseslong-termincome.Wehighlightfourlinesof

inquirytounderstandmoreabouttheunderlyingmechanisms.First,long-termimpactsare

critical for assessing whether the short-run interventions actually addressed the

underlyingproblems,orrather just lastedabit longer thanacash transfer.Forexample,

Graduation programs typically last two years while the Graduation studies cited above

measured impacts three years after the assets were transferred. If the household visits

were a critical component in driving the observed impacts, longer-term measurement

wouldbeimportant,toassesswhetherthebehavioralchangesmotivatedbythehousehold

visitspersistedformorethanjustoneyearafterthehouseholdvisitsceased.

Second, as some of the studies above have begun to do,morework is needed to

tease apart the different components: asset transfer (addresses capitalmarket failures),

savings account (lowers savings transaction fee), information (addresses information

failures), life-coaching (addresses behavioral constraints, and perhaps changes

expectations and beliefs about possible return on investment), health services and

information(addresseshealthmarketfailures),consumptionsupport(addressesnutrition-

basedpovertytraps),etc.Therewillbenosimpleanswertotheabovequeries,butfurther

work can help isolate the conditions under which each of these components should be

deemednecessary to address.And furthermore, for several of thesequestions, there are

keyopenissuesforhowtoaddressthem;forexample,life-coachingcantakeonaninfinite

number of manifestations. Some organizations conduct life-coaching through religion,

othersthroughinteractiveproblem-solving,andothersthroughpsychotherapyapproaches

(Boltonetal.2003;Boltonetal.2007;Pateletal.2010).Muchremainstobelearnednot

justaboutthepromiseofsuch life-coachingcomponents,buthowtomakethemwork(if

theyworkatall).

Third, general equilibrium effects should be considered, particularly as the

programsaretakentoscale.Here,thefirsttaskistobemorespecificindatacollection,as

general equilibriumeffects encompass awide varietyof indirect effects, such aspriceof

transferred assets; spillovers from explicit sharing of granted resources; and increased

economicactivityfromincreasingthepoor’swealth.Atypicalexperimentaldesignwould

- 35 -

eitherrandomizeacrossandwithinvillages(assumingthatthevillageistheboundaryfor

generating general equilibrium effects), or for some issues, examining spillovers to non-

participantsintreatmentversuscontrol(asinAngelucciandDeGiorgi2009).

Fourth,importantlessonscanbelearnedfromunderstandingtheconsumptionpath

taken by households after participating in these programs. The Graduation program for

example found important and cost-effective, but still modest, increases in long-term

consumption.Thisfindingsuggeststhathouseholdsarenotcaughtinanextremepoverty

trap, where one simply needs to get households over a particular hump and they will

immediately converge to the equivalent of the middle class. Further work is needed to

understand the long-term dynamics of such programs andwhat can be done to further

increaseincomemobility.

B.BuildingLong-TermFinancialAssets-Pensions

Non-contributory pension programs are an important form of social protection inmany

developing countries—such as Brazil, South Africa, India, etc.—, and given the shift in

demographics and the risk of poverty for the elderly (UNDESA 2013) they are likely to

growinimportance.Theprogramsvaryinshapeandform:Rofmanetal.(2015)compare

pension programs across 14 different Latin American countries, showing differences in

paymentsizes, in timingofpayment, inwhether thepensionsare targeted,etc. While to

the best of our knowledge there are few experimental studies of pension programs in

developing countries,22RCTs can help us understand how differences in these design

choices affect the labor market choices of working-age adults, retirement age, saving

patterns,andhowfundsareusedwithinthehousehold.

VI. IdeasOnlyGoSoFar:ImplementationMattersTooAtransferprogrammaylooklikeawinneronpaper,butmaybeatotalflopinpracticeif

theimplementationishaphazard.Someofthismaybepurelyadministrative,e.g.,ensuring

that therightnumberof staff ishired,and that theyareproperly trainedandmotivated.

22Althoughasofthetimeofthischapter,thereareseveralexcitingongoingstudiesinChileandIndia.

- 36 -

This may require incentives to not shirk on the job, as well as to not engage in bad

behaviors, e.g. siphon off funds or food, or reallocate the funds to friends or political

supportersratherthanthosewhoaremostinneed.

Therefore,indesigningrandomizedevaluationsofanti-povertyprograms,itisalso

importanttothinkaboutwhether,theoretically,aparticularaspectoftheimplementation

is likely tobeparticularlyvulnerable toproblems.Thereare two typesofvariationsone

can think about: (1) evaluations that vary the underlying structure of the program (2)

evaluations that layer on complementary actions that can be undertaken to improve

programimplementationgivenafixedprogramdesign.

Experimentallyvaryingthecoreelementsoftheunderlyingstructureofaprogram

is challenging, especially if aspectsof theprogramhavebeenwritten into law.However,

thedetailsoftheunderlyingstructure—fromwhoshouldimplementtheprogramtohow

oneshouldmakethetransfers—maymattertremendously,affectingthe levelof leakages

and corruption, the targeting, the costs for beneficiaries to access the program, and

potentially how beneficiaries spend their entitlements. For example, there is extensive

work, in general, exploring how officials’ incentives affect theirwork output, but to our

knowledgelessworkonhowtheincentivesprovidedtoofficialsaffectstransferprogram

delivery.Similarly,thereisastrainofresearchthatshowsthatthetypeofpersonrecruited

mayaffectgovernmentefficiency(see,forexample,recentempiricalevidencefromAshraf

etal.2014;HannaandWang2014),butlessspecificallyonhowchangingwhoisselectedto

implement transfer programs affects the ultimate outcomes of households. For future

research, one could vary the salary and incentive structure (amount and conditions) for

currentworkers—andduring the recruitmentofnewworkers—toexplorehow itaffects

targetinganddelivery.

One important question is whether governments should even directly implement

theseprograms,orwhethertheyshouldcontractoutdeliverymechanisms.Banerjeeetal.

(2014)experimentallyvarywhetherlocalofficialsdistributeagovernment-runsubsidized

riceprogramorwhetherprivate citizensalsobid for the right to run theprogram.They

findthatthebiddingreducestheprice-markupthatcitizenspay,withoutreducingquality.

Follow-upwork can include testing out differentways of contracting out, from changing

- 37 -

whoiseligibletobidtohowthebiddingprocessoccurstohownewimplementersarere-

evaluated. Moreover, the bidding process in that paper focuses on local government

provision(i.e.atthevillagelevel). Futureresearchcouldalsohelpshedlightonwhether

theprocurementprocessshouldbedoneatthatlocallevelwhereindividualspossesslocal

informationabouthowtogetthingsdoneinthatvillage,orshould itbedoneatahigher

level of government (e.g. district or province level) where one may also benefit from

economiesofscale.

Aniceseriesofrecentpaperstestswhetherthenatureofthedeliverymechanismin

itselfaffectsoutcomes.Forexample,Akeretal.(2011)experimentallytestfortheimpactof

providing cash versus mobile money in a short-run transfer program in Niger. An

innovative feature of their experimental designwas to also have a treatment group that

simplygotcashandacellphone,tonetoutpotentialeffectsofahavingacellphonemore

generallyfrommobilemoney.Mobilemoneynotonlyreducedthenon-profit’sdistribution

costs, but it also reduced thehouseholds’ costs topickup their entitlement.This second

feature of mobile money many be particularly important if we believe high transaction

costsinducebeneficiaryhouseholds“leavemoney”onthetable(CurrieandGahvari2008).

Importantly, they also showed that spending patterns changed due to mobile money,

hypothesizing that it also conferred greater privacy over one’s finances. Further testing

thesemechanismsinthecontextoflargergovernmentprogramstounderstandlonger-run

effectswould be an important extension of thiswork: For example,would the ease and

potential secrecy of payments attract richer people to apply for these types of transfer

programs?Inthelong-run,wouldlocalofficialswhomayhavepreviouslysiphonedoffcash

duringdisbursements findotherwaysto“tax”citizenswhonowreceivecashdirectlyvia

mobilemoney?

AnambitiousprojectbyMuralidharanetal. (2014) alsoaims toaddress someof

these types of questions. They evaluated the impact of biometrically-authenticated

payments infrastructure ("Smartcards") on beneficiaries of employment (NREGS) and

pension (SSP) programs in Andhra Pradesh, India. The smart cards changed both how

householdscollectedtheirpayments,aswellaswhowasinchargeofthecashdistribution

(asbanksandtechnologyserviceprovidersmanagedthenewcashdisbursalsystem).The

- 38 -

statewasrollingouttheprogramacross its158sub-districts,sotheauthorsrandomized

which sub-districtswere converted first. Following the introduction of the program, not

onlydidthetimeit tookbeneficiariestocollectapayment fall,butthedelay inreceiving

thepaymentwasreducedbyalmost30percent.Theeaseofpaymentinducedhouseholds

to actually work more. Households, thus, earned more, while payments to officials

remained the same—hence, leakages fell quitedramatically. Similarly, anRCT conducted

by Banerjee et al. (2014) shows that by simply asking local officials to input all of the

namesofthepeoplewhoparticipatedinNREGAintoadatabasesysteminordertoreceive

the funds transfer—i.e. increasing informational requirements for releasing funds and

reducing the administrative tiers in the flow of funds process—led to a stark decline in

leakagesofpublictransfers,withnocorrespondingdecreaseinactualNREGAwork.

Experimentallytestingcomplementaryprogramsthatarelayeredontopofexisting

programscanalsobe important in improving thedeliveryof socialprotectionprograms.

These programs do not necessarily require changing the existing program rules or

functioning,but insteadprovideadditional informationorservices tohelpcitizensbetter

access their entitlements. For example, one could test how increased information on

eligibility andprogram rules affects overall program leakages. Ravallion et al. (2013) do

this: they experimentally vary whether beneficiaries see a half hour video on their

entitlementsunderNREGA.Theyshowthatthisformofinformationhasverylittleimpact

onemployment.Giventhattheformandlevelofinformationmaymatter,onemayalsotest

betweenvaryingtypesofinformation:forexample,Banerjeeetal.(2014)showthatacard

that informs households of their eligible status and entitlements reduces leakages in a

subsidizedriceprogram,andthatmakingthecardinformationpublicwithinthevillagehas

even larger impacts. Moreover, one can imagine experiments designed to test how

providing householdswith direct helpwith their paperworkwhen applying affectswho

enrolls.

Questionsabout implementationatscalealsorelatetocriticalquestionsaboutthe

roleofrandomizedtrialsgivenhowtheyareoftenconductedindevelopingcountries.For

example, there is a broader debate about whether we would observe similar program

results in NGO and government settings, given differences in implementation capacity

- 39 -

between two (see, forexampleBoldetal.2013;DhaliwalandHanna2014).Of course, if

oneisevaluatingaprogramwithanNGOthatwillbescaledupbythatNGOorsimilarones,

wemaynotparticularlycareiftheprogramwouldlookdifferentifrunbythegovernment.

However,oftentimeswemayalsowanttounderstandhowevaluationswithNGOswould

differ in government and vice versa. Naturally, the fundamental problem here is of

generalizabilityandsamplesize:acomparisonofanyoneNGOtoanyonegovernmentonly

comparesthatNGOtothatgovernment.NGOsarenotamonolithicsetofinstitutions,and

neitheraregovernments.Thusitmaybewrongtoaskwhetheragovernmentisbetter,or

worse, at implementing than an NGO, and be more appropriate to ask whether “an”

institution with certain specific characteristics or in certain specific cultural or political

environments will be better at implementing than an institution with a different set of

specificcharacteristicsorenvironmentalfactors.

Treatmenteffectsmaydependon institutional type (governmentorNGO) for two

broadreasons:behavioralresponsesandimplementationefficiency.Intermsofbehavioral

responses,treatmenteffectsmaydependuponthelegitimacyofwhodeliverstheprogram.

Inthespecificcaseofsocialprotectionprograms,howweexpecthouseholdstorespondto

aparticular transferof foodorcash isunlikely tochangebasedonwho isdistributing it.

But,howpeoplerespondtothespecificsof theprogrammaymatter.Forexample, inthe

studyinMoroccowithlabeledcashtransfers(Benhassineetal.2015),itcouldbethatsuch

labelsonlywork from trustedandwell-known institutions.Again, this is lessof an issue

overwhethertheNGOorgovernmentisdeliveringtheservice,butabouttheoveralllevel

oflegitimacyoftheinstitution.Thus,oneinterestingdesignwouldbetoseeiftheresponse

to the nudges changes when households are randomized to receive more or less

informationonhowlegitimatetheorganizationhasbeeninimplementingtheseprograms

inthepast.

In termsof implementationefficiency,onecanalso imaginethatdifferent typesof

organizations may have different strengths and weaknesses in terms of the types of

programs that they can deliver. Suppose, for example, that citizenswould be indifferent

between cash andan in-kind transfer if bothwere implementedperfectly.However, one

typeoforganizationhasstrengthinreducingtheleakagesinthedeliveryofcashrelativeto

- 40 -

asecondorganization,andthesecondorganizationisbetteratreducingleakagesinthein-

kind transfer. Thus, citizens would ultimately prefer different types of transfers from

different types of organizations due to the relative differences in leakages. Again, this

preference may be symptomatic of a difference between government and an NGO, but

speaks to larger differences in the relative implementation abilities of different

organizationsandhowcanoneimproveupontheirweaknesses.

VII. Conclusion:KeyAreasforFurtherWorkSincetheinnovativeandinstrumentalrandomizedevaluationofProgresainMexico,there

has been a burst of important and exciting experiments in this area. This has greatly

informedour understanding ofwhat can “work” in trying to redistribute to the poor, as

wellaswhatcanreducebothbehavioralconstraintsandmarketfailures.

So,thenthequestionbecomes,whereshouldwefocusourresearcheffortsnext?We

highlight three areas for furtherwork, aside from thosediscussed above: interactions of

demandandsupply,long-termeffects,andgeneralequilibriumeffects.

A.KeyAreasforFurtherWork1.InteractionsofDemandandSupply

A vital question is how transfer programs work across different contexts. For example,

GalianiandMcEwan(2013)documentthattheeffectoftheHonduranPRAFCCTprogram

wasmuchlargerinthetwopooreststrata,withtheeffectnotbeingstatisticallysignificant

inthethreericherareas.

Thisquestionissimilartotheoneatthecenterofthedebateoverwhoimplements

(e.g.anon-profitorgovernment):ifonewantstounderstandhowaprogramwillworkina

specificcontext,wemaynotcarewhethertheevaluationfindingsareportabletoanother

context. But, ifwewant tounderstandwhetheraprogramwouldhavesimilarresults in

another area, or if the results will change with policy changes in the current area, it is

important to understand how the theoretically important underlying features of an area

impactoutcomes.

- 41 -

More broadly, there are lots of unanswered questions about the interaction of

transfer programs with existing conditions, particularly supply-side conditions: for

example, howdoes school quality or health care availability affect the adherence to CCT

conditions?Do foodorother in-kind transfersworkbetter thancash inareaswithmore

limitedfoodsupplies?Dotransfersfacilitateaccesstofinancebyreducingrisktolenders?

Andsoforth.

To answer these kinds of questions, one would ideally not only vary the

introductionofatransferprogram,butwouldalsocrossthiswithanexperimentalchange

in a supply side feature. For example, to isolate how increased health care availability

affects the adherence to CCT health conditions, one would randomize areas to four

treatments:apurecontrol,CCTonly,anincreaseinnursesonly,andCCTandanincreasein

nurses.

Anextensionofthiswouldbetotesttheeffectivenessofdifferenttypesoftransfer

programsunderdifferentconditions:forexample,wemightthinkthataUCTmaybemore

effective at redistributing to the poor than a CCT in areaswhere there is limited health

availability, since the inability to adhere to the conditions may scare off or reduce

payments to the poor. Thus, rather than having just a pure control, one may want to

compare CCTs to UCTs across areas with and without the induced increases in nurse

availability.

2.Long-TermEffects

Thereisatensionbetweentryingtomeasureaprogram’slong-runimpactsversusscaling

upa“working”programtothecontrolgroup.However,long-runimpactsareimportantto

measure, especially if there are reasons to believe that a program’s impactsmay evolve

differentlyastimegoeson(andpotentiallyhavegeneralequilibriumeffectsaswediscuss

below).

For example, whilewe know quite a bit about the short-run impacts of different

targeting methods, we know less about their relative long-run effects. Using quasi-

experimental variation, Camacho and Conover (2011) show that Colombia’s targeting

systemwasmanipulatedovertime,as localofficialsbetter learnedtherulesof thegame.

- 42 -

One can imagine experimentally varying different targeting methods across different

locations,and thenrepeating thesamemethod ineachrespective locationduring there-

certification process, to determine whether the relative efficacy of different methods

changeasbothhouseholdsandofficialslearnthesystemsovertime.

Similarly, there are many questions about the long-run impacts of the transfers

themselves:What happens to households after the transfers are complete? For example,

did CCTs achieve the goal of changing the outcomes the next generation, i.e. did the

childrenwhoattendedschoolforlonger,orhadimprovedtestscores,ultimatelydobetter

inthelabormarket?InthecaseoftheGraduationstudiesdiscussedabove,theBangladesh

andWestBengal,Indiasiteshavefollowedhouseholdsforsevenyears,andfoundthatthe

positivetreatmenteffectsincreasedfromthreetosevenyears(A.Banerjeeetal.2016).

3.GeneralEquilibriumEffects

With relatively large sums being distributed, anti-poverty programs may have broader

effects than one initially expects, with these effects being potentially quite large. These

effects can take various forms. Anti-poverty programs can affect insurance and lending

marketswithinvillages(AngeluccianddeGiorgi2009),naturalresourcedemand(Gertler

etal.2013;HannaandOliva2015)andlabormarkets(Muralidharanetal.2015).Someof

theeffectscouldbepositive,whileotherscanbenegative.Whilewetouchedontheideaof

general equilibrium effects above, it is important enough of a topic to warrant its own

sectionhere.

Importantly, the general equilibrium effects may differ by type of transfers. For

example,Cunhaetal.(2013)documentdifferenteffectsonpricesofconsumergoodswhen

villages have been randomized to cash versus in-kind transfer programs, particularly in

remoteareas.Thegeneralequilibriumeffectsmayalsovaryacrosscontexts:forexample,

CCTs induce positive peer effects on the schooling outcomes of ineligible children in

Mexico’sProgressa(BobonisandFinan2009;LaliveandCattaneo2009),butnoeffectson

ineligible children in the Honduran PRAF (Galiani and McEwan 2013). At the more

extreme,Barrera-Osorioetal.(2011)findnegativespillovers:siblings(particularlysisters)

ofCCTrecipientsarelesslikelytoattendschoolandmorelikelytodropout.

- 43 -

While there have been a few studies, including those discussed above, that have

triedtocapturebroadereffects,thisisstillanareawhereourunderstandingisrelatively

sparse andwhere there is a need for further research. However, given themultitude of

types of general equilibrium effects that may be possible, this is a case where we are

particularly worried about multiple hypothesis testing. Careful theory detailing

predictions,coupledwithpre-specifiedhypotheses,maybeimportantinbuildingarobust

model that successfully predicts outcomes in new settings, thus properly guides

policymaking.

Identifying spillover effects should be built into the research planwhenplausible

andviable.Threebasicapproacheshavebeenemployed:(a)throughexperimentaldesign:

randomizingthedensityoftreatmentwithinageographicarea(orwithinanyunitwithin

whichoneexpectstheretobespilloversorgeneralequilibriumeffects),(b)throughdata

collectiononineligibles:thisisstrengthenedwhencombinedwiththefirst,butevenonits

own can shed important insights (Angelucci and de Giorgi 2009), and (c) through data

collectiononprocesschanges: forexample, collectingspecificdataon informal transfers,

credit and savings could identify behaviors that indicate the presence of general

equilibriumeffects.

B.FinalThoughts

Putting these elements together poses its own challenges. Naturally, there is no simple

diagnostic thatassesseswhichmarketsaremissingforasociety, tothenprovideaneasy

prescriptionforwhichoftheaboveprogramsto implement.And,ontheotherendofthe

spectrum,itissimplynotpracticaltoimplementatscaleaprogramthatassessesforeach

individualwhat constraints they face, and then provides the exact program that targets

theirparticularsituation.Thepolicychallengeliesinhowtofindthepolicythatbalances

theoperational constraintsof scalewith the targeting constraintsofboth identifying the

poorest and minimizing the false positives, i.e., policies targeting an issue that are not

relevantforahouseholdorcommunity.Nationalsocialprotectionstrategiesshouldthink

holistically: how do specific policies interact with each other as either complements or

substitutes,whatpopulationsarecriticallymissingfromexistingpolicies,andhowwelldo

- 44 -

existingpolicies improve long-termoutcomessoastoreducetheeventual taxburdenon

society.

WORKSCITED

Aker,JennyC.2014.“ComparingCashandVoucherTransfersinaHumanitarianContext:EvidencefromtheDemocraticRepublicofCongo.”

Aker,JennyC,RachidBoumnijel,AmandaMcClelland,andNiallTierney.2011.“ZapIttoMe:TheShort-TermImpactsofaMobileCashTransferProgram.”CenterforGlobalDevelopmentWorkingPaperNo.268.

Alatas,Vivi,AbhijitBanerjee,RemaHanna,BenjaminA.Olken,RirinPurnamasari,andMatthewWai-Poi.2013.“DoesEliteCaptureMatter?LocalElitesandTargetedWelfareProgramsinIndonesia.”WorkingPaper18798.NationalBureauofEconomicResearch.http://www.nber.org/papers/w18798.

Alatas,Vivi,AbhijitBanerjee,RemaHanna,BenjaminAOlken,andJuliaTobias.2012.“TargetingthePoor:EvidencefromaFieldExperimentinIndonesia.”AmericanEconomicReview102(4):1206–40.doi:10.1257/aer.102.4.1206.

Alatas,Vivi,AbhijitBanerjee,RemaHanna,BenjaminOlken,RirinPurnamasari,andMatthewWai_Poi.2015.“Self-Targeting:EvidencefromaFieldExperimentinIndonesia.”JournalofPoliticalEconomyforthcoming.

Alzúa,MaríaLaura,GuillermoCruces,andLauraRipani.2013.“WelfareProgramsandLaborSupplyinDevelopingCountries:ExperimentalEvidencefromLatinAmerica.”JournalofPopulationEconomics26(4):1255–84.

Angelucci,Manuela,andGiacomoDeGiorgi.2009.“IndirectEffectsofanAidProgram:HowDoCashTransfersAffectIneligibles’Consumption?”AmericanEconomicReview99(1):486–508.doi:10.1257/aer.99.1.486.

Angelucci,Manuela,DeanKarlan,andJonathanZinman.2015.“MicrocreditImpacts:EvidencefromaRandomizedMicrocreditProgramPlacementExperimentbyCompartamosBanco.”AmericanEconomicJournal:AppliedEconomics7(1):151–82.doi:10.1257/app.20130537.

Ashraf,Nava,OrianaBandiera,andScottSLee.2014.“Do-GoodersandGo-Getters:CareerIncentives,Selection,andPerformanceinPublicServiceDelivery.”SuntoryandToyotaInternationalCentresforEconomicsandRelatedDisciplines,LSE.

Ashraf,Nava,DeanKarlan,andWesleyYin.2006.“TyingOdysseustotheMast:EvidencefromaCommitmentSavingsProductinthePhilippines.”QuarterlyJournalofEconomics121(2):673–97.

Attanasio,Orazio,BrittaAugsburg,RalphDeHaas,EmlaFitzsimons,andHeikeHarmgart.2015.“TheImpactsofMicrofinance:EvidencefromJoint-LiabilityLendinginMongolia.”AmericanEconomicJournal:AppliedEconomics7(1):90–122.doi:10.1257/app.20130489.

Augsburg,Britta,RalphDeHaas,HeikeHarmgart,andCostasMeghir.2015.“TheImpactsofMicrocredit:EvidencefromBosniaandHerzegovina.”AmericanEconomicJournal:AppliedEconomics7(1):183–203.doi:10.1257/app.20130272.

Baird,Sarah,CraigMcIntosh,andBerkÖzler.2011.“CashorCondition?EvidencefromaCashTransferExperiment.”TheQuarterlyJournalofEconomics126(4):1709–53.doi:10.1093/qje/qjr032.

Baker,JudyL.,andMargaretE.Grosh.1994.“PovertyReductionthroughGeographicTargeting:HowWellDoesItWork?”WorldDevelopment22(7):983–95.doi:10.1016/0305-750X(94)90143-0.

- 45 -

Bandiera,Oriana,RobinBurgess,NarayanDas,SelimGulesci,ImranRasul,andMunshiSulaiman.2016.“LaborMarketsandPovertyinVillageEconomies.”LSEWorkingPaper.http://sticerd.lse.ac.uk/dps/eopp/eopp43.pdf.

Banerjee,Abhijit,EstherDuflo,RaghabendraChattopadhyay,andJeremyShapiro.2016.“LongTermImpactofaLivelihoodIntervention:EvidencefromWestBengal.”WorkingPaper.

Banerjee,Abhijit,EstherDuflo,RachelGlennerster,andCynthiaKinnan.2015.“TheMiracleofMicrofinance?EvidencefromaRandomizedEvaluation.”AmericanEconomicJournal:AppliedEconomics7(1):22–53.doi:10.1257/app.20130533.

Banerjee,Abhijit,RemaHanna,JordanKyle,BenjaminOlken,andSudarnoSumarto.2014.“InformationIsPower:IdentificationCardsandFoodSubsidyProgramsinIndonesia.”WorkingPaper.

Banerjee,Abhijit,DeanKarlan,andJonathanZinman.2015.“SixRandomizedEvaluationsofMicrocredit:IntroductionandFurtherSteps.”AmericanEconomicJournal:AppliedEconomics7(1):1–21.

Banerjee,Abhijit,andAndrewNewman.1993.“OccupationalChoiceandtheProcessofDevelopment.”JournalofPoliticalEconomy101:274–98.

Banerjee,AbhijitV.,RemaHanna,GabrielKreindler,andBenjaminA.Olken.2015.“DebunkingtheStereotypeoftheLazyWelfareRecipient:EvidencefromCashTransferProgramsWorldwide.”http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2703447.

Bardhan,PranabK.,andDilipMookherjee.2000.“CaptureandGovernanceatLocalandNationalLevels.”AmericanEconomicReview90(2):135–39.doi:10.1257/aer.90.2.135.

Barrera-Osorio,Felipe,MarianneBertrand,LeighLinden,andFranciscoPerez-Calle.2011.“ImprovingtheDesignofConditionalTransferPrograms:EvidencefromaRandomizedEducationExperimentinColombia.”AmericanEconomicJournal:AppliedEconomics3(2):167–95.

Barrera-Osorio,Felipe,andDeonFilmer.2013.“IncentivizingSchoolingforLearning:EvidenceontheImpactofAlternativeTargetingApproaches.”WorldBankPolicyResearchWorkingPaper,no.6541.

Beath,Andrew,FotiniChristia,andRubenEnikolopov.2013.“DoElectedCouncilsImproveGovernance?ExperimentalEvidenceonLocalInstitutionsinAfghanistan.”MITPoliticalScienceDepartmentResearchPaper2013(24).

Benhassine,Najy,FlorenciaDevoto,EstherDuflo,PascalineDupas,andVictorPouliquen.2015.“TurningaShoveintoaNudge?A‘LabeledCashTransfer’forEducation.”AmericanEconomicJournal:EconomicPolicy,Forthcoming.

Berry,James.2014.“ChildControlinEducationDecisions:AnEvaluationofTargetedIncentivestoLearninIndia.”WorkingPaper.

Besley,Timothy,andStephenCoate.1991.“PublicProvisionofPrivateGoodsandtheRedistributionofIncome.”TheAmericanEconomicReview81(4):979–84.

———.1992.“WorkfareversusWelfare:IncentiveArgumentsforWorkRequirementsinPoverty-AlleviationPrograms.”AmericanEconomicReview82(1):249–61.

Blattman,Christopher,NathanFiala,andSebastianMartinez.2014.“GeneratingSkilledSelf-EmploymentinDevelopingCountries:ExperimentalEvidencefromUganda.”TheQuarterlyJournalofEconomics129(2):697–752.

Blattman,Christopher,EricGreen,JeannieAnnan,andJulianJamison.2014.“TheReturnstoCashandMicroenterpriseSupportAmongtheUltra-Poor:AFieldExperiment.”ColumbiaUniversityWorkingPaper.http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2439488.

Bobonis,GustavoJ.,andFredericoFinan.2009.“NeighborhoodPeerEffectsinSecondarySchoolEnrollmentDecisions.”TheReviewofEconomicsandStatistics91(4):695–716.

- 46 -

Bold,Tessa,MwangiKimenyi,GermanoMwabu,AliceNg’ang’a,andJustinSandefur.2013.“ScalingupWhatWorks:ExperimentalEvidenceonExternalValidityinKenyanEducation.”CenterforGlobalDevelopmentWorkingPaper,no.321.

Bolton,Paul,JudithBass,TheresaBetancourt,LiesbethSpeelman,GraceOnyango,KathleenF.Clougherty,RichardNeugebauer,LauraMurray,andHelenVerdeli.2007.“InterventionsforDepressionSymptomsAmongAdolescentSurvivorsofWarandDisplacementinNorthernUganda:ARandomizedControlledTrial.”JAMA298(5):519.doi:10.1001/jama.298.5.519.

Bolton,Paul,JudithBass,RichardNeugebauer,HelenVerdeli,KathleenFClougherty,PriyaWickramaratne,LiesbethSpeelman,LincolnNdogoni,andMyrnaWeissman.2003.“GroupInterpersonalPsychotherapyforDepressioninRuralUganda:ARandomizedControlledTrial.”Jama289(23):3117–24.

Bryan,Gharad,DeanKarlan,andScottNelson.2010.“CommitmentDevices.”AnnualReviewofEconomics2(1):671–98.doi:10.1146/annurev.economics.102308.124324.

Bursztyn,Leonardo,andLucasC.Coffman.2012.“TheSchoolingDecision:FamilyPreferences,IntergenerationalConflict,andMoralHazardintheBrazilianFavelas.”JournalofPoliticalEconomy120(3):359–97.doi:10.1086/666746.

Camacho,Adriana,andEmilyConover.2011.“ManipulationofSocialProgramEligibility.”AmericanEconomicJournal:EconomicPolicy3(2):41–65.

Cameron,A.Colin,andDouglasL.Miller.2010.“RobustInferencewithClusteredData.”WorkingPapers,UniversityofCalifornia,DepartmentofEconomics.http://www.econstor.eu/handle/10419/58373.

Chen,Shaohua,MartinRavallion,andYoujuanWang.2006.“DiBao:AGuaranteedMinimumIncomeinChina’sCities?”Vol.3805.WorldBankPublications.Washington,D.C.

Chetty,Raj,andAdamLooney.2006.“ConsumptionSmoothingandtheWelfareConsequencesofSocialInsuranceinDevelopingEconomies.”JournalofPublicEconomics90(12):2351–56.doi:10.1016/j.jpubeco.2006.07.002.

Christian,Paul.2014.“TheDistributionalConsequencesofGroupProcurement:EvidencefromaRandomizedTrialofaFoodSecurityPrograminRuralIndia.”WorkingPaper.

Coady,David.2004.“DesigningandEvaluatingSocialSafeteyNets:Theory,Evidence,andPolicyConclusions.”FoodConsumptionandNutritionDivisionDiscussionPaperNo.172.

Cohen,Jessica,andPascalineDupas.2010.“FreeDistributionorCost-Sharing?EvidencefromaRandomizedMalariaPreventionExperiment*.”QuarterlyJournalofEconomics125(1):1–45.doi:10.1162/qjec.2010.125.1.1.

Covarrubias,Katia,BenjaminDavis,andPaulWinters.2012.“FromProtectiontoProduction:ProductiveImpactsoftheMalawiSocialCashTransferScheme.”JournalofDevelopmentEffectiveness4(1):50–77.

Crépon,Bruno,FlorenciaDevoto,EstherDuflo,andWilliamPariente.2015.“EstimatingtheImpactofMicrocreditonThoseWhoTakeItUp:EvidencefromaRandomizedExperimentinMorocco.”AmericanEconomicJournal:AppliedEconomics7(1):123–50.doi:10.1257/app.20130535.

Cunha,JesseM,GiacomoDeGiorgi,andSeemaJayachandran.2013.“ThePriceEffectsofCashVersusIn-KindTransfers.”

Currie,Janet,andFirouzGahvari.2008.“TransfersinCashandIn-Kind:TheoryMeetstheData,.”JournalofEconomicLiterature46(2):333–83.

deJanvry,Alain,MarcelFafchamps,andElisabethSadoulet.1991.“PeasantHouseholdBehaviourwithMissingMarkets:SomeParadoxesExplained.”TheEconomicJournal101(409):1400.doi:10.2307/2234892.

- 47 -

deMel,Suresh,DavidMcKenzie,andChristopherWoodruff.2008.“ReturnstoCapitalinMicroenterprises:EvidencefromaFieldExperiment.”QuarterlyJournalofEconomics123(4):1329–72.

Dhaliwal,Iqbal,andRemaHanna.2014.“DealwiththeDevil:TheSuccessesandLimitationsofBureaucraticReforminIndia.”WorkingPaper20482.NationalBureauofEconomicResearch.http://www.nber.org/papers/w20482.

Duflo,Esther,PascalineDupas,andMichaelKremer.2010.“EducationandFertility:ExperimentalEvidencefromKenya.”WorkingPaper.

Duflo,Esther,WilliamGale,JeffreyLiebman,PeterOrszag,andEmmanuelSaez.2006.“SavingIncentivesforLow-andMiddle-IncomeFamilies:EvidencefromaFieldExperimentwithH&RBlock.”QuarterlyJournalofEconomics121(4):1311–46.

Dupas,Pascaline,VivianHoffmann,MichaelKremer,andAlixPetersonZwane.2013.“Micro-Ordeals,Targeting,andHabitFormation.”WorkingPaper.

Edmonds,Eric,andNorbertSchady.2012.“PovertyAlleviationandChildLabor.”AmericanEconomicJournal:EconomicPolicy4(4):100–124.

Elbers,Chris,TomokiFujii,PeterLanjouw,BerkÖzler,andWesleyYin.2007.“PovertyAlleviationthroughGeographicTargeting:HowMuchDoesDisaggregationHelp?”JournalofDevelopmentEconomics83(1):198–213.

Epple,Dennis,andRichardRomano.2008.“EducationalVouchersandCreamSkimming*.”InternationalEconomicReview49(4):1395–1435.

Fernald,LiaC.H.,andMelissaHidrobo.2011.“EffectofEcuador’sCashTransferProgram(BonodeDesarrolloHumano)onChildDevelopmentinInfantsandToddlers:ARandomizedEffectivenessTrial.”SocialScience&Medicine72(9):1437–46.doi:10.1016/j.socscimed.2011.03.005.

Ferre,Celine.2009.“AgeatFirstChild:DoesEducationDelayFertilityTiming?TheCaseofKenya.”SSRNScholarlyPaperID1344718.Rochester,NY:SocialScienceResearchNetwork.http://papers.ssrn.com/abstract=1344718.

Fiszbein,Ariel,andNorbertRüdigerSchady.2009.ConditionalCashTransfers:ReducingPresentandFuturePoverty.AWorldBankPolicyResearchReport47603.WashingtonD.C:WorldBank.

Fudenberg,Drew,andDavidLevine.2006.“ADualSelfModelofImpulseControl.”AmericanEconomicReview96(5):1449–76.

Galiani,Sebastian,andPatrickJMcEwan.2013.“TheHeterogeneousImpactofConditionalCashTransfers.”JournalofPublicEconomics103(C):85–96.

Gentilini,Ugo,MaddalenaHonorati,andRuslanYemtsov.2014.“TheStateofSocialSafetyNets2014.”Washinton,DC:WorldBankGroup.

Gertler,PaulJ,SebastianWMartinez,andMartaRubio-Codina.2012.“InvestingCashTransferstoRaiseLong-TermLivingStandards.”AmericanEconomicJournal:AppliedEconomics4(1):164–92.

Gertler,Paul,OrieShelef,CatherineWolfram,andAlanFuchs.2013.“HowPro-PoorGrowthAffectstheDemandforEnergy.”WorkingPaper19092.NationalBureauofEconomicResearch.http://www.nber.org/papers/w19092.

Glewwe,Paul,AlbertPark,andMengZhao.2014.“ABetterVisionforDevelopment:EyeglassesandAcademicPerformanceinRuralPrimarySchoolsinChina.”UniversityofMinnesotaCenterforInternationalFoodandAgriculturalPolicyWorkingPaperWP12-2.

Grenier,James,andCassandraWolosPattanayak.2011.“RandomizedEvaluationinLegalAssistance:WhatDifferenceDoesRepresentation(offerandActualUse)Make.”YaleLJ121:2118.

Hanna,Rema,andPaulinaOliva.2015.“TheEffectofPollutiononLaborSupply:EvidencefromaNaturalExperimentinMexicoCity.”JournalofPublicEconomics122:68–79.

- 48 -

Hanna,Rema,andShing-YiWang.2014.“DishonestyandSelectionintoPublicService:EvidencefromIndia.”

Hidrobo,Melissa,JohnHoddinott,AmberPeterman,AmyMargolies,andVanessaMoreira.2014.“Cash,Food,orVouchers?EvidencefromaRandomizedExperimentinNorthernEcuador.”JournalofDevelopmentEconomics107(March):144–56.doi:10.1016/j.jdeveco.2013.11.009.

Hidrobo,Melissa,AmberPeterman,andLoriHeise.2013.“TheEffectofCash,VouchersandFoodTransfersonIntimatePartnerViolence:EvidencefromaRandomizedExperimentinNorthernEcuador.”Washington,DC:InternationalFoodPolicyResearchInstitute.https://www.wfp.org/sites/default/files/IPV-Hidrobo-Peterman_Heise_IPV%20Ecuador%203%2028%2014.pdf.

Hoddinott,John,SusannaSandström,andJoannaUpton.2014.“TheImpactofCashandFoodTransfers:EvidencefromaRandomizedInterventioninNiger.”AvailableatSSRN2423772.http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2423772.

Jacoby,HananG.1997.“Self-SelectionandtheRedistributiveImpactofin-KindTransfers:AnEconometricAnalysis.”JournalofHumanResources,233–49.

Jessee,Cassandra,LeanPrencipe,DanSherman,AlefaBanda,LiseteliNdiyoi,NathanTembo,SilvioDaidone,etal.2013.“Zambia’sChildGrantProgram:24-MonthImpactReport.”AmericanInstitutesforResearch.

Karlan,Dean,andJonathanZinman.2011.“MicrocreditinTheoryandPractice:UsingRandomizedCreditScoringforImpactEvaluation.”Science332(6035):1278–84.doi:10.1126/science.1200138.

Kazianga,Harounan,DamienDeWalque,andHaroldAlderman.2012.“EducationalandChildLabourImpactsofTwoFood-for-EducationSchemes:EvidencefromaRandomisedTrialinRuralBurkinaFaso.”JournalofAfricanEconomies21(5):723–60.

Kling,JeffreyR.,JeffreyB.Liebman,LawrenceF.Katz,andLisaSanbonmatsu.2004.“MovingtoOpportunityandTranquility:NeighborhoodEffectsonAdultEconomicSelf-SufficiencyandHealthfromaRandomizedHousingVoucherExperiment.”http://papers.ssrn.com/sol3/papers.cfm?abstract_id=588942.

Kremer,Michael,EdwardMiguel,andRebeccaThornton.2009.“IncentivestoLearn.”TheReviewofEconomicsandStatistics91(3):437–56.

Kuziemko,Ilyana,MichaelI.Norton,EmmanuelSaez,andStefanieStantcheva.2015.“HowElasticArePreferencesforRedistribution?EvidencefromRandomizedSurveyExperiments†.”AmericanEconomicReview105(4):1478–1508.doi:10.1257/aer.20130360.

Lalive,Rafael,andMAlejandraCattaneo.2009.“SocialInteractionsandSchoolingDecisions.”TheReviewofEconomicsandStatistics91(3):457–77.

Leroy,JefL,PaolaGadsden,TeresaGonzálezdeCossío,andPaulGertler.2013.“Cashandin-KindTransfersLeadtoExcessWeightGaininaPopulationofWomenwithaHighPrevalenceofOverweightinRuralMexico.”TheJournalofNutrition143(3):378–83.

Ma,Xiaochen,SeanSylvia,MatthewBoswell,andScottRozelle.2013.“OrdealMechanismsandInformationinthePromotionofHealthGoodsinDevelopingCountries:’EvidencefromRuralChina.”RuralEducationActionProjectWorkingPaperNo.266.

McKenzie,David,andChristopherWoodruff.2006.“DoEntryCostsProvideanEmpiricalBasisforPovertyTraps?EvidencefromMexicanMicroenterprises.”EconomicDevelopmentandCulturalChange55(1):3–42.doi:10.1086/505725.

Meager,Rachael.2015.“UnderstandingtheImpactofMicrocreditExpansions:ABayesianHierarchicalAnalysisof7RandomisedExperiments.”AvailableatSSRN2620834.http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2620834.

- 49 -

Merttens,Fred,AlexHurrell,MartaMarzi,RamlaAttah,MahamFarhat,AndrewKardan,andIanMacAuslan.2013.“KenyaHungerSafetyNetProgrammeMonitoringandEvaluationComponent.”ImpactEvluationFinalReport.OxfordPolicyManagement.

Meyer,BruceD.1995.“LessonsfromtheUSUnemploymentInsuranceExperiments.”JournalofEconomicLiterature33(1):91–131.

Mickelwright,John,andGyulaNagy.2010.“TheEffectofMonitoringUnemploymentInsuranceRecipientsonUnemploymentDuraction:EvidencefromaFieldExperiment.”LabourEconomics17(1):180–87.

Miguel,Edward,ColinCamerer,KatherineCasey,JoshuaCohen,KevinM.Esterling,AlanGerber,RachelGlennerster,etal.2014.“PromotingTransparencyinSocialScienceResearch.”Science343(6166):30–31.

Muralidharan,Karthik,PaulNiehaus,andSandipSukhtankar.2014.“BuildingStateCapacity:EvidencefromBiometricSmartcardsinIndia.”NBERWorkingPaperNo19999.

Nichols,AlbertL,andRichardJZeckhauser.1982.“TargetingTransfersthroughRestrictionsonRecipients.”TheAmericanEconomicReview,372–77.

Norton,M.I.,andD.Ariely.2011.“BuildingaBetterAmerica--OneWealthQuintileataTime.”PerspectivesonPsychologicalScience6(1):9–12.doi:10.1177/1745691610393524.

Olken,Benjamin.2007.“MonitoringCorruption:EvidencefromaFieldExperimentinIndonesia.”JournalofPoliticalEconomy115:200–249.

Olken,BenjaminA.2015.“PromisesandPerilsofPre-AnalysisPlans.”TheJournalofEconomicPerspectives29(3):61–80.

Osili,UnaOkonkwo,andBridgetTerryLong.2008.“DoesFemaleSchoolingReduceFertility?EvidencefromNigeria.”JournalofDevelopmentEconomics87(1):57–75.doi:10.1016/j.jdeveco.2007.10.003.

Ozier,Owen.2011.“TheImpactofSecondarySchoolinginKenya:ARegressionDiscontinuityAnalysis.”WorkingPaper.http://economics.ozier.com/owen/papers/ozier_JMP_20110117.pdf.

Patel,Vikram,HelenAWeiss,NeerjaChowdhary,SmitaNaik,SulochanaPednekar,SudiptoChatterjee,MaryJDeSilva,BhargavBhat,RicardoAraya,andMichaelKing.2010.“EffectivenessofanInterventionLedbyLayHealthCounsellorsforDepressiveandAnxietyDisordersinPrimaryCareinGoa,India(MANAS):AClusterRandomisedControlledTrial.”TheLancet376(9758):2086–95.

Paxson,Christina,andNorbertSchady.2010.“DoesMoneyMatter?TheEffectsofCashTransfersonChildDevelopmentinRuralEcuador.”EconomicDevelopmentandCulturalChange59(1):197–229.

Ravallion,Martin.1991.“ReachingtheRuralPoorthroughPublicEmployment:Arguments,Evidence,andLessonsfromSouthAsia.”TheWorldBankResearchObserver6(2):153–75.

Ravallion,Martin,DominiqueP.VandeWalle,PujaDutta,andRinkuMurgai.2013.“TestingInformationConstraintsonIndia’sLargestAntipovertyProgram.”WorldBankPolicyResearchWorkingPaper,no.6598.http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2323980.

Rawls,John.1971.ATheoryofJustice.Rev.ed.Cambridge,Mass:BelknapPressofHarvardUniversityPress.

Rofman,Rafael,IgnacioApella,andEvelynVezza.2015.BeyondContributoryPensions:FourteenExperienceswithCoverageExpansioninLatinAmerica.WorldBankPublications.https://books.google.com/books?hl=en&lr=&id=5GOzBQAAQBAJ&oi=fnd&pg=PP1&dq=Beyond+Contributory+Pensions:+Fourteen+Experiences+with+Coverage+Expansion+in+Latin+America&ots=OCI8ZK8PDk&sig=Bza3L1VoqX0fNtcDLvnIl3KrOUY.

- 50 -

Saavedra,Juan,andSandraGarcia.2012.“ImpactsofConditionalCashTransferProgramsonEducationalOutcomesinDevelopingCountries.”ProductPage.http://www.rand.org/pubs/working_papers/WR921-1.html.

Schady,Norbert,MariaCaridadAraujo,XimenaPeña,andLuisFLópez-Calva.2008.“CashTransfers,Conditions,andSchoolEnrollmentinEcuador[withComments].”Economía:JournaloftheLatinAmericanandCaribbeanEconomicAssociation.,43–77.

Schultz,TPaul.2004.“SchoolSubsidiesforthePoor:EvaluatingtheMexicanProgresaPovertyProgram.”JournalofDevelopmentEconomics74(1):199–250.

Seabright,Paul.1996.“AccountabilityandDecentralisationinGovernment:AnIncompleteContractsModel.”EuropeanEconomicReview40(1):61–89.

Singer,Peter.1997.“TheDrowningChildandtheExpandingCircle.”NewInternationalist,April,28–30.Skoufias,Emmanuel,MishelUnar,andTeresaGonzalezdeCossio.2013.“ThePovertyImpactsofCash

andin-KindTransfers:ExperimentalEvidencefromRuralMexico.”JournalofDevelopmentEffectiveness5(4):401–29.

Tarozzi,Alessandro,JaikishanDesai,andKristinJohnson.2015.“TheImpactsofMicrocredit:EvidencefromEthiopia.”AmericanEconomicJournal:AppliedEconomics7(1):54–89.doi:10.1257/app.20130475.

TheKenyaCT-OVCEvaluationTeam.2012.“TheImpactofKenya’sCashTransferforOrphansandVulnerableChildrenonHumanCapital.”JournalofDevelopmentEffectiveness4(1):38–49.

UNDESA.2013.“WorldPopulationAgeingReport.”UNDepartmentofEconomicandSocialAffairs.UNHCR.2014.“StatisticalOnlinePopulationDatabase.”UnitedNationsHighCommissionerfor

Refugees.VandenBerg,GerardJ.,andBasVanderKlaauw.2006.“CounselingandMonitoringofUnemployed

Workers:TheoryandEvidencefromaControlledSocialExperiment*.”InternationalEconomicReview47(3):895–936.

Vermeersch,Christel,andMichaelKremer.2004.“SchoolsMeals,EducationalAchievementandSchoolCompetition:EvidencefromaRandomizedEvaluation.”TheWorldBank.