World Bank Document · better job opportunities. However, until recently these opportunities have not kept pace with the increasing supply of (mostly) young and low-skilled workers

TURKEY:EVALUATING THE IMPACT OF İŞKUR’SVOCATIONAL TRAINING PROGRAMS

August 2013

Human Development Sector UnitEurope and Central Asia Region

Report No: 82306-TR

WORLD BANK

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

wb350881

Typewritten Text

82301 v1

TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS 1

TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS

August 2013

Human Development Sector UnitEurope and Central Asia Region

Report No: 82306 - TR

TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS2

ACRONYMS AND ABBREVIATIONS

ALMP Active labor market program GDP Gross domestic product IRR Internal rate of return ITT Intent-to-treat effect İŞKUR TurkishEmploymentAgency LATE Local average treatment effect MHI MentalHealthIndex MIS Managementinformationsystem OECD Organization for Economic Co-operation and Development OTJ On-the-jobtraining PISA Program for International Student Assessment SIEF SpanishImpactEvaluationFund TL TurkishLira TUIK TurkishStatisticalInstitute UN United National VQI Vocational Qualification Institution WAP Working age population WB World Bank

CURRENCY EQUIVALENTS(ExchangeRateEffectiveAugust,2013)CURRENCY = TLU$ 1.00 = 1.92 TL

Vice President: PhilippeH.LeHouerou,ECAVP CountryDirector: MartinRaiser,ECCU6 Sector Director: AnaRevenga,ECSHD Sector Manager: RobertaGatti,ECSHD Task Team Leader: CristobalRidao-Cano,ECSHD


CONTENTS

Acknowledgements v

Overview vii

Section1:TheContext:Jobs,SkillsUpgrading,andİŞKUR 1

Section2:TheEvaluationofİŞKURTrainingPrograms:Design,Data,andMethods 5

Design 5

Data 7

Methods 9

Section3:TheEvaluationofİŞKURTrainingPrograms:Results 13

ProfileofİŞKURtraineesinthesampleunderthestudybeforecoursesstarted 13

ImpactsofİŞKURvocationaltrainingunderreview 20

Whydoesthetrainingunderreviewhavealimitedimpact onemploymentandwhattypesoftrainingworkbetter? 26

Cost-benefitanalysisofİŞKURtrainingunderreview 31

Section4:StrengtheningtheImpactofİŞKURTrainingandServices 33

References 37

Annex1:EstimationMethodology 39

Annex2:DefinitionsofKeyOutcomeVariables 40

Annex3:EstimationResults 42

LIST OF BOXES

Box1:İŞKURVocationalTrainingProgram 4


LIST OF FIGURES

Figure1: Moreeducatedentrantsbutlarge“stock”oflow-skilledworkers 2

Figure2: Geographicdistributionofprovinces 6

Figure3: HigheducationalattainmentamongİŞKURtrainees 14

Figure4: LimitedjobsearchamongİŞKURtrainees 17

Figure5: İŞKURservicesareusedbymostjobseekers 19

Figure6: TraineesattachgenuinevaluetoİŞKURtraining 21

LIST OF TABLES

Table1:ProfileofİŞKURtraineesrelativetotheunemployedandthejoblessinurbanareas 16

Table2:LargeexpectedimpactofİŞKURtrainingonthelabormarket 20

Table3: İŞKUR’strainingshavenegligibleoverallimpactsonemploymentandsmall impactsonjobquality 22

Table4:NosignificantvariationintheimpactofİŞKURtrainingsonemployment bygenderandage 24

Table5:Bigmismatchbetweenexpectationsandrealities 25

Table6:Modestimpactsoftrainingonothermeasuresofindividualandhousehold well-being 26

Table7:Coursecharacteristicsassociatedwithbetterimpacts 28

Table8: Itisreallythecombinationofprivateprovisionandcompetitionthatworksbest? 29


ACKNOWLEDGEMENTS

ThisstudyistheresultofarequestbytheTurkishEmploymentAgency(İŞKUR)totheWorldBank(WB)toevaluatetheimpactofvocationaltrainingprogramsfortheunemployedand to identifyways to strengthen theseprograms.The studycouldnothavebeenpossiblewithoutthetechnicalsupportofİŞKURthroughoutits design and implementation. The WB team included Cristobal Ridao-Cano,RitaAlmeida,SarojiniHirshleifer,DavidMcKenzie,andA.LeventYener,withtheinvaluableassistanceofAyçaDönmez,ElçinKoç, andElifYükseker.The İŞKURteamcomprisedMehmetAslan,FeridunGiresun,AbdülkadirYanıcı,SerkanYücel,FeridunKaya,EmreÖzen,andDeryaDuyarCoşkun.OverallguidancewasprovidedbyMartinRaiser(CountryDirectorforTurkey,WB),UlrichZachau(formerCountryDirector for Turkey, WB), Roberta Gatti (Sector Manager, Human DevelopmentEconomics, Europe and Central Asia, WB), Jesko Hentschel (former SectorManager,HumanDevelopment Economics, Europe andCentral Asia,WB),AnaRevenga (Director,HumanDevelopment, Europe andCentralAsia,WB),NusretYazıcı(DirectorGeneralofİŞKUR),AsımGökerKeskin(DeputyDirectorGeneralofİŞKUR),MustafaKemalBiçerli(FormerDGofİŞKUR),NamıkAta(FormerDeputyUndersecretaryofMoLSS,andformerDGofİŞKUR),andBirolAydemir(FormerUndersecretaryofMoLSS).

ThefinalreportgreatlybenefitedfromcommentsprovidedbyChristianBodewig(WB),DavidRobalino(WB),andStefanieBrodmann(WB),aswellasfromfeedbackreceivedatnumerouspresentationsofthedesignandpreliminaryresultsofthestudyinWashingtonDC,Ankara,andIstanbul.ThisstudywasfundedbytheSpanishImpactEvaluationFund(SIEF),theGenderActionPlan,theWB,andİŞKUR.

TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS v



OVERVIEW

Turkey’seconomicperformanceduringthe 2000s is a global success story;high-incomestatusisnowwithinreachbutthiswillrequirecreatingmoreandbetterjobs.TheTurkisheconomygrewbyanaverageof5.5percentbetween2002and2011.Turkeytodayisthe16thlargesteconomyintheworld,withpercapita income exceeding US$ 10,000.Economic growth has been rapid andinclusive,asthewelfareofthepoorest40percentofthepopulationgrewfasterthan the average during the 2000s.Inclusivegrowth ispartly the resultofrapid urbanization generating more and betterjobopportunities.However,untilrecently these opportunities have notkept pace with the increasing supplyof (mostly) young and low-skilledworkers looking for jobs in cities,resulting in stagnant and even declining employment rates during the last twodecades.InTurkeytoday,lessthanhalfoftheworkingagepopulation(15-to64-year-olds)isemployedand42percentofworkersareinformal.Theoutstandingemployment growth after the 2008-09crisismayheraldachangeintrend(theWorld Bank is currently carrying outmoredetailedanalysisintothereasonsfor recent rapid employment growth)but with reported skill shortages,unemploymentratesaround10percentand lowemployment ratescoexisting,

policies to improve the functioning oflabormarketsremainapriority.

Upgradingtheskillsofthecurrentlaborforce is crucial for creating more and better jobs. Although the young arebecomingmore educated and skilled,half of the working age population(WAP)stillhaslessthanbasiceducation,accountingformostofthejoblessandinformalworkers,whilethedemandforskills in formal non-agricultural sectors continuestoincrease.Evenrecenthighschool and college graduates havedifficultyfindingjobsforwhichtheyareinprinciple“qualified,”despitethefactthat businesses report to have a hardtimefindingworkerswiththerightskills,indicatingaskillsmismatch(McKinsey2012).While the evidence shows thatskills are best acquired the first timearound,Turkeyalsoneeds toenhancetheskillsofitsexistinglaborforce.

The Turkish Employment Agency(İŞKUR)playsaleadingroleinupgradingthe skillsof jobseekers and facilitatingtheiraccesstoproductiveemploymentby providing vocational training andother employment support services.İŞKURhascomealongwaysince2008,significantly expanding its coverage(from30,000traineesin2008to464,000in2012,representing19.6percentofthe

TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS vii


registeredunemployed).Theexpansionof İŞKUR’strainingcoveragehasbeencomplementedwithfurtherreformstoimprovethequalityandeffectivenessofvocationaltrainingprograms,includingthe introduction of quality criteria inthe selection of training providers aswell as vocational and job counselingservices.

The increasing importance of İŞKURvocational training prompted thegovernmenttocommissionthepresentstudy to evaluate its impact and toidentify ways to enhance it. It is themost recent product of a long standing collaboration between the Ministryof Labor and Social Security (MoLSS)and İŞKUR with the World Bank toexaminepolicyoptionstohelpimproveemploymentprospectsforjobseekers,particularlyamongwomenandyouth.The study could have been possiblewithoutthetechnicalsupportprovidedbyİŞKUR.

The study evaluates themain type ofİŞKUR’svocationaltrainingcoursesataparticularpointintime,andthusitdoesnot take into account recent reforms.The study evaluates a representativesample of general vocational training courses,whichaccountforabouttwo-thirdsofİŞKURvocationaltrainees(theotherone-thirdmostlytakecourseson-demandbyemployers)thattookplacebetween December 2010 and June

2011.However, approximately80%ofthe2012courseswerejobguaranteedcourses. The active labor marketservices regulation was amended in 2013 to increase the rate of jobguaranteed courses. In this respect, itis expected that the2013courseswillhaveasimilarrate.

The evaluation has an experimentaldesign, exploiting the excess demandfor İŞKUR vocational training coursesto randomly assign eligible trainingapplicants into those who receivetraining (treatment group) and thosewho do not (control group). Originaldatawascollectedfrom5,902applicants(henceforth called İŞKUR trainees)(evenly split between treatment andcontrol groups) between September2010andJanuary2011(beforetrainingstarted) and approximately one yearlater. The experimental design allowsthedifferenceinemploymentoutcomesafter the training between treatmentand control groups to be attributed to training, and training only. This is thefirst randomized evaluation of a large-scalevocationaltrainingprogramfortheunemployedinadevelopingcountry.

Thestudyinvestigates:(i)theprofileandjobsearchbehaviorof İŞKUR traineesin the sample before courses started,as well as their assessment of İŞKURservicesbeforethestartofthecourses;(ii) the average impact of training on

TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMSviii


employment;(iii)differentialimpactsfordifferentgroupsoftrainees,particularlyintermsofgender,age,andeducationlevel;(iv)theprocessbywhichtrainingleads to employment outcomes,including identificationof the typesoftraining that have a higher impact onemployment; and (v) the costs andbenefitsof İŞKURtraining (overallandfor different types of training). Finally,the study makes use of the findingsfrom this evaluation, complementedby evidence from other countries, tosuggestanumberofpolicyoptionstofurtherstrengthentheimpactofİŞKURtraining and services.

The profile of İŞKUR trainees in the sample before courses started

İŞKUR trainees are significantlyyounger, have less work experience,andaremoreeducatedthantheaveragejobseeker in urban areas. Women are overrepresented among İŞKURtrainees (63 percent) compared tourban jobseekers (30percent).1 İŞKURtrainees are significantly younger (45percent are between the ages of 15and25comparedto31percentamongurban jobseekers) andhave lesswork

experience.Themoststrikingdifferenceis that İŞKUR trainees are remarkablymoreeducatedthanurbanjobseekers:74 percent of İŞKUR trainees havecompletedatleasthighschool,versus42 percent of urban jobseekers. Thisdifference persists even after limiting the urban sample to 20- to 29-year-olds.

Thesignificantlyhighereducationlevelof İŞKURtraineesismostlyduetothedesign and implementation of İŞKURtraining. The difference in educationalattainmentbetweenİŞKURtraineesandthe average jobseeker in urban areascould be explained by a number offactors:(i)İŞKURtraineesmusthaveatleastprimaryeducationandmustmeetother skill prerequisites dependingon the course; (ii) many courses aredesignedforpeoplewithmediumlevelsofeducation;(iii)intheeventofexcessdemandforcourses,trainingproviderstend to prefer individuals with higherlevels of education;2 and (iv) moreeducatedjobseekersaremorelikelytoapplyforİŞKURtraining.3

İŞKUR trainees engage in little jobsearch, although most of those who

1- DataonurbanjobseekersarefromtheLaborForceSurvey.

2- Amongotherthings,moreeducatedindividualsareperceivedtobemorelikelytocompletetraining,whichiswhatdeterminespaymenttoproviders.Until2008,paymenttoproviderswastiedtojobplacementtargetsandmoreeducatedpeopleareperceivedtodobetterinthelabormarketregardlessoftheeffectivenessoftraining.

3- Thiscouldbebecausethecoursesaremoreattractiveforbettereducatedjobseekers,theyaremoreinformedaboutthem,ortheyexpecttogetahigherreturnfromthem(althoughasshownlater,wefindlittleevidenceofthelatter).

TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS ix


do tend to use and value İŞKURemploymentservicestohelpthemfindajob.Overthelastfourweekspriortotheface-to-facebaselinesurvey(beforecourses started), only about half ofİŞKUR trainees not currently workingor going to school actively searchedfor a job.Interestingly, İŞKUR traineesare less likely to be looking for a jobthan urban jobseekers, even thoughany jobseeker registered in İŞKUR issupposed to look fora job inorder tobenefit from İŞKUR services. Traineesthatdosearchdevotesignificanttimetoitmostlylookforhigh-paying,full-timewageemployment.About74percentoftraineessearchingforajobuseİŞKURservices.. In particular, İŞKUR servicesareregardedasthesecondbestchanneltofindwageemploymentafterapplyingdirectlytoemployers,andcomebeforethenetworksoffamilyandfriends.

İŞKUR trainees attach a great exantelabor market value to İŞKUR training,evenrelativetoothertrainings,becausethey perceive it as being of qualityand valued by employers. Before courses start, 80 percent of traineesbelieve İŞKUR training will help themdo better in the labor market. Theexpected impacts after oneyear arelarge: trainees expect İŞKUR trainingtoincreasetheirprobabilityoffindingajobby32percentagepointsrelativetonotrainingandby14percentagepointsrelative to other trainings. Almost all

traineesthinkİŞKURtrainingimprovesjob-readinessskillsandtheknowledgeof the profession and are confidentabout the quality of İŞKUR training,particularly if provided by publicinstitutions. Almost all trainees also find trainingusefulbecause they thinkemployersvalue the İŞKURcertificate.Despite thegenuinevalueattached toİŞKURtraining,forone-thirdoftraineesthe stipend received is an importantreasonfortakingthetraining,andtwo-thirdswould not take it if they had topayforit.

Impacts of İŞKUR vocational training under review

The overall impact of İŞKUR trainingcourses considered under this studyon employment is negligible, butcourseshavea small but significantimpact on the quality of employment,withlittlevariationofimpactacrossage,gender, or level of education. İŞKURtrainingcoursesconsideredunderthisstudyare found tohavenosignificantimpactonthelikelihoodofworking,thenumberofhoursworkedperweek,orthemonthlyincomereceived.However,coursesare found to have a small butstatistically significant impact on thequality of employment. In particular,İŞKURtrainingincreasestheprobabilityofworkingintheformalsectorbyupto3percentagepoints,whichisequivalentto a 10 percent increase in formal sector

TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMSx


employment. And training increasesincomefromformalemployment(byupto13percent)andoccupationalquality.4 Theimpactoftrainingonemploymentappears larger formen over 25 yearsofage,butthedifferencesbyageandgenderarenotsignificant,norare thedifferences by level of education.Thesmall impacts of İŞKUR training onemploymentareinlinewithfindingsforsimilarprogramsaroundtheworld.

The actual impacts of İŞKUR trainingunder review differ significantly fromthe expectations formed before thetrainings, which may in part indicatelimited information about İŞKURtrainings. On average, individualsare reasonably close in terms of theirexpectations of what employmentlevels will be like in the absenceof training. However, individualssubstantiallyoverestimate thebenefitsfrom training.Thiscouldbeexplainedbythenaturaltendencytooverestimatefuture benefits and underestimate future costs of action taken today,or changes in context from the timewhen expectations were formed. Itmay also suggest, however, that theaverage trainee does not have muchinformationabout trainingsofferedbyİŞKURandhencethetrainingmaynotbe sufficiently targeted. Interestingly,thehighevaluationofİŞKUR’sservices

persists even after the training iscompleted, and even though for theaveragetraineethetrainingmakeslittledifference in terms of employment orearnings. Possibly, İŞKUR’s serviceshaveanintrinsicvaluenotcapturedbyactualemploymentstates.

Why does training have a limited impact on employment and what types of training work better?

Thesmall impactofthetrainingunderreviewdoesnotseemtobeexplainedby the labormarketcontext.Althoughat the time of training there was awide variation in unemployment ratesacross the 23 evaluation provinces,trainees from high unemploymentprovinces did not do better or worse atfindingemploymentthanthosefromlow unemployment provinces. Thisresult suggests that the small impactof training on employment does notappear to be driven by the buoyantlabormarketat the time the follow-updatawerecollected(early2012).

LowlevelsofactivejobsearchamongİŞKURtraineesmaybeonereasonthetraininghaslittleimpactonemploymentoutcomes. Wage and employmentexpectations among trainees appeartoexceed themeanof those inactualemploymentbyaconsiderablemargin.

4- Thisisacontinuousmeasurerangingfrom16(e.g.,domestichelpers)to90(e.g.,judges),andiscodedaszeroforindividualsnotworking(seeAnnex2fordetails).

TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS xi


Trainees may decide not to activelyseek a job or not to take the jobson offer post-training, even if theiremployment prospects improved.Improved targeting of training and improved incentives to encourage active job search are part of İŞKUR’srecentreformsandtheevidencelendssupporttotheirimportance.

İŞKUR courses contracted to privateproviders, particularly those facingmore competition5 have a largepositive impact on employment. Theresults show that there is a higherreturn from taking a course offered by private provider or a provider thatfacesmorecompetition.Butitisreallythe combination of private provisionand intense competition (having twoor more competitors) that makesthe biggest difference: 17 percent ofthe courses included in the study fitthesecriteria,andthetraininginthosecourses increases the likelihood ofworkingfor20ormorehoursperweekby9percentagepoints.Whilesomeofthisdifferentialimpactrelativetoothertypes of training is due to differencesin the distinctive courses offered byprivateprovidersandthetraineestheyattract, the differential impact remainseven after controlling for course and

trainee characteristics. This suggests

that private provision may be one

route to ensure training courses meet

the needs of the labor market and

hence improve their effectiveness in

overcomingmismatches.

Overall, İŞKUR training courses under

review have a negligible net return,

but İŞKUR courses offered by private

providers that face more competition

have a big net return. The total cost

to İŞKUR of providing a training

course averages 2,429TLper person.

The average cost per person is 2,671

TL if the İŞKUR course is offered by

privateproviderssubject tosignificant

competition.6 The average gain in

monthlyincomefromİŞKURtrainingis

closetozero,sotheannualinternalrate

ofreturn(IRR)over30yearsisalsoclose

tozero.Incontrast,thegaininmonthly

income for İŞKUR trainees taking

courses offered by private providers

facing significant competition is 128

TLpermonth.Basedonthisestimated

gain, it would take 21months for the

gain in income to offset the costs of

provision, and the IRR over 30 years

wouldbe58percent(or48percentwith

10percentperyeardepreciationofthe

gains).

5- Thecompetitionvariableisbasedonthequestiontoprovidersaboutthenumberofcompetitorstheyface:(i)zero;(ii)one;(iii)twotofive;(iv)sixtonine;or(v)10ormorecompetitors.

6- Thetrainingcostreflectsthecostoftrainingduringthestudywasimplemented.

TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMSxii


Strengthening the impact of İŞKUR training and services

After thedata for thisevaluationwerecollected, İŞKUR continued expandingand introduced reforms to address some ofthechallengesidentifiedinthisstudy.Indeed, two recent reforms includingthe selection of training providerson the basis of specific quality andperformancecriteria(notjustcost)andthe introduction of job and vocationalcounselors will help address at leasttwoissuesidentifiedbythisevaluation:(i) thequalityandemployment impactof training; and (ii) the informationtrainees have about the courses theytake. In2012, İŞKURstartedhiring joband vocational counselors to advise jobseekers on occupational choiceand training courses. A new regulation approvedinMarch2013assignsmoreweight to quality in the selection ofproviders and introduces modules to the training programs to increaseemployability. The regulation alsoextends jobs placement requirementsto providers of general vocational training, and incentivizes employersto hire and keep young and femaletrainees employed in occupations forwhichtheyaretrained.

This section builds on these goodinitiativesandachievementstosuggestsome options to further streng thenİŞKUR training and services based on

theresultsoftheevaluationandlessonsfrom international experience. Reformcanonlyhappengradually,introducinga few new initiatives and evaluating thembeforemovingtothenextsetofreforms.ThisistheapproachİŞKURistakingandthisjointevaluationisagoodillustrationofthis.

Improving the relevance of skills training. The study finds that the lowoverall impactofİŞKURtrainingunderreviewonemploymentmayinpartbedue to the low value-added of thesecourses in terms of the skills theyhelp tobuild.As İŞKURdevelopsnewmodules to be added to the trainingprogram, it may consider puttingmore emphasis on behavioral skills,whicharehighlyvaluedbyemployersin Turkey (McKinsey 2012). Andthere is increasing evidence of theemploymentimpactofbehavioralskillstraining (Almeida et al. 2012). Goingforward,itisalsoimportanttocontinuestrengtheningthelinkbetweenİŞKUR,trainingproviders,andlocalemployers,building on the experience of theProvincial Employment and TrainingCouncils.

Incentivizing and supporting more job search. Thelowincidenceofjobsearchamong İŞKUR trainees in the studysuggests theneedtoencouragemorejob search and expand employmentservices (job placement, counseling,

TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS xiii


job-search assistance). The newregulation linking training to jobssearch is a step in the right direction.International evidence shows that it ismore cost-effective to first encourage jobseekers to look for a job andto assist them in this task throughemployment services before theyget any training than to offer trainingfirst (as in Turkey). To encourage jobsearch,itisimportanttotiethereceiptof unemployment or social assistancebenefitstoit—Turkeydoesthisalready,but perhaps compliance could beimproved.Employmentservices,whicharelimitedinTurkey,shouldbecentralto employment activation efforts. Forexample,intheU.K.’sjobcenterplus,allregistered unemployed people areoffered some employment services(rangingfromminimumjobplacementservices to job-search assistance forthe hard-to-employ) and are requiredto take individual actions to find a job (i.e., the market test– receipt ofbenefits is conditional on that) beforeit is determined whether they needadditionalservices(includingtraining).

Defining the priority groups for training. ThelowoverallimpactofİŞKURtrainingunder review on employment couldalsoberelatedtowhoİŞKURisactuallytrainingandwhether these jobseekersaretheoneswhocanbenefitthemostfromtrainingrelativetoothergroupsofjobseekers. İŞKURis trainingthemost

educated jobseekers. Aside from theminimum requirement to have basiceducation,thisismostlytheresultofthecoursesİŞKURoffersandtheselectionofapplicantsbyproviders.Theevaluationdoes not showdifferential impacts bygenderandage,buttheseresultsrefertothecurrentprofileofİŞKURtrainees(e.g., highly educated) as well as thetraining and services İŞKUR currentlyoffers. Low-skilled workers account for most of the labor force and face thegreatest jobschallenge.Andtheyfacebarriers to productive jobs other thanskills, including information. Countrieswithwell-developedpublicemploymentservices (e.g., the U.K., Germany, andAustralia) serve all jobseekers thatregister,butthebulkoftheirresources(includingskillstraining)supporthard-to-employ jobseekers. İŞKUR hasrecently started to serveone segmentof the hard-to-employ, namely thosereceiving social assistance benefits and able to work.

ThefocusofİŞKURtrainingsonwomenand youth seems a priori appropriatefrom a policy perspective. Activityrates are especially low for womenand youth. International evidenceshows that well-designed skills trainingprogramshaveahigherreturnforyouththan others (Betcherman et al. 2007),mainly because it is easier to learnwhenyoung.Successfulprograms foryouthintheU.K.,theU.S.,andseveral

TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMSxiv


Latin American countries (e.g., theJovenesprogram)targetdisadvantagedout-of-schoolyouth(typically15-to29-year-olds with less than a secondaryeducation). And in countrieswith lowfemale employment, like Colombia(Attanasio et al. 2011) and theDominican Republic (Ibarraran 2012),well-designedprograms foryouthcanhavealargepayoffforyoungwomen.

Better information on jobseekers to adjust services to needs.Thenew joband vocational counselors hired byİŞKUR will most certainly result in abetter match of trainees to courses(a possible problem identified in thisevaluation),buttheydonotassesstheemployability of jobseekers and thustheirneedfortraining(orotherservices)to begin with. Countries with well-developedpublicemploymentserviceslike the U.K. and Australia do makean initial employability assessmentof jobseekers, which is then usedto “profile” jobseekers into differentgroupsreceivingdifferentemploymentsupport packages, with the bulk ofresourcesgoingtothehard-to-employ.

More contracting of services to the

private sector while ensuring quality.

The high return of İŞKUR courses

offeredbyprivateprovidersfacingmore

competition suggests that increasing

the share of courses subcontracted

to private providers and increasing

competition among them would

significantly increase the employment

impactofİŞKURtraining.However,itis

alsoimportanttoensurethequalityof

providersandmakethemaccountable

for results. İŞKUR has already taken

a number of measures to increase

the quality of providers through the

selection process. İŞKUR has also

recently extended jobs placement

requirements to providers of general

training.Thereisstillroomtostrengthen

thecontractswithprivateproviders to

improve the impact of training. And

to avoid “creaming” off the easy-to-

employ by private providers, and to

take into account the higher cost of

helping the hard-to-employ become

employed, the contracting of services

forthemcouldbedoneseparately,asis

doneintheU.K.

TURKEY: EVALUATING THE IMPACT OF İŞKUR’S VOCATIONAL TRAINING PROGRAMS xv



SECTION 1: THE CONTEXT: JOBS, SKILLS UPGRADING, AND İŞKUR

1. Turkey’s labor market continues to

be characterized by low employment

rates—particularly amongwomenand

youth—and low labor productivity.

Despite the remarkable upturn in

employment after the crisis, still less

thanhalfoftheworkingagepopulation

(15-to64-year-olds)wereemployedas

ofmid-2012and theemployment rate

among working age women was under

30percent.About35percentofyouth

(15- to 24-year-olds), mostly women,

are neither working nor attending

school—the highest share of inactive

youth among OECD countries. Job

informality (defined as jobs without

socialsecuritybenefits)hascomedown

significantlysinceitwasfirstmeasured

in2005,butitstillaffected42percentof

workers in2011(28percentexcluding

the agricultural sector), contributing

to Turkey’s lower labor productivity

compared with the OECD and other

peercountries(WorldBank2013a).

2. Thelarge“stock”oflow-skilledworkersisakeystructuralfactorbehindthejobschallenge inTurkey.New entrants into the labor force are quickly becomingmoreeducated:theproportionof25-to34-year-oldswithatleastahighschooleducation increased from 26 percent in2000toabout41in2010(Figure1).These remarkable changes, however,areslowtoaffectthestockofeducationin the labor force:half of theworkingagepopulation(WAP)stillhaslessthana basic education. As firms strive to staycompetitiveinaglobalmarket,the“skills bar” in formal non-agriculturalsectorsisincreasing—skillsarealreadythe thirdmost important constraint tobusiness operations cited by Turkishfirms.7 The large supplyof low-skilledworkers and the decreasing demandfor them are the main forces behindthepoorjobperformanceofthisgroup,which accounts for 64 percent of thejobless and 65 percent of informalworkers.

7-BusinessEnvironmentandEnterprisePerformanceSurvey(BEEPS)for2008.


3. In addition, there are indications ofamismatchbetweentheskillsobtainedineducationbyyoungworkersandtheneeds of the labormarket.According to a recent comparative study byMcKinsey(McKinsey,2012)8,56percentofemployersinTurkeysaytheycannotfind workers with the right skills andthis despite the fact that more thanonethirdofyouthareneitherworkingnorattendingschool.Themainreasonbehind this apparent paradox is themismatch between the skills suppliedby youth and those demanded byemployers. According to the samestudy,youthinTurkeyoftendonotgoto collegebecause theybelieve it haslittle valueadded.This is validatedby

the experience of those who actuallywenttocollege(54percentthinkcollegeeducation did not improve their jobopportunities) and by actual data (thereturntocollegeeducationinTurkeyisamongthelowestintheOECD).Finally,theskillsthatemployersacrossTurkeyvaluethemostarebehavioral(e.g.,workethics, teamwork, and communicationskills),not technical.This isconsistentwith findings for the other countriesincluded in the study, which highlightthat as countries move up the valuechainhigher-levelcognitiveskills(e.g.,problemsolving,communication),andbehavioral skills (e.g., perseverance,self-discipline, teamwork) becomeincreasinglymoreimportantthatbasic

8- Theother7countriescoveredincludeBrazil,Germany,India,Mexico,Morocco,SaudiArabia,theU.K,andtheU.S.

FIGURE 1

Moreeducatedentrantsbutlarge“stock”oflow-skilledworkers

New entrants are more educated(%of25-34-year-oldswithatleastahighschooleducation)

But there is a large “stock” of low-skilled workers(EducationlevelsoftheWAPin2010,bygender)

Source:TUIK(LaborForceSurvey),UN(urbanization,populationprojections),andauthors’calculations.

LessthanbasicBasicSecondaryCollage

45

40

35

30

25

20

15

10

5

02000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010

60

50

40

30

20

10

0

52

43

56

1922

1619

23

16

1012

8

Total Men Women


cognitive (numeracy and literacy) andtechnicalskills.

4. The government has launchedsignificant education reforms, withsignificant achievements already interms of increased coverage and student learning.9 But the impact ofthese education reforms will onlymaterialize in the long run, while thegrowthpotentialoftheTurkisheconomyis hampered by the large segment ofthecurrentlaborforcethatisinneedofskills upgrading.

5. The Turkish Employment Agency(İŞKUR)playsakeyroleinupgradingtheskillsof jobseekers.Skills upgrading is relevant for existing workers, throughon-the-job training (OJT), and forjobseekers. Despite being one ofthe countries where employers aremore concerned about the skills ofthe workforce, Turkey is among thecountrieswith the lowest incidenceofOJT:only about 29percent of firms inTurkeyprovideOJTtotheiremployees(World Bank 2010). İŞKUR is themainproviderofskillstrainingforjobseekers.Vocational training is one of İŞKUR’smain active labor market programs (ALMP) to help jobseekers findemployment.İŞKURisalsoresponsibleforotherALMPs (employmentservices,on-the-job training, entrepreneurshipservices, public works) as well asfor passive labor market programs

(unemployment insurance, short-timework schemes, employment incentives,andwageguarantees).

6. TheincreasingimportanceofİŞKURvocational training prompted thegovernmenttocommissionthepresentevaluation. As part of the 2008 laborreform, participation in ALMPs by allregisteredjobseekerswasfullyfundedbytheUnemployment InsuranceFundregardless of they are eligible forunemploymentinsurance.Thenumberof vocational trainees increased from 30,000 in 2008 to 464,000 in 2012,representing 19.6 percent of registered jobseekers. Aware of the importanceof İŞKUR training to improve theemployability of jobseekers, theleadership of the Ministry of Laborand Social Security (MoLSS) andİŞKUR partneredwith theWorld Bankto evaluate İŞKUR training programsin order to inform the expansion andimprovement of these programs.The present study evaluates generalvocational trainingcourses (seeBox1for a description of these programs),which is the main type of trainingofferedbyİŞKUR.Theevaluationdoesnot cover recent reforms but some of itsmainfindingsarereflectedinthesereforms. Moreover, as a result of thejoint evaluation İŞKUR’s capacity tocontinuouslymonitor and improve itsportfolioofALMPhasbeenenhanced.

9- Turkeyhasvirtuallyachieveduniversalprimaryeducationandincreasednetsecondaryschoolenrollmentto67percent,whileatthesametimerecordinganimpressivehalfayearofschoolgaininPISAscoresbetween2003and2009(WorldBank2013b).


BOX 1: İŞKUR Vocational Training Programs

İŞKUR provides two basic types of training for the unemployed: general training,whichaccountsforabouttwo-thirdsofcoursesandbeneficiariesandisthesubjectofevaluationunderthisstudy;andjob-guaranteedtraining.ThelatterinvolvesemployersapproachingİŞKURandrequestingthetrainingofacertainnumberofnewemployeesinaparticularfield.Inthiscase,employersareexpectedtoofferjobplacementstoatleasthalfof the trainees.Theevaluation focusesongeneral trainingcourses,wherethereisnojobplacementrequirement.

However,approximately80%of the2012courseswere jobguaranteedcourses.Theactive labormarketservices regulationwasamended in2013 to increase the rateofjobguaranteedcourses.Inthisrespect,itisexpectedthatthe2013courseswillhaveasimilar rate.

Generaltrainingcourseshaveanaveragelengthofthreemonths,coverawiderangeof fields, and are planned through the İŞKUR provincial offices through the year.Trainingcoursesareeithercontracteddirectlytopublicinstitutions(MinistryofNationalEducation) (about 50percentof trainees)or areopen to competition amongprivateproviders.Trainingparticipantsreceiveastipendof15TLperday(20TLfrom2012)tocovertheindirectcostsofattendingtraining.

Tobeeligible toparticipate in the course, individualsmustbeat least 15 yearsold,haveat leastprimaryeducation,andmeetotherskillprerequisiteswhichdependonthecoursetheywishtoparticipatein(forexample,softwarecoursesmayrequiresomepre-existingITknowledgeorskills).Givenexcessdemandforcoursesandadesiretogettheunemployedintojobs,individualsareonlyallowedtotakeoneİŞKUR-supportedcourse ina24-monthperiod.Coursesareadvertisedandpotential traineesapply tothem.Applicationsarethenscreenedtoensureapplicantsmeettheeligibilitycriteria.Iftherearemoreeligibleapplicantsthantrainingseats(i.e.,ifthereisexcessdemand,asisthecasewithmostİŞKURcourses),thetrainingproviderinterviewscandidatesandchoosesthosebestsuitedforthecourse.


SECTION 2:THE EVALUATION OF İŞKUR TRAINING PROGRAMS: DESIGN, DATA, AND METHODS

Design

1. The study makes use of anexperimental design to evaluatethe main type of İŞKUR’s vocationaltraining courses at a particular point in time. The study evaluates the impactof general vocational training courses using an experimental design thatexploits the excess demand for mostİŞKURvocationaltrainingcourses.Theevaluation focuses on courses thatwereoversubscribed,startedbetweenOctoberand December of 2010, andwere completed by June 2011. Theevaluationwasdesignedtoanswerthefollowingmainquestions:

• Whatistheaverageimpactoftrainingonemployment?

• Whichtraineesbenefitthemostfromtraining?

• Whatarethechannelsthroughwhichtrainingaffectsemployment?

• Whattypesoftraininghaveahigherimpactonemployment?

2. Evaluation provinces.Thegoalwastoensureabroadgeographicdistributionand range of labor market conditions. Theselectionofprovincesbeganwitha list of the 39 provinces that had atleast two significantly oversubscribedtraining courses in 2009. Theseprovinces were first stratified bywhether their unemployment ratewasaboveorbelowthemedianof10percent in 2009. Ten provinces were then randomly selected from eachstrata with a probability proportionaltothepercentageofindividualstrainedin 2009. Three additional provinces(Antalya, Gaziantep, and Diyarbakir)were included in the sample at therequest of İŞKUR because of theirimportance in representing varyinglabormarketconditionsacrossTurkey.Asaresult,23provinceswereselectedfor inclusion in the evaluation (Figure2).


3. Evaluation courses. The targetsample size of individuals needed to estimate the impact of training wascalculated as 5,700. This target wasdivided amongst the 23 provinces inproportion to the number of traineesintheseprovincesinthepreviousyear.ThusIstanbulaccountsfor21.8percentof the sample; Kocaeli, Ankara, andHatayprovincescollectivelyaccountforanother28percent;andtheremaininghalf of the sample is split among theother 19 provinces. The key criteriausedtodecidewhichcoursestoincludeintheevaluationwere:(i)thelikelihood

of the course being oversubscribed

(which ensures that themost popular

typesoftraining,forwhichtherewould

be demand for further scale-up, are

included); (ii) inclusion of a diversity

oftypesoftrainingproviderstoenable

comparison of private and public course

provision;and (iii)coursestartingand

endingdates, i.e.,courses thatstarted

between October and December 2010

andfinishedbyJune2011,asthistends

to be a timeof yearwhen jobseekers

aremorelikelytoseektrainingthrough

İŞKUR.

FIGURE 2

Geographicdistributionofprovinces

Note:Thenamesofevaluationprovincesareindicatedintheorangerectangles.


4. This resulted in a set of 130evaluation courses spread throughoutTurkey, of which 39 were offered byprivate providers and the rest weregovernment-operated.Thesinglemostcommon course was computerized accounting, for which 24 percent oftrainees applied. Twenty-one percentof trainees were in service courses (babysitter,cashier,waiter,caringfortheelderly); 15.4 percent were craftsmanor machine operators (welder, naturalgas fitter, plumber, mechanic); 14.7percent were in technical courses(computer technician, computer-aideddesign,electricalengineering);and12.2percent were in professional courses (webdesigner,computerprogrammer,IT support specialist). The averagecourse size was 28 trainees, and theaverage course length was threemonths(typicallyaroundsixhoursperday),bothwithsignificantvariation.

5. Evaluation participants. Courses were advertised and potential trainees applied tothemfollowingstandardprocedures.Applications were screened to ensure applicantsmet the eligibility criteria ofİŞKURandthecourseprovider.Trainingproviderswere then asked to select alistofpotentialtraineesthatwasatleast2.2timesthecoursecapacity.Typicallythis involved short interviews witheligible applicants. These individuals’applicationdetailswerethensubmittedinto İŞKUR’sManagement Information

System (MIS). The MIS stratifiedapplicants for each course by genderand age (i.e., whether or not theywere less than 25 years old). Withinthese strata, trainees were randomlyallocatedbytheMISintothreegroups:atreatmentgroupthatwasselectedfortraining,acontrolgroup thatwasnot,andawaitlistedgroupthatthetrainingprovidercouldselectintothetrainingifthereweredropouts.Thefinalevaluationsample consisted of 5,902 applicants,ofwhich3,001wererandomlyassignedtotraining(treatmentgroup)and2,901werenot(controlgroup).

6. Compliance with treatment. Not all those accepted into the course(treatmentgroup) tookup the training(23 percent). Of those who did, mostcompletedthecourse(72percent)andreceived a certification (69 percent).About 3 percent of the control groupactually received a certification in anİŞKURcoursethatwasnotincludedinthesampleofevaluationcoursesduringtheperiodthatevaluationcoursestookplace.Sincepartialtrainingisunlikelytobeofinteresttoemployers,weconsiderthose who received a certification tohavereceivedtreatment.

Data

7. TheMIScontainedbasicinformationaboutthecourseandthegender,age,and education level of applicants. The


maindatafortheevaluationcamefromsurveysadministeredtotheevaluationparticipants between September 2010 andJanuary2011(baselinesurvey)andapproximately one year later (follow-up survey). Data were also collectedfrom a survey of training providersparticipatingintheevaluation.

8. Thebaselineand follow-upsurveyswere conducted through in-personinterviewsbyathird-partyprofessionalsurvey firm. The baseline surveytook place on a rolling basis between September 13, 2010, and January 31,2011.10 The goal was to conduct thesurveys before courses began, butgiventheshortwindowoftimebetweenselectionofapplicantsandthestartofthecourse,inpracticeonlyone-thirdofthosesurveyedwereinterviewedbeforethestartofthecourse,while46percentof those surveyed were interviewedwithin11daysofthestartofthecourse.Theoverallbaselineresponseratewas90 percent.

9. The follow-up survey took placebetween December 27, 2011, andMarch 5, 2012, which corresponds toaperiod approximatelyone year afterthe end of training. It collected dataon employment outcomes, aswell as

individual and household well-being.The response rate was 94 percent,including472individualswhowerenotinterviewedatbaseline. In total, 5,057individuals were interviewed at bothbaselineandfollow-up.Wefoundverysmall differences between the non-response rates (to the baseline andfollow-upsurveys)ofthetreatmentandcontrol groups.

10. The randomization of evaluationparticipants into treatment and control groupswassuccessful,whichvalidatesthe impact evaluation strategy. Theevaluation strategy (see below formoredetails) involves thecomparisonofemploymentoutcomesbetweenthetreatment and control groups and theattributing of any difference to İŞKURtraining, and İŞKUR trainingonly.Thisstrategy relies on the assumptionthat the treatment and control groupswere statistically equivalent (i.e., theyhad similar average demographiccharacteristics).Thesummarystatisticsforthebaselineandfollow-upsurveysshow that the differences in terms ofdemographic characteristics betweenthe treatment and control groupsinterviewedatbothbaselineandfollow-up were small and not statisticallysignificant(seeTableA1inAnnex3).

10-ApplicantsweretoldthatthepurposeofthesurveywastohelpimprovetheservicesofferedbyİŞKUR,andthattheirparticipationinthesurveyhadnoimpactonbeingacceptedintoanytrainingcourse,norwouldtheirindividualdatabesharedwithanybody.


Methods

11. Estimating the average impacts ofvocational training.Theaverageimpactof vocational training on employmentis estimated using two alternative methods. The first measure, theintent-to-treat effect (ITT), comparesthe average outcomes of individualsassigned to training (treatment) andthosenotassignedtotraining(control)regardless of whether people in thetreatment group actually completedthe training. This comparison alsocontrolsforthefactthattherandomizedassignment of individuals to treatment and control groups was done withingroups of individuals applying to thesame course and belonging to thesame gender and age group (called randomizationstrata).Itisimportanttonote that inbothcases, theestimatedimpactreferstothepeoplethatapplytoandareeligibleforİŞKURtraining,nottothelargerpopulationofjobseekers.Assuch,thestudyevaluatestheimpactofİŞKURtrainingasitisactuallydesignedand implemented.

12.Theotherrelevantmeasure,calledthe local average treatment effect(LATE),comparestheaverageoutcomesof people who completed training tothosewhodidnot.However, to avoidthebiasarisingfromthefactthatpeoplewho are more likely to complete thetrainingarealsomorelikelytodowell

inthelabormarket,trainingcompletionis instrumented in the estimation bytraining assignment (see Annex 1 fordetails). As such, this measure triesto capture the impact of completingtrainingforanindividualwhotakesuptrainingwhenheisassignedtotraining.It assumes,however, that traininghasno impact on employment for thosewhodonotcompletethecourse,whichmay not always hold. This estimationmethodology is subject to the samecontrols as the other one. Both ITTand LATE average impact results are reportedbelow,althoughotherresultsarebasedonITTgiventheassumptionsunderlyingLATE.

13. Outcomes of interest. The studyfocuses on the impact of vocationaltraining on employment through arange ofmeasures, includingwhetherindividualsareemployedatall,aswellashowmuchtheyareworking,howmuchtheyearnfromthiswork,thequalityandformality of this employment, and thecompositeindexofallthesemeasures(the aggregate employment index).The study also looks at the impact ofvocational training on measures of individual and household well-being.Annex2explainshowthekeyvariableswere constructed.

14. Investigating differences in impacts among different groups of individuals. The study also looks at how impacts


vary across individuals with differentcharacteristicstoanswerthequestion:Who benefits themost from training?The variation in impacts is estimatedby interacting the relevant treatmentvariables (assigned training and completed training) with differentindividual and course characteristics(seeAnnex1fordetails).

15. Understandinghowtrainingaffectsemploymentoutcomes.To understand how training could bemore effective,the study looks at four intermediatesteps inacausal chain throughwhichwemight expect to see selection intoa course influencing employmentoutcomes:

(1) Individuals selected for courses must show up and complete training. Thishypothesis is testedby interacting treatment with thepercent of individuals assigned to acoursewhoattendedthecourseorwhocompletedit.

(2) Higher quality courses should have more impact. The basicquestionis:Whattypesoftrainingworkbest?This is investigatedbyinteracting treatment with coursecharacteristics potentially relatedtoquality,includingcourselength,trainer’seducationandexperience,whether the course provider ispublic or private, and the degree

of competition faced by courseproviders.

(3) Skill acquisition, signaling, or job matching. A third step in thecausalchainisforindividualswhotakecoursesofsufficientqualitytousewhattheyhavelearnedinthecoursetofindjobs.Therearethreemain channels through whichvocational educationmay help inthisrespect.First,itmightincreasehuman capital by teaching newtechnical skills. Second, it mayact tocertifyskills that individualsalreadyhaveandactasasignalingmechanism to employers. Third,it may teach individuals newstrategies for finding jobs ina certain profession, or betteralert them to job opportunities,thereby improving job matching.To examine the extent to whichcoursesareplayingeachof theseroles,andwhichtreatmenteffectsvary with them, the follow-upsurvey asked course participantswhether they thought the coursehad done each of these threethings. The percentage of courseparticipantswho thought that thecoursetaughtnewtechnicalskills,certified existing skills, taughtnew strategies for finding jobs,ormade themmoreawareof jobopportunities was then interactedwithtreatment.


(4) Training impacts and unemployment rates. The finalstep in the causal chain is forindividuals who receive trainingtoobtainjobsthattheywouldnototherwise get. This depends onthe labormarket they face.Whenunemploymentratesarelow,thereshouldbemore jobopportunitiesavailable, making it easier forpeople to use their training tofind jobs. But it is also possiblethat firms facing labor shortagesmight hire workers regardless ofwhether or not they have beentrained. When unemploymentrates are high, if employers arenot hiring, then having new skills

may not help the unemployedfind jobs. But it is also possiblethat employers become choosierandsotrainingmayofferworkersa way to distinguish themselvesfrom other workers competingforthesamejobs.Asaresult,itistheoretically ambiguous whetherweshouldexpecttrainingtohavemore or less impact in situations ofhigherorlowerunemployment.This hypothesis is investigatedby interacting treatment with anindicatorforwhethertheprovincewhere the coursewas conductedhadanunemploymentrateabovethemedianunemploymentrateforthe23provincesunderthestudy.



SECTION 3: THE EVALUATION OF İŞKUR TRAINING PROGRAMS: RESULTS

Profile of İŞKUR trainees under the study in the sample before courses started

1. This sectionmakes use of the datafrom the baseline survey to look at:(i) the characteristics of evaluationparticipants (İŞKUR trainees) andhow they compare with those ofjobseekers in urban areas;11 (ii) thejobsearchbehaviorofİŞKURtrainees;and (iii) trainees’ assessment of andexpectations about İŞKUR services(includingtraining)beforethestartofthecourses.Themaingoalsofthisanalysisareto:(i)determinewhoreceivesİŞKURtrainingamong the largerpopulationofcomparable jobseekers,which can alsohelptobetterunderstandtheimpactoftrainingonemployment;and(ii)analyzetheextentandnatureofthedemandforİŞKURvocationaltraining.

2. Most İŞKUR trainees are womenandyouth.About63percentofİŞKURtraineesarewomen.Twelveoutofthe130coursesintheevaluationonlyhavefemale trainees (babysitting, weaving,

haircare,andsewingmachineoperator)and17courseshavenofemaletrainees(applied basic electronics, welder,plumber,furnituremanufacture,naturalgasfitter,andforkliftoperator).Womenalso comprise themajorityof traineesin a wide range of other courses,including computerized accounting,computer operator, foreign trade andcustoms professional, computer-aideddesign, computer network design,salesperson, and web designer. Theaverage age among İŞKUR trainees is27years,and60percentarebetween20and29yearsof age.Male traineestend to be somewhat younger thanfemale trainees (26 years old versus28).Partlyasaresultoftheiryouth,only12 percent of trainees are householdheadsand34percentaremarried.

3. Most traineeshave completedhighschool.About 76 percent of men and 73 percent of women trainees havecompletedatleastsecondaryeducation,and 30 percent of men and women trainees have completed at least twoyearsoftertiaryeducation(Figure3).This

11-Thecomparison iswith jobseekers inurbanareasbecause İŞKURtrainingcoursesare restricted tourbanresidents.DataonurbanjobseekersarefromtheLaborForceSurvey.


higheducationalattainmentispartlyduetotheageprofileofİŞKURtrainees,butolder cohorts are also well educated(65percentofmenand59percentofwomen34yearsorolderhaveatleastahighschooleducation).Mosttraineeshave some degree of specializationor technical background. The mostcommon technical backgrounds arecomputer technologies/programmingand accounting. About one-third of

trainees have a background in thefield of training they apply to, whileothers have general degrees (e.g.,business) and are interested in İŞKURtrainingtoacquiresomespecialization.Interestingly, 20 percent of men and29 percent of women trainees havealreadytakenvocationaltraininginthepastfiveyears,mostlyfromİŞKUR(47percent), and in areas closely relatedtothetrainingtowhichtheyappliedatbaseline.12

FIGURE 3

HigheducationalattainmentamongİŞKURtrainees(Distributionoftraineesbyhighestlevelofeducationcompleted,inpercentages)

12- BecauseİŞKURtraineescannotenrollintrainingsiftheyhavetakenacourseinthelasttwoyears,mostofthetrainingstendtohavebeencompletedmorethantwoyearsago,between2008and2010.

Source:Baselinesurveyofevaluationparticipants.

Men Women Total

Primary or less Lower sec. High school Voc HS College

35

30

25

20

15

10

5

0

8

16

1113

30 29 29

18

1315

2931 30

1613


4. İŞKUR trainees have limited priorexperience. About 37 percent of trainees have never worked before. Amongthose with prior work experience, 20percentworkedfor less thanoneyear(averageexperienceisfiveyears).Menare significantly more likely to haveworkedbeforethanwomen(71percentversus 53 percent). The limited workexperienceoftraineesispartlyexplainedby the age and gender compositionof trainees. More than 90 percent ofpeoplewithpriorexperienceworkedasprivatesectoremployeesandabout58percent had social security coverage.Traineesworked on average 54 hoursper week and received an average of TL 677permonth.

5. İŞKUR trainees are significantlyyounger, have less work experience,and are more educated thanjobseekers in urban areas.13 Women are overrepresented among İŞKUR

traineescomparedto theunemployed

butnotrelativetothe jobless(defined

asneitherworkingnorgoingtoschool)

(Table1).İŞKURtraineesaresignificantly

youngerandhavelessworkexperience

than the average unemployed or

jobless individual. Interestingly, İŞKUR

traineesarelesslikelytobelookingfor

a jobthantheaverageunemployed in

urban areas, and the incidence of job

informality among İŞKUR trainees (for

their most recent job) is higher than

that for the average wage employee

(42 percent versus 24 percent). The

most striking difference is that İŞKUR

traineesaresignificantlymoreeducated

thantheaverageunemployed,jobless,

or even employed individual in urban

areas.And this is not only due to the

younger profile of İŞKUR trainees,

as the difference remains even after

limitingtheurbansampleto20-to29-

year-olds.

13- ThecomparisonwithjobseekersisrestrictedtourbanareastomakeitcomparabletoİŞKURtrainees.


6. The significant difference ineducationalattainmentbetweenİŞKURtrainees and the average jobseeker inurban areas could be explained by anumberof factors: (i) İŞKUR traineesmusthaveatleastprimaryeducationandmeetotherskillprerequisitesdependingon the course; (ii) many courses aredesignedforpeoplewithmediumlevelsofeducation;(iii)inthecaseofexcessdemandforcourses,trainingproviderstend to prefer individuals with higher

levels of education;14 and (iv) moreeducatedjobseekersaremorelikelytoapplyforİŞKURtraining.15

7. Only half of İŞKUR trainees wereactivelysearchingforajobatbaseline. Over the last four weeks prior to theface-to-face survey, only about half ofİŞKURtraineesnotcurrentlyworkingorgoingtoschoolwereactivelysearchingforajob.Theshareofapplicantsactivelysearching increases significantly with

TABLE 1

ProfileofİŞKURtraineesrelativetotheunemployedandthejoblessinurbanareas(Percentages)

Note:Joblessisdefinedasnotworkingandnotgoingtoschool.

Source:Data for theunemployedand joblessare fromthe2010LaborForceSurvey(urbansample).Data for İŞKURtraineesarefromthebaselinesurvey.

14- Amongotherthings,moreeducatedindividualsareperceivedtobemorelikelytocompletetraining,whichiswhatdeterminespaymenttoproviders.Until2008,paymenttoproviderswastiedtojobplacementtargetsandmoreeducatedpeopleareperceivedtodobetterinthelabormarketregardlessoftheeffectivenessoftraining.

15-Thiscouldbebecausethecoursesaremoreattractive,theyaremoreinformedaboutthem,ortheyexpecttogetahigherreturnfromthem(althoughasshownlater,wefindlittleevidenceofthelatter).

Female

Married

Aged 15-24

High school or more

High school or more (20-29)

Worked before

Looking for a job in last 4 weeks

All Men Women All Men Women All Men Women

İŞKUR trainees Unemployed (urban) Jobless (urban)


8. Trainees that do search devotesignificanttimetoitandlookforhigh-paying, full-time wage employment.Although women are less likely tosearchforjobsthanmen,thosewhodotendtodevotemoretimetoitthanmen

the level of education (Figure 4). Thisshare also varies significantly acrossprovinces: Bayburt andMuş have thelowestratesofjobsearch(2percentand11percent,respectively),whileDüzce,Tekirdağ, and İzmir have ratesabove70percent.Menaresignificantlymore

(15hoursperweekversus10hoursperweek).Mostjobseekersarelookingforfull-time wage jobs (80 percent). Thereservationwage of İŞKUR trainees ishigh: only 38 percentwould accept amonthlywageofTL1000(theaverage

likelytobelookingforajobthanwomen(61 percent versus 42 percent), adifferencethatremainsaftercontrollingfordifferencesineducationlevel,whichmay reflect in part the greater timeconstraintthatwomenfaceasaresultoftheirhouseholdresponsibilities.

FIGURE 4

LimitedjobsearchamongİŞKURtrainees(%activelylookingforajobinthelastfourweeks,bygenderandlevelofeducation)


Primary or less Lower sec. High school Voc HS College

Men Women Total

80

70

60

50

40

30

20

10

0

72

55

32

43

53

3945

57

4249

6358 60

29

39


monthlywageinTurkey in2010),withno significant difference by level ofeducation. About 40 percent would not acceptajobwithoutsocialsecurity(20percent among those with less thanhigh school education) and, amongthose who would, their reservationwagewouldbeTL1800permonth(withno significant differences across levels ofeducation).Anumberoffactorsmayexplain the high aspirations of İŞKURtrainees,includingtheirhigheducationalattainment, family background (theycome from families with significantlyhigher per capita household incomethantheaverageunemployedinurbanareas),andworkexperience (onlyhalfof jobseekers come from a previousworkrelationship).

9. Most jobseekers use and valueİŞKURservicestohelpthemfindajob.Although the most common channelfor job search is family and friends,

the second most popular channelis İŞKUR, which is used by mostjobseekers (74 percent), followed bydirect contact with employers (Figure5).16Only12percenthaveusedprivateemploymentagencies,reflectinginparttheirlimitedavailability.Theinternetisalso frequentlyusedand, toa smallerextent, newspaper advertisements.İŞKURtendstobelessfrequentlyusedbywomenthanmen.JobseekerswithaprimaryeducationorlesstendtorelymostheavilyonİŞKURservices,whilecollegegraduatesuseittheleast.Thereare differences in the use of İŞKURservices across provinces,whichmayreflect inpart theavailabilityof İŞKURservices.17 Most jobseekers believethat applying directly to employers isthe best channel to find a wage job,with İŞKUR ranking immediatelyafter:28percentofjobseekersviewİŞKURasthebestchannel tosearch forawagejob.

16- Notethatmultiplechannelscanbeused.

17- In Düzce, Gaziantep, Hatay, and Tekirdağ,more than 92 percent of jobseekers use İŞKUR,while inMuş,Eskişehir,andElazığ,thepercentageisbetween30percentand40percent.


10.İŞKURtraineesattachagreatexantelabor market value to İŞKUR training,even relative toother trainings. About 80 percent of trainees believe İŞKURtrainingwillhelpthemdobetterinthelabor market. The expected impactsafteroneyearare large:relativetonotraining,traineesexpectİŞKURtrainingtoincreasetheirprobabilityoffindingajobby32percentagepointsonaverage(Table2)and to increasewagesby39

percent. The expected impacts arelarger for women than for men, buttherearenosystematicdifferencesbyage. Interestingly, trainees with lessthan secondary education completed,whomake up the minority of İŞKURtrainees,havehigherexpectationsthantrainees with higher education levels.And the expected impact of İŞKURtrainingissignificantlyhigherthanthatforothertypesofvocationaltraining.18

18-Thereisvariationacrossprovinces(traineesinAnkara,Antalya,andIspartahavethehighestexpectedimpacts)andcourses(traineesinwaiter,cook,nurse,andhairdressingcourseshavethehighestexpectedimpacts).

FIGURE 5

İŞKURservicesareusedbymostjobseekers(%useofdifferentjobsearchtools,bygender)


Employer ISKUR Friends and Private Printed ads Internet relatives employment agencies

Men Women Total

73 75 7468

7989

12

5060

71

86

11

43

6475

87

12

46

1009080706050403020100


11. Thehighexpectedvalueof İŞKURtraining is explained by the highperceived quality of training and thevalue perceived to be attached byemployerstoİŞKURcertificates. About 94 percent of trainees think İŞKURtraining improves job-readiness skillsand the knowledge of the profession(Figure 6). Almost all trainees areconfident about the quality of İŞKURtraining, particularly if provided bypublicinstitutions(50percentareveryconfidentabouttheirquality).Thequalityof İŞKUR training is also perceived tobe higher than that of other training,particularly other privately providedtraining. In addition to quality, 92percent of trainees also find training useful because they think employersdovaluetheİŞKURcertificate.Despitethe genuine value attached to İŞKUR

training,one-thirdoftraineesadmitthatamainreasonfortakingthetrainingisthe stipend provided to trainees (thisproportiondecreaseswith the levelofeducationofthetrainee).Andthefreenatureof İŞKUR trainingalsomatters:two-thirds of trainees would not payforothertrainingiftheydidnotgetanİŞKUR training spot, the main reasonbeing the lack of savings/cash andlimited access to credit.

Impacts of İŞKUR vocational training under review

12. The overall impact of İŞKURtraining (considered under this study)on employment is negligible, trainingdoeshaveasmallbutsignificantimpacton the quality of employment. Using the methods described in Section 2

TABLE 2

LargeexpectedimpactofİŞKURtrainingonthelabormarket(Probability of finding a job after one year under different scenarios, in percentage points)


Less than HS

High school

Voc. HS

College

Notraining

Notraining

Notraining

All Women Men

Training Training TrainingİŞKUR training

İŞKUR training

İŞKUR training


Trainees attach genuine value to İŞKUR training(%agreeingİŞKURtrainingisusefulfordifferentthings)

And perceive its quality to be higher than that of other options

(%confidentthequalityoftrainingwillbegood)

and measurements of employmentoutcomes taken approximately oneyear after the trainings (follow-upsurvey data), İŞKUR training coursesconsidered under this study arefoundto have no significant impact on thelikelihood of working, the number ofhoursworkedperweek,orthemonthlyincome received (Table 3). However,courses are found to have a smallbut significant impact on the qualityof employment. In particularly, beingassigned to İŞKUR training increasestheprobabilityofworkingintheformal

sector by 2 percentage points, withthe LATE impact being 3 percentagepoints. Given that 29 percent of thecontrol group is formally employedat follow-up, this LATE estimate isequivalent to a 10 percent increasein formal sector employment. Andtraining increases income from formal employment (byup to13percent)andas occupational quality.19 The overallimpact on employment, as measuredby the aggregate employment index(seeAnnex2), issmallbutstatisticallysignificant.

FIGURE 6

TraineesattachgenuinevaluetoİŞKURtraining


Job-readiness Knowledge of Employers’ Networking to Changing Stipend stills the profession value of find jobs profession certification

Agree Strongly agree

1009080706050403020100

94

41

95 92 90

79

344638 37

2811

ISKUR, public Other, public ISKUR, private Other, Private

Confident Very confident

120

100

80

60

40

20

0

97

53

8794

67

3643

21

19- Thisisacontinuousmeasurerangingfrom16(e.g.,domestichelpers)to90(e.g.,judges),andiscodedaszeroforindividualsnotworking(seeAnnex2fordetails).


13. Theimpactsoftrainingunderreviewonemploymentappearlargerformenover25yearsofage,butthedifferencesbyageandgenderarenotsignificant,nor are the differences by level ofeducation(Table4).Thestudyrandomlyassigned individuals to treatment and controlgroupswithineachcourseandwithingroupsofpeopledefinedbyage(underandabove25yearsofage)andgender, which allows a comparisonof the impacts across these groups.The consequence of using smallergroups(asopposedtothefullsample),

however, is that the estimates of theimpactsbecomelessprecise.Therearenosignificantdifferencesintheimpactoftrainingbetweenthosewithatleasta high school education completedandthosewithlessthanahighschooleducation,althoughthosewithpreviouswork experience or previous trainingseems to benefit more from İŞKURtraining(seeTableA2inAnnex3).

14. TheactualimpactsofİŞKURtrainingunder review differ significantly fromthe expectations formed before the

TABLE 3

İŞKUR’strainingshavenegligibleoverallimpactsonemploymentandsmallimpactsonjobquality

Notes:Robuststandarderrors inparentheses.*,**,and*** indicatesignificanceat the10,5,and1percent levels,respectively.SeeAnnex1fordetailsontheestimationmethodologyandAnnex2forvariabledefinitions.

ITT Estimate

LATE Estimate

Control Group Mean

Control Group Standard Deviation

Sample Size

Employed20 hours+

0.012

(0.013)

0.019

(0.018)

0.361

0.480

5529

Working at all

0.020

(0.013)

0.029

(0.018)

0.420

0.494

5497

OccupationalStatus

0.962*

(0.573)

1.452*

(0.828)

17.128

21.763

5418

MonthlyIncome

17.316

(12.271)

25.989

(17.624)

299.109

464.600

5396

Formal Income

22.166*

(11.965)

33.243*

(17.184)

257.887

448.160

5464

WeeklyHours

0.860

(0.680)

1.294

(0.980)

17.922

25.545

5439

Formal Work

0.020*

(0.012)

0.030*

(0.017)

0.293

0.455

5508

TransformedMonthlyIncome

0.121

(0.093)

0.182

(0.134)

2.541

3.516

5396

AggregateEmployment

Index

0.039*

(0.023)

0.059*

(0.033)

0.001

0.871

5497


trainings. On average, individualswerereasonablycloseintermsoftheirexpectationsofwhatemploymentlevelswouldbelikeintheabsenceoftraining:themeanresponsewasfora31percentchanceofbeingemployed;inpractice,36 percent of the control group isemployed(Table5).However,asshownearlier(Table2),individualssubstantiallyoverestimatethebenefitsfromtraining:theLATEestimateisfora2percentagepointincreaseinemploymentasaresultoftraining,whereasthemeanexpectedincreaseamongthoseassigned to thetreatment group is for a 32 percentage point increase.20 This overestimationoccurs for all gender and age groups. Three factors may help to explainthis mismatch between expectationsand reality: (i) the natural tendencyto overestimate future benefits and underestimate future costs of action takentoday,particularlyamongyouth;(ii) the different context in whichexpectations were formed relative tooneyearafterthetrainings;and(iii)theaverage trainee does not knowmuchabouttrainingsofferedbyİŞKUR.

15.ThesmallimpactsofİŞKURtrainingunderreviewonemploymentareinline

with the impacts of similar programsaround the world. A review of theevidencefromtheU.S.(Heckmanetal.1999) andEurope (Kluve2010) showsthatvocationaltrainingprogramshaveat most modest positive impacts on adult earnings,withlittleimpactsforyouth.Indevelopingcountries,areviewofnon-experimental studies by Betchermanet al. (2004) also foundmixed resultsof training the unemployed. The fewrandomized evaluations of vocational training programs in developing countries have focused on programsfor youth. Results in the DominicanRepublic(Cardetal.2011)showingnoimpact on employment and modestincreases in incomeare consistent with the earlier literature. Somewhatmore encouraging results are found in Colombia, particularly among youngwomen(Attanasioetal.2011).21Theseresults are, of course, context- andprogram-specific. And in some cases,they do provide clues as to whatmakes these programs more cost-effective (moreon this later).But theydosuggest thatwhile theseprogramscan be beneficial for (at least some)jobseekers, they are not apanacea toreducelarge-scaleunemployment.

20-IncontrastwithTable2,Table4 reportsexpectations for the treatmentgrouponly tooffsetanyconcernsthatindividualsinthecontrolgroupwhohadfoundouttheywerenotselectedfortrainingbythetimeoftheinterviewmightunderstatetheirexpectationsofthevalueoftraining.Inpractice,however,thecontrolandtreatmentgrouphaveverysimilarexpectations.

21-Amongyoungwomen, training increased the likelihoodof employmentby7percent andearningsby20percent.Traininghadnoimpactonthoseoutcomesamongyoungmen.


TABLE 4

NosignificantvariationintheimpactofİŞKURtrainingsonemploymentbygenderandage

Notes:Robuststandarderrors inparentheses.*,**,and*** indicatesignificanceat the10,5,and1percent levels,respectively.Testofequalityteststhenullhypothesisofequalityoftreatment impactacrossthefourageandgendergroups.SeeAnnex1fordetailsontheestimationmethodologyandAnnex2forvariabledefinitions.

ITT for males under 25

ITT for males over 25

ITT for females under 25

ITT for females over 25

p-value for testing equality:

Control Mean: young men

Control Mean: older men

Control Mean: young women

Control Mean: older women

Sample size

Employed20 hours+

-0.034

(0.030)

0.047

(0.031)

0.016

(0.025)

0.018

(0.019)

0.276

0.407

0.548

0.356

0.247

5529

Working at all

-0.021

(0.030)

0.069**

(0.029)

0.011

(0.025)

0.023

(0.020)

0.188

0.489

0.621

0.403

0.295

5497

OccupationalStatus

-0.495

(1.299)

2.997**

(1.332)

0.567

(1.161)

1.034

(0.917)

0.294

19.2

24.3

17.3

12.4

5418

MonthlyIncome

-12.112

(31.384)

35.304

(37.501)

14.938

(20.999)

26.052

(17.321)

0.712

347

518

269

187

5396

Formal Income

20.734

(30.160)

32.019

(36.733)

15.571

(20.519)

23.058

(16.788)

0.982

275

458

238

161

5464

WeeklyHours

-0.952

(1.626)

2.917*

(1.772)

0.952

(1.355)

0.761

(1.001)

0.457

20.1

27.3

18.1

12.0

5439

Formal Work

0.003

(0.028)

0.047

(0.032)

0.017

(0.024)

0.019

(0.018)

0.768

0.307

0.466

0.295

0.196

5508

TransformedMonthlyIncome

-0.241

(0.223)

0.383

(0.244)

0.140

(0.181)

0.174

(0.141)

0.268

2.90

3.94

2.50

1.69

5396

AggregateEmployment

Index

-0.029

(0.052)

0.112*

(0.058)

0.046

(0.045)

0.034

(0.035)

0.340

0.070

0.381

-0.015

-0.215

5497


16. TheimpactsofİŞKURtrainingunderreviewonothermeasuresofindividualand household well-being are equallymodest. One year after the trainingis completed may not be enoughfor the full impact of the training onemploymenttomanifestitself.Andthebenefitsoftrainingcanextendbeyondemployment,particularlyamongyouth(e.g.,throughreducedcrimeorconflict)(Heckman et al. 1999). To this end,the study collected information fromevaluation participants on future jobsprospects as well as other measuresof individual and household well-being. Indeed,more people expect tobeemployedintwoyears(54percent)

thannow(42percent),but there isnodifference between those who wereassigned to training and those whowere not (Table 6). Training does nothave an impact on trainees’ mentalhealth, but it makes trainees assesstheir well-being somewhat higher. Intermsofhouseholdwell-being,trainingdoes not have an impact on currenthousehold income, but it seems toincrease long-termhouseholdwelfare,asmeasuredbyadurableassetindex.Finally,thereisnoevidencethattrainingempowersindividualsinthehouseholdorinfluencestheirgenderattitudes(seeTableA3inAnnex3).

TABLE 5

Bigmismatchbetweenexpectationsandrealities

Notes: Individualexpectationsarethoseofthetreatmentgroupatbaseline.ActualcontrolemploymentlevelandtheLATEestimateforemploymentimpactarefortheoutcomeofworking20hoursormoreperweek.

Full Sample

Males under 25

Males 25 and over

Females under 25

Females 25 and over

ActualControl

EmploymentLevels

36.1

40.7

54.8

35.6

24.7

Expected percent chance

of being employed ifnot trained(Std. dev)

31.3

(24.1)

35.4

(24.6)

33.5

(26.1)

31.5

(22.5)

27.7

(23.5)

Employment Levels(%)

ActualImpactLATE

Estimate(std. error)

0.019

(0.018)

-0.044

(0.047)

0.081

(0.054)

0.021

(0.033)

0.022

(0.027)

IndividualExpected ImpactMean

(Std. dev)

0.324

(0.275)

0.297

(0.278)

0.302

(0.280)

0.329

(0.266)

0.346

(0.277)

Treatment Impact (Prop.)


Why does the training under review have a limited impact on employment and what types of training work better?

17.Thissectionexaminesthereasonsbehind the limited impact of trainingunder review on employment andidentifies the types of training thatwork better. The study investigatesfour channels through which trainingisexpectedtoaffectemployment(seeSection 2): (i) course attendance andcompletion; (ii) thequalityof training;(iii)thevalue-addedoftraining;and(iv)thelabormarketcontext.

18. Course attendance and completion rates are not good indicators of howtrainees value courses. As noted earlier, 77 percent of those selectedfor treatment attended the coursebeyondthesecondday,and72percentcompleted it. The impact estimates inTable 1 already adjust for attendance.The impact of training does not varysignificantly with the percentage oftrainees that attend the course andcompleteit(Table6).Thisresultsuggeststhatcourseattendanceandcompletionrates are not good indicators of theextent to which individuals taking thecourse value it in practice.

TABLE 6

Modestimpactsoftrainingonothermeasuresofindividualandhouseholdwell-being


ITT Estimate

Control Group Mean

Control Group Std. Dev.

Sample Size

MHI-5 Mental Health(higher better)

-0.060

(0.092)

18.508

3.410

5437

Expected Prob.Of Workingin 2 years

0.923

(0.915)

54.1

33.4

4878

Individual outcomes Household Outcomes

HouseholdIncome in last

year

485.524

(392.553)

21711

14674

5396

Subjective Well- Being in

5 years

0.061

(0.053)

5.822

1.980

5289

CurrentSubjective Well- Being

0.066*

(0.040)

4.436

1.514

5508

DurableAssetIndex

0.106**

(0.044)

-0.082

1.736

5495


19. The quality of training: privatelyprovided and competitive courses have a large impact on employment.The median course length is 320hours, which does suggest enoughhourstoenablelearningtotakeplace.Surprisingly, longer courses have lessimpact on employment than shortercourses (the difference is small,however; see Table 7). One possibleexplanation is that individuals reducejobsearchwhilsttakingpartintraining,so individuals in longer training courses have had less time to look for jobs.Havingmoreexperiencedoreducatedtrainersdoesnotseemtomatter,whichmayjustindicatethatteachereducationand experience explain little of theactualvariationinteachereffectiveness(HanushekandRivkin2006).

20. Both private provision and

competition yield higher impacts.

Previousnon-experimentalliteraturein

developed countries has found some

evidencetosuggestthattheimpactsof

training arehigher for trainingoffered

by private providers (Jespersen et al.

2008).Possiblereasonsarethatprivate

training providers are more responsive

to private sector employer demand

and/or they potentially face more

competition and thus must increase

quality in response. Table 8 shows

that İŞKUR courses offered by private

providersaswellascoursesofferedby

providersthatfacecompetition(having

two or more competitors) do yield

higherimpactsonemployment.


TABLE 7

Coursecharacteristicsassociatedwithbetterimpacts

Notes:Robuststandarderrors inparentheses.*,**,and*** indicatesignificanceat the10,5,and1percent levels,respectively.SeeAnnex1fordetailsontheestimationmethodologyandAnnex2forvariabledefinitions.Thecompetitionvariable isbasedonthequestiontoprovidersaboutthenumberofcompetitorstheyface:(i)zero;(ii)one;(iii) twotofive;(iv)sixtonine;or(v)10ormorecompetitors.

Treatment Interaction with:

Proportion assigned to course who attended it

Proportion assigned to course who completed it

Proxies for course quality

Course length above 320 hours

Average teacher experience greater than 12 months

Percent of course teachers with tertiary education

Course has two or more competitors

Course offered by private provider

Measures of what trainees thought course did

Proportion who thought course taught new technical skills

Proportion who thought course certified skills they had already

Proportion who thought course taught new job finding strategies

Proportion who thought course made them aware of new jobs

Measure of labor market demand

Provincial unemployment rate is above median

Differential Impact on:

Employed20+ hours

0.114

(0.092)

0.055

(0.083)

-0.042*

(0.024)

0.009

(0.026)

-0.000

(0.000)

0.040

(0.025)

0.044*

(0.023)

-0.087

(0.093)

-0.012

(0.108)

-0.025

(0.090)

-0.052

(0.072)

0.013

(0.023)

Control Mean (Std. Dev.) of Interact. Var.

0.765

(0.140)

0.723

(0.164)

0.423

(0.494)

0.418

(0.493)

65.0

(43.2)

0.674

(0.468)

0.348

(0.477)

0.796

(0.127)

0.842

(0.103)

0.604

(0.147)

0.449

(0.183)

0.463

(0.499)

SampleSize

5497

5497

5494

4833

4833

4833

5494

5497

5497

5497

5497

5497

AggregateEmployment

Index

0.173

(0.172)

0.118

(0.155)

-0.082*

(0.044)

0.019

(0.048)

-0.000

(0.001)

0.083*

(0.048)

0.117*** (a)

(0.043)

-0.209

(0.185)

-0.096

(0.213)

-0.067

(0.166)

-0.153

(0.140)

0.033

(0.044)


21. But it is really the combinationof private provision and competition thatbringsthehighestpayoff?Table8exploresfurthertheinteractionbetweencompetition and private provision. There is a large and highly significantpositive treatment effect for courses offered by private providers facinglots of competition: 17 percent of thecoursesevaluatedfitthesecriteria,andthe treatment impact is 9 percentagepointsforemployment,and0.2standarddeviations for theoverall employmentindex.Thusprivatecoursesthatdonot

facemuchcompetitionalsodonotoffersignificantimpacts,butneitherdopubliccoursesfacinglotsofcompetition.Thissuggests that it is important to haveboth the performance spur providedbycompetitionandtheincentivesandabilitytorespondtothiscompetitiontogeneratelargeimpactsonemployment.It isworth noting that this result is insharpcontrastwith theperceptionsoftrainees(beforecoursesstart),whoseeprivatecoursesashavinglowerqualitythanpubliccourses.

TABLE 8

Itisreallythecombinationofprivateprovisionandcompetitionthatworksbest?


ITT Impact for:

Public Course, 0 or 1 competitor

Public Course, 2 to 5 competitors

Public Course, 6+ competitors

Private Course, 0 or 1 competitor

Private Course, 2 to 5 competitors

Private Course, 6+ competitors

p-value for testing equality of impact

Sample Size

Proportion of samplein this group

0.24

0.21

0.23

0.09

0.07

0.17

Employed20+ hours

-0.027

(0.025)

0.014

(0.027)

0.012

(0.024)

0.002

(0.030)

-0.049

(0.043)

0.090***

(0.031)

0.064

4850

AggregateEmployment Index

-0.051

(0.045)

0.025

(0.051)

0.040

(0.048)

0.045

(0.069)

-0.045

(0.054)

0.203***

(0.055)

0.013

4820


22. Private provision and competition dohaveagenuinemarketvalue. Table A4inAnnex3showsthatthesecoursesdodiffer fromother types in termsofcharacteristics and applicants. One ofthekeydifferencesisthatthesecoursesaremore focused on accounting thanothertypesofcourses,andaccountingcourses have a larger impact onemployment than courses in otherfields. However, privately providedcourses, particularly those offered byproviders that face intensecompetition,still yield a significantly higher impactonemploymentthanothercoursesaftercontrolling for differences in course and trainee characteristics (Table A5in Annex 3). This evidence suggeststhatprivateproviders,especiallythosefacing more competition, have betterincentivestoofferhighqualitycoursesand services more attuned to labor market needs.

23. One year after the trainings,mosttrainees in thestudystillattachvalue-addedto İŞKURtrainingbut thatdoesnot seem to be enough to make adifference in the labor market. One yearaftercompletingthetraining,mostcourse participants believe training providedsomevalue-added,particularlyasasignalingdevicetoemployers:84percentofparticipantsthinkthecoursescertifiedskillstheyalreadyhad,and80percent think they taught new skills,while60percentsaythecoursehelped

with job-finding strategies and 45percentwithmakingthemmoreawareof job opportunities (Table 6). Those“assessments”are lowercompared tothesituationbefore thecoursestarted(Figure8),butstillyieldhighmarksforİŞKURtrainingasperceivedbycourseparticipants.However,therealityisthatthosewho value İŞKUR trainingmoredonotdobetterthanthosewhovalueitless.Theseresultssuggestthatthevalueattachedto İŞKURtrainingmayreflectotherattributesthanjustanincreaseinjobplacementprobabilityandmaybeduetothestrongeffectoflaborsupplyonemploymentoutcomes.

24. Thelabormarketcontextdoesnotseem to matter much. Although atthe time of training there was a widevariationinunemploymentratesacrossthe23evaluationprovinces(between5percentand17percent),traineesfromhighunemploymentprovincesdidnotdo better or worse than those fromlow unemployment provinces. TheevaluationofİŞKURtrainingtookplaceinthecontextofaremarkablesurgeinemployment after the crisis. Althoughtheunemploymentrateisnottheonlyindicatoroflabormarketperformance,this result suggests that the smallimpactoftrainingonemploymentwasnotdrivenbythebuoyantlabormarketat the time the follow-up data werecollected(early2012).


Cost-benefit analysis of İŞKUR training under review

25. İŞKUR courses offered by privateproviders facing competition have alarge net return. This section looksat how the benefits of İŞKUR trainingcomparetoitscostsandidentifiesthetypes of İŞKUR training that aremostcost-effective.

26. Fromtheprovidersurvey,themeanestimatedcost foran İŞKURcourse is1,574TLperperson(privatelyprovidedcourses have a mean cost of 1,792TL per person compared to 1,455 TLfor publicly provided courses). İŞKURcoursesprovidedbyprivateprovidersfacing six or more competitors havea mean cost of 1,771 TL per person.Participants receive a stipend of 15TL per daywhile they are in training;for an average course length of 57days, this equates to a further cost of855 TL. Thus the average cost to thegovernment of providing a course is 2,429TLperperson,2,671TLforprivateand competitive courses.

27. The LATE estimate of the overallgain in monthly income (Table 2) is

26TL (although this isnot statistically

significant).Assumingthecourseleads

to a permanent increase in income of

this amount, it would take 93months

forthegaininincometooffsetthecost

of the course, and the annual internal

rate of return (IRR) over 30 years of

thesegainswouldbe12percent.Ifthe

gains depreciate at 10 percent a year,

the IRRwouldbe3percent.However,

given that the point estimate of the

gain is insignificant, we cannot rule

out zero returns. Training was more

effectiveforprivatelyprovidedcourses

offered by providers facing significant

competition.TheLATEestimateofthe

overallgaininmonthlyincomeforthis

group is a statistically significant 128

TLpermonth.Basedonthisestimated

gainpersisting,itwouldtake21months

forthegaininincometooffsetthecosts

ofprovision,andtheIRRover30years

would be 58 percent (or 48 percent

with 10 percent per year depreciation

ofthegains).Bywayofcomparison,in

the evaluation conducted inColombia

(Attanasioetal.2011),coursescostan

average of US$750 and hadan IRR of

22-35percentforyoungwomen,andno

measureablebenefitsforyoungmen.



SECTION 4:STRENGTHENING THE IMPACT OF İŞKURTRAINING AND SERVICES

1. İŞKUR has come a long way since2008, increasing the coverage andquality of vocational training. As part of the2008 labor reform,participationin ALMPs was opened to all registered unemployed regardless of whetherthey are eligible for unemploymentinsurance. The number of vocationaltraineesincreasedfrom30,000in2008to 464,000 in 2012, representing 19.6percentoftheregisteredunemployed.İŞKUR has been working to improvethequalityoftrainingthroughanumberof measures. One such measure hasbeen the introduction of the NationalQualifications System, includingthe development of 294 nationaloccupational standards in 16 sectors (127 are under implementation).Also in 2011 the selection of trainingproviders introduced specific qualityand performance criteria (not justcost).In2012,İŞKURstartedhiringjoband vocational counselors to advise jobseekersonoccupationalchoiceandtraining courses.

2.AnewregulationpublishedinMarch2013 introduces a number of important initiativestoimprovetheeffectiveness

of vocational training and otherservices. In terms of training, newmodules (not exceeding 15 percentof thecurriculacontent)willbeaddedto training programs to improve employability of participants (e.g., jobsearchskills,interviewtechniques,andbasic skills). Also past job placementperformance and accreditation by theVocationalQualificationInstitution(VQI)willbemorerewardedwhenselectingprivate providers. Service providers are alsorequiredtouselicensinginstitutions(accreditedbyVQI)during testingandcertification of participants of training programs. All training programs for whichVQI has approvedoccupationalstandardswillbedevelopedinlinewiththenationaloccupationalstandards.

3. Service providers with a low jobplacementperformancewillberequiredto provide additional services. Providers ofgeneraltrainingarenowalsosubjectto jobplacement requirements,whichincrease to: at least 50 percent oftrainees finding employment for aperiod that cannot be lower than theduration of the course or at least 120days.Asanincentivetohireandkeep


young and female trainees employedin occupations for which they aretrained, the initial 6-month period inwhich employers’ social contributionsforyoungandfemalehiresarepaidbythegovernmentisnowextendedtoanadditional36monthsiftheyareISKURtraineesandtheyareemployed in theoccupationsforwhichtheyaretrained.The new regulation also links trainingto jobsearch: traineeswhorefuse joboffers suitable to their characteristicsare not allowed to benefit from anyALMP for 24months. Finally, job andvocational counselors will be more activeintheselectionofparticipantsoftraining programs.

4. These important recent reforms,which are not captured by this study,already build on the results of thisevaluation. Thisfinalsectionbuildsonthesegoodinitiativesandachievementsto suggest some options to furtherstrengthen the employment impact ofİŞKURtrainingandservicesbasedon:(i)theresultsoftheevaluation;and(ii)lessons from international experience.Morework isneededto furtherdefinethese(andotherpolicy)options.Reformcanonlyhappengradually,introducinga few new initiatives and evaluating thembeforemovingtothenextsetofreforms.ThisistheapproachİŞKURis

takingandthisevaluationillustratesit.

5. Improving the relevance of skills training. The study find that the lowoverall impactofİŞKURtrainingunderreview on employment is likely to bepartly due to the low value-added ofthesecoursesintermsoftheskillstheyhelp tobuild.As İŞKURdevelopsnewmodules to be added to the trainingprogram, it may consider puttingmore emphasis on behavioral skills,whicharehighlyvaluedbyemployersTurkey (McKinsey 2012). There isincreasinginternationalevidenceoftheemploymentimpactofbehavioralskillstraining (Almeida et al. 2012). Goingforward,itisalsoimportanttocontinuestrengtheningthelinkbetweenİŞKUR,trainingproviders,andlocalemployers,building on the experience of theProvincial Employment and TrainingCouncils.22

6. Incentivizing and supporting more job search. Only half of İŞKUR trainees inthestudywerelookingforajobbeforecourses started and not all of themusedİŞKURservices.Thissuggeststheneed to encourage more job searchandexpandemploymentservices (jobplacement, counseling, job-searchassistance).Thenewregulationlinkingtraining to jobssearch isastep in the

22-Thesecouncilswerecreatedin2008tobringtogetherthetrainingandlabormarketsidesatthelocallevel.Theyarechairedbytheprovincialgovernorsandhaverepresentationfromemployers,educators,andotherlocalactors,withİŞKURandMONEservingasthesecretariat.


right direction. International evidenceshows that it ismore cost-effective tofirst encourage jobseekers to look fora job and to assist them in this taskthrough employment services beforethey get any training than to offertrainingfirst(asinTurkey).Toencouragejob search, it is important to tie thereceipt of unemployment or socialassistance benefits to it—Turkey doesthis,butperhapscompliancecouldbeimproved.Employmentservices,whicharelimitedinTurkey,shouldbecentralto employment activation efforts. Forexample, in the U.K.’s jobcenter plus,all registered unemployed people areoffered some employment services(rangingfromminimumjobplacementservices to job-search assistance forthe hard-to-employ) and are requiredto take individual actions to find a job (i.e., the market test– receipt ofbenefits is conditional on that) beforeit is determined whether they needadditionalservices(includingtraining).

7. Defining the priority groups for training. The low overall impactof İŞKUR training under review onemployment could also be related towho İŞKUR is actually training andwhetherthesejobseekersaretheoneswhocanbenefitthemostfromtrainingrelative to other groups of jobseekers(and are most likely to be activelyseekingemployment).İŞKURistrainingthe most educated jobseekers. Aside

fromtheminimumrequirementtohavebasiceducation,thisismostlytheresultof the courses İŞKUR offers and theselectionofapplicantsbyproviders.Theevaluation does not show differentialimpactsbygenderandage,but theseresults refer to the current profile ofİŞKURtrainees(e.g.,highlyeducated)aswellasthetrainingandservicesİŞKURcurrently offers. Low-skilled workersaccountformostofthelaborforceandface the greatest jobs challenge. Andthey face barriers to productive jobsotherthanskills,includinginformation.Countries with well-developed publicemployment services (e.g., the U.K.,Germany, and Australia) serve alljobseekersthatregister,butthebulkoftheirresources(includingskillstraining)support hard-to-employ jobseekers.İŞKURhasstartedservingonesegmentof the hard-to-employ, namely thosereceiving social assistance benefits and able to work.

8. The focus of İŞKUR trainings onwomen and youth seems a prioriappropriate fromapolicyperspective.The lack of differential impacts oftrainingonemploymentbygenderandageistiedtothecurrentprofileofİŞKURtraineesaswell as to the trainingandservicesİŞKURcurrentlyoffers.Activityrates are especially low for womenand youth. International evidenceshowsthatwell-designedskillstrainingprograms have a higher return for


youththanothers,mainlybecauseitiseasiertolearnwhenyoung.SuccessfulprogramsforyouthintheU.K.,theU.S.,and several Latin American countries (e.g., the Jovenes program) targetdisadvantaged out-of-school youth(typically 15- to 29-year-oldswith lessthan a secondary education). And incountrieswithlowfemaleemployment,like Colombia (Attanasio et al. 2011)andtheDominicanRepublic(Ibarraran2012), well-designed programs foryouthcanhavealargepayoffforyoungwomen.

9. Better information on jobseekers to adjust services to needs. The newjob and vocational counselors hiredby İŞKURwill most certainly result ina better match of trainees to courses(a possible problem identified in thisevaluation),buttheydonotassesstheemployability of jobseekers and thustheirneedfortraining(orotherservices)to begin with. Countries with well-developedpublicemploymentserviceslike the U.K. and Australia do makean initial employability assessmentof jobseekers, which is then usedto “profile” jobseekers into differentgroupsreceivingdifferentemployment

support packages, with the bulk ofresourcesgoingtothehard-to-employ.

10. More contracting of services to the private sector while ensuring quality. The high return of İŞKURcourses offered by private providersfacing more competition suggests that increasing the share of coursessubcontracted to private providers and increasing competition among them would significantly increase theemployment impactof İŞKURtraining.However,itisalsoimportanttoensurethequalityofprovidersandmakethemaccountable for results. İŞKUR hasalready taken a number of measuresto increase the quality of providersthrough the selection process. İŞKURhas also recently extended jobsplacement requirements to providersof general training.There is still roomtostrengthenthecontractswithprivateproviders to improve the impact oftraining. And to avoid “creaming” offthe easy-to-employ by providers, andtotakeintoaccountthehighercostofhelping the hard-to-employ becomeemployed, the contracting of servicesforthemcouldbedoneseparately,asisdoneintheU.K.


REFERENCES

Abdulkadiroglu, Atila, Joshua Angrist,SusanDynarski,ThomasKaneandParagPathak(2011)“Accountabilityand Flexibility in Public Schools:Evidence from Boston’s Chartersand Pilots.” Quarterly Journal of Economics 126(2):699-748.

Rita Almeida, Jere Behrman, DavidRobalino(Editors).2012.The Right Skills for the Job?: Rethinking Effective Training Policies for Workers,WorldBank,Washington,DC.

Attanasio, Orazio, Adriana Kugler,CostasMeghir (2011) “SubsidizingVocational Training for DisadvantagedYouth inColombia:Evidence from a Randomized Trial”,American Economic Journal: Applied Economics3(3):188-220.

Benjamini, Yoav and Yosef Hochberg(1995) “Controlling the FalseDiscovery Rate: A Practical andPowerful Approach to MultipleTesting”, Journal of the Royal Statistical Society Series B, 57(1):289-300.

Betcherman, Gordon, Karina Olivas,and Amit Dar (2004) “Impacts ofActive Labor Market Programs:New Evidence from Evaluations withParticular Attention to Developing and Transition Countries. ”World

Bank Social Protection Discussion Paper Series 0402.

Betcherman, Gordon, M. Godfrey,S. Puerto, F. Rother and A.Stavreska, 2007, Global Inventory of Interventions to Support Young Workers Synthesis Report, World Bank.

Bruhn, Miriam and David McKenzie(2009), “In Pursuit of balance:Randomization in Practice in Development Field Experiments,”American Economic Journal: Applied Economics1(4):200-232.

Burbidge, John, Lonnie Magee andA. Leslie Robb (1988). “AlternativeTransformationstoHandleExtremeValues of theDependent Variable”Journal of the American Statistical Association83(401):123-127

Card,David,PabloIbarraran,FerdinandoRegalia, David Rosas-Shady, andYuri Soares (2011) “The LaborMarketImpactsofYouthTraininginthe Dominican Republic”, Journal of Labor Economics29(2):267-300.

Fink,Günther,MargaretMcConnellandSebatianVollmer(2012)“TestingforHeterogeneous Treatment Effects in ExperimentalData:FalseDiscoveryRisks and Correction Procedures”,Mimeo. Harvard School of PublicHealth.


Ganzeboom,HarryandDonaldTreiman(1996) “InternationallyComparableMeasures of Occupational Status forthe1988InternationalStandardClassification of Occupations”,Social Science Research 25(1):201-39.

Hanushek, Eric and Steven Ritkin(2006)“TeacherQuality”,pp.1051-78 in E. Hanushek and F. Welch(eds.)Handbook of the Economics of Education Volume 2. Elsevier,Amsterdam.

Heckman,JamesJ.,RobertJ.Lalondeand Jeffrey A. Smith (1999) “TheEconomics and Econometrics of Active Labor Market Programs”,Handbook of Labor Economics Volume 3, Part A,1865-2097

Ibarraran, Pablo, LauraRipani, BibianaTaboada, Juan Miguel Villa,Brigida Garcia (2012) “ Life Skills,Employability and Training forDisadvantaged Youth: Evidencefrom a Randomized Evaluation Design”, IZA discussion paper No. 6617.

Jespersen, Svend, Jakob Munch andLars Skipper (2008) “Costs andBenefits of Danish Active LabourMarket Programmes”, Labour Economics 15(5):859-84.

Kluve,Jochen(2010)“Theeffectivenessof European active labour market policy”, Labour Economics 17(6):904-18.

McKinsey&Company(2012), Education to Employment: Designing and System that Works, McKinsey&Company.

WorldBank (2010)“Turkey: InvestmentClimate Assessment—From Crisisto Private Sector Led Growth.”Report 54123-TR. World Bank,Washington,DC.

World Bank (2013a) “Managing LaborMarkets through the EconomicCycle,” Report 70130-TR. WorldBank,Washington,DC.

World Bank (2013b) “PromotingExcellence in Turkey’s Schools.”Report 77722-TR. World Bank,Washington,DC.


ANNEX 1: ESTIMATION METHODOLOGY

We canmeasure the intent-to-treat (ITT) effect of vocational training on a particularoutcomeofinterestbyestimatingthefollowingequation,analogoustoequation(1):

(1)

where i denotes individuals, s denotes a course*gender*age group (randomizationstrata),and,δi,sisabinaryvariableindicatingwhetherindividuali belongs to group s. Thiscontrolsforrandomizationstrata(BruhnandMcKenzie2009)aswellascontrollingforindividualswhoappliedformorethanonecourse(Abdulkadirogluetal.2011).

Sincewedonothavebaselinedataforallapplicantswhoansweredthefollow-upsurvey,andbecausesomeofthebaselinedatawerecollectedafterthecoursehadbegun,wedonotcontrolforbaselinevaluesoftheoutcomevariableofinterest.βisthentheaverageeffectofbeingselectedforavocationaltrainingcourseonthisoutcome.

WecanalsoestimatetheimpactofactuallycompletingtrainingbyreplacingAssigned to TrainingwithCompleted Trainingin(1),andinstrumentingthiswithtreatmentassignment.Undertheassumptionthatassignmenttotraininghasnoimpactonoutcomesforthosewho do not complete the course, and that there are no individualswhowould takecoursesonly if assigned to thecontrolgroup, thisyields the localaverage treatmenteffect(LATE).Thisistheimpactofcompletingtrainingforanindividualwhotakesuptrainingwhenheisselectedinthecourselottery,anddoesnottakeitupotherwise.Aconcernwith thisestimation is thepossibility thatsimplybeingselected foracoursemayaffectemploymentoutcomes,evenifindividualsdonottakethecourseoriftheydropoutafteronlyafewdays.

Inadditiontoestimatingtheoverall impactof training,weare interested inexploringtheheterogeneityofimpactstohelpunderstandwhethercertaintypesofcoursesofferlargerimpactsorwhethercertaintypesofpeoplebenefitmorefromtraining.ToestimateheterogeneitywithrespecttocharacteristicX,weestimate:

(2)

Coursecharacteristicsonlyvaryat thecourse level,soweclusterstandarderrorsbycourse.Recently,Finketal.(2012)havecriticizedrandomizedexperimentslookingforheterogeneityintreatmenteffectsfornotcontrollingadequatelyformultiplehypothesistesting.TheyrecommendtheuseoftheBenjaminiandHochberg(1995)approachwhichholdsconstantthefalsediscoveryrate(theexpectedproportionoffalselyrejectednullhypotheses).Weusethisapproachtoexaminewhichdimensionsofheterogeneityarerobusttothisconcern.


ANNEX 2:DEFINITIONS OF KEY OUTCOME VARIABLES

Employment outcomes

The following employment outcomes are defined based on data collected in thefollow-upsurvey.

Working at all: An indicator variable thattakes the value one if the individual hasworkedforcashorin-kindincomeatallinthepastfourweeks(andzerootherwise).

Employed 20 hours +: Anindicatorvariablethattakesthevalueoneiftheindividualiscurrentlyworkingfor20hoursaweekormore(andzerootherwise).

Weekly hours:Hoursworkedperweek inthe lastmonthemployed.This iscodedaszeroforindividualscurrentlynotworking,andtop-codedas100hoursperweek(the99thpercentileofthebaselineresponse)toreducetheinfluenceofoutliers.

Monthly income: Totalmonthlyincomefromworkinthelastmonth.Thisiscodedaszeroforindividualsnotworking,andtop-codedasthe99thpercentileofthecontrolgroupearningsdistribution(2500TL)toreducetheinfluenceofoutliers.

Transformed monthly income: Theinversehyperbolicsinetransformationofmonthlyincome fromwork in the lastmonth, log(y+(y2+1)1/2 ).This is intended tobemorerobusttooutliersthanlevelsofincomeandissimilartothelogarithmtransformation,butisalsodefinedwhenincomeiszero(Burbidgeetal.1988).

Occupational status: ThisiscodedbasedonworkoccupationusingtheinternationalmeasuresofsocioeconomicoccupationalstatusofGanzeboomandTreiman(1996).This isacontinuousmeasurerangingfrom16(e.g.,domestichelpers) to90 (e.g.,judges),andiscodedaszeroforindividualsnotworking.

Formal work: Thisisanindicatorvariablecodedasoneiftheindividualiscurrentlyworkinginajobcoveredbysocialsecurity(andzerootherwise).

Formal income: Thisismonthlyincomeearnedinjobscoveredbysocialsecurity.

Aggregate employment index:Astandardizedindexobtainedastheaverageofeachoftheabovevariables,aftereachhasbeenstandardizedbysubtractingthemeananddividingbyitsstandarddeviation.Thismeasureissettomissingforindividualswhoaremissingdataforthe“workingatall”variable;otherwise,itistheaverageoftheoverallemploymentvariableswithnon-missingdata.


Well-being measures

Expected probability of working in two years: Theexpectedchanceofhavingajobintwoyears’time,codedasmissingifanansweroutsideofthe0to100rangeisgiven.

Mental health index (MHI-5): This isa five-item index;ithasamaximumscoreof25andminimumscoreof 5.Higher scoresaredesirable in that they indicate theexperienceofpsychologicalwell-beingandtheabsenceofpsychologicaldistress.Individualsareaskedhowofteninthepastfourweekstheyhavedoneeachofthefollowing(eachansweredona5pointscale,where1denotesnoneofthetimeand5allofthetime):

•Beenanervousperson(reverse-coded)

•Feltsodowninthedumpsthatnothingcouldcheerthemup(reverse-coded)

•Feltcalmandpeaceful

•Feltdownheartedandblue(reverse-coded)

•Beenahappyperson

TheMHI-5isthesumoftheseresponses.

Current subjective well-being: Individualsareaskedonwhichstepofa10-stepCantrilladder(wherethepoorestpeoplestandonthefirststepandtherichestonthetenthstep)theythinktheirhouseholdstandstoday.

Subjective well-being in five years: ThestepoftheCantrilladderonwhichindividualsthinktheirhouseholdwillbeinfiveyears.

Total household income in past year: Incomefromallsources,topcodedasthe99th percentileofthecontrolgroupdistribution(74,000TL).

Transformed household income: The inverse hyperbolic sine of total householdincome.

Durable asset index: The first principal component of 15 indicators of householddurable asset and infrastructure ownership (own a gas or electric oven; own amicrowaveoven;ownadishwasher;ownaDVD/VCDplayer;ownacamera;haveDigiturk/Satelite;ownanairconditioner;ownaCDplayeroriPod;ownatelephone;ownacomputer;haveaninternetconnection;ownaprivatecar;ownataxi,minibus,orcommercialvehicle;ownabicycle;havefourormoreroomsintheirhouse).


TABLE A1

Summarystatisticsandtestingofdifferencesbetweentreatmentandcontrolgroups

Notes:Standarderrorsarerobuststandarderrors.*,**,and***indicatesignificanceatthe10,5,and1percentlevels,respectively.Samplesizesforthefollow-updatafortheindividualcharacteristicsmeasuredatbaselineareforthesampleinterviewedatbothbaselineandfollow-up;therearealsoindividualsinterviewedatfollow-upbutnotatbaseline.

Individual Characteristics (administrative data)

Female

Age

At least high school

Individual characteristics (baseline data)

Years of education

Has done previous training course

Household head

Household size

Married

Ever employed

Total years working for pay

Control Mean(S.D.)

0.629

(0.483)

27.1

(7.2)

0.724

(0.447)

11.3

(3.3)

0.264

(0.441)

0.134

(0.340)

4.09

(1.57)

0.346

(0.476)

0.631

(0.483)

3.38

(4.91)

N

5308

5308

5308

5255

5308

5276

5308

5033

5308

5277

Baseline Sample Follow-up Sample

Control Mean(S.D.)

0.623

(0.485)

27.0

(7.2)

0.724

(0.447)

11.3

(3.3)

0.265

(0.441)

0.133

(0.340)

4.10

(1.57)

0.351

(0.478)

0.626

(0.484)

3.33

(4.88)

N

5529

5529

5529

5008

5057

5027

5057

4797

5057

5027

TreatmentDifference(Std. Error)

0.005

(0.013)

-0.389**

(0.198)

0.009

(0.012)

-0.005

(0.069)

-0.007

(0.012)

-0.000

(0.008)

0.024

(0.040)

0.004

(0.011)

-0.021*

(0.012)

0.006

(0.111)

TreatmentDifference(Std. Error)

0.010

(0.013)

-0.306

(0.194)

0.005

(0.012)

0.014

(0.071)

-0.008

(0.012)

-0.003

(0.008)

0.011

(0.040)

0.002

(0.012)

-0.020

(0.013)

0.006

(0.113)

ANNEX 3: ESTIMATION RESULTS


TABLE A2

Differences in the impact of İŞKUR training by level of education, previous workexperience,andtraining

Notes:Robuststandarderrors(se)inparentheses.*,**,and***indicatesignificanceatthe10,5,and1percentlevels,respectively.“Treatmentb”istheLATEestimateoftheimpactofİŞKURtraining,while“Interactionb”istheLATEestimateoftheinteractionofeducation/previousworkexperience/previoustrainingwithtraining;e.g.,theimpactoftrainingfortraineeswithatleastahighschooleducationcompleted.SeeAnnex1fordetailsonestimationmethodologyandAnnex2 for variable definitions.


Years education

Previous training

Ever worked

se

(0.042)

(0.042)

(0.039)

(0.006)

(0.006)

(0.006)

(0.043)

(0.042)

(0.040)

(0.038)

(0.037)

(0.036)

b

-0.032

-0.006

0.013

-0.002

0.001

0.000

0.046

0.093**

0.065

0.027

0.058

0.062*

Interaction Sample sizeTreatment

Working at all

Employed 20 hours +

Formal Emp.

Working at all

Employed 20 hours +

Formal Emp.

Working at all

Employed 20 hours +

Formal Emp.

Working at all

Employed 20 hours +

Formal Emp.

se

(0.036)

(0.035)

(0.032)

(0.070)

(0.069)

(0.065)

(0.022)

(0.022)

(0.021)

(0.028)

(0.028)

(0.025)

b

0.054

0.025

0.022

0.050

0.009

0.025

0.019

-0.004

0.014

0.018

-0.012

-0.003

5497

5529

5508

5497

5529

5508

5497

5529

5508

5497

5529

5508


TABLE A3

ImpactsofİŞKURtrainingonempowermentandgenderattitudes

Notes:Robuststandarderrors inparentheses.*,**,and*** indicatesignificanceat the10,5,and1percent levels,respectively.Testofequalityteststhenullofequalityoftreatment impactacrossthefourage*genderstrata.Numberofdecisionsmade isoutof7.Themaximumnumberofpro-genderequalitybeliefs is5.SeeAnnex1 fordetailsonestimationmethodologyandAnnex2forvariabledefinitions.

Panel A: Pooled Sample

ITT Estimate

Control Mean DepVar

Sample Size

Panel B: Age*Gender Heterogeneity

ITT for males under 25

ITT for males over 25

ITT for females under 25

ITT for females over 25

p-value for testing equality:

Control Mean: young men

Control Mean: older men

Control Mean: young women

Control Mean: older women

Is householdhead orspouse

0.004

(0.010)

0.188

5472

-0.002

(0.021)

0.002

(0.031)

-0.004

(0.013)

0.016

(0.015)

0.785

0.14

0.54

0.06

0.13

Number ofdecisions

made

0.058

(0.044)

5.601

5515

0.024

(0.071)

0.056

(0.088)

0.105

(0.088)

0.042

(0.086)

0.910

6.3

6.1

5.5

5.1

Number ofpro-gender

equal beliefs

0.057

(0.036)

2.540

5469

0.179**

(0.079)

0.072

(0.086)

0.053

(0.069)

-0.010

(0.060)

0.300

2.0

2.1

2.8

2.8


TABLE A4

Whatisdifferentaboutcompetitiveandprivatecourses?

Notes:Samplemeans shown. *, **, and*** indicate statisticallydifferent frompublicproviders at the10, 5, and1percentlevels,respectively.‘+,++,and+++indicatestatisticallydifferentfrom0or1competitorcoursesatthe10,5,and1percentlevels,respectively.Publiclyprovidedcoursestestedagainstprivatecourses;morecompetitivecoursesagainsttheleastcompetitivecourses;thelastcolumnsimplyshowsmeansfortheprivatecourseswith6+competitors,butdoesnottesttheseagainstanyothergroup.

Course Characteristics

Course length (days)

Course length (hours)

Accounting course

Professional course

Craftsman course

Technical course

Service course

Course in Istanbul

Average teacher experience greater than 12 months

Percent of course teachers with tertiary education

Individual Characteristics (administrative data)

Female

Age


Individual characteristics (baseline data)

Years of education

Has done previous training course

Household head

Household size

Married

Ever employed

Total years working for pay

Raven test score (out of 12)

Numeracy score (out of 4)

Tenacity

2 to 5competitors

59

344+++

0.12+++

0.17+++

0.22

0.15+++

0.16+++

0.24+++

0.39+++

80.1+++

0.55+++

26.7

0.75+++

10.2+++

0.23

0.14+++

3.72++

0.28+

0.62+++

3.60+++

6.41+++

3.45+++

8.42++

0 or 1competitor

60

375

0.19

0.05

0.24

0.11

0.33

0.09

0.24

59.8

0.63

27.1

0.69

9.6

0.21

0.10

3.88

0.31

0.49

2.57

5.44

3.21

8.31

PublicProvider

55

328

0.13

0.11

0.23

0.19

0.24

0.27

0.45

71.9

0.61

27.0

0.66

9.7

0.25

0.12

3.86

0.30

0.57

3.17

6.10

3.31

8.36

6 + competitors

50+++

287+++

0.36+++

0.14+++

0.06+++

0.20+++

0.09+++

0.26+++

0.58+++

58.9

0.64

26.2+++

0.81+++

10.8+++

0.26+++

0.11

3.73+++

0.26+++

0.57+++

2.84+

6.26+++

3.48+++

8.32

6+ competitorsand private

60

352

0.48

0.23

0.00

0.10

0.05

0.29

0.51

60.0

0.68

25.8

0.89

11.2

0.20

0.10

3.58

0.24

0.58

2.64

6.29

3.50

8.39

PrivateProvider

60***

353***

0.45***

0.14***

0.02***

0.07***

0.14***

0.12***

0.34***

51.2***

0.66***

26.5***

0.85***

10.9***

0.20***

0.10**

3.64***

0.27*

0.55

2.64***

5.93*

3.51***

8.35


TABLE A5

The“value-added”ofprivateprovisionandcompetitionremainsaftercontrollingfordifferencesincourseandapplicantcharacteristics

Notes:Robuststandarderrors inparentheses.*,**,and*** indicatesignificanceat the10,5,and1percent levels,respectively. See Annex 1 for details on the estimationmethodology and Annex 2 for variable definitions.We usetwoalternativemethods toestimate thedifferential impactofprivatecoursesandprivatecourseswith6+providerscontrollingfordifferencesinthecoursesandtraineescharacteristicsinTableA4.First,weestimateapropensityscoreforthelikelihoodofbeinginthosetypeofcoursesasafunctionofthecharacteristicsinTableA4.Wethenre-runequation(2)inAnnex1weightingcoursesbythispropensityscore,whichyieldsthedifferentialimpactestimatecalled“weightedbypropensity-score.”Thealternativemethodinvolvesre-runningequation(2)inAnnex1andcontrollingfortheinteractionbetweenthetreatmentvariableandthepropensityscore.

Treatment Interaction with:

Course offered by private provider

Unweighted (as reported in Table 6)

Weighted by propensity-score

Controlling for interaction of treatment with propensity score

Course offered by private provider with 6+ competitors

Unweighted

Weighted by propensity-score

Controlling for interaction of treatment with propensity score

AggregateEmployment

Index

0.117***

(0.043)

0.112*

(0.059)

0.107*

(0.054)

0.190***

(0.058)

0.232***

(0.070)

0.214***

(0.065)

Employed20+ hours

0.044*

(0.023)

0.058*

(0.030)

0.061**

(0.029)

0.095***

(0.034)

0.127***

(0.037)

0.110***

(0.036)

Differential Impact on:

Documents

World Bank Document · better job opportunities. However, until recently these opportunities have not kept pace with the increasing supply of (mostly) young and low-skilled workers