OECD LEED Seminar: What works, what doesn’t? Evidence-based approachesOECD, 15th October 2014

The long-term impact of employment services

David GrubbEmployment Analysis and Policy Division

2

Q. Do intensive case-management interviews with jobseekers push them into short-term, unstable jobs?

A. Rigorous random-assignment evaluations have identified large and relatively long-term impacts

Background – employment services / work-first

3

• Target group: unemployment benefit recipients in 1989

• Restart process “combines counseling and encouragement with tighter enforcement of conditions necessary qualify for unemployment benefits”

• The control group was interviewed after 6 months of unemployment (the norm at the time); the treatment group after 12 months

• By month 12, the treatment group’s unemployment rate was about 10 percentage points higher

• There was little evidence of an longer-term impact for women. But for men, the treatment group’s unemployment rate in years 3, 4 and 5 stayed about 6 points higher (c.40% instead of 34%).

• Gender difference: in response to interviews, women more often left the labour force while men more often started work

UK Restart interviews

4

• Target group: AFDC recipients (lone parents)

• Meta-analysis of 24 evaluation studies covering 64 programs that operated from 1983 to 1996

• The interventions studies were classified as “work first” (referral to jobs, work experience) vs. “human capital” (training)

• For “work first” the average impact on total quarterly earnings was immediately positive. For “human capital”, the impact was at first zero. In both cases, the impact increased for some time but started to decline after 3 years, and fell to zero by year 6. (Based on other studies, the authors suggest that the impact of voluntary, high-cost, training programs may “linger longer”.)

• Over 24 quarters (6 years), the impact on earnings of “work first” ($4000) was much greater than that of “human capital” ($1500)

US Welfare to Work experiments

5

• SSP (at random assignment in 1993/94) offered parents on welfare for a year a large cash supplement (up to 3 years) if they worked 30 h/week

• SSP Plus offered, in addition, an employment plan, resumé service, job clubs, workshops, coaching, and job leads (find/retain/regain a job)

• One year into the program, SSP participants were twice as likely to be working 30 h/week as controls. In total they earned $3400 more (+20%). But when the cash supplement stopped, the impact stopped.

• In years 1-2, SSP Plus had little impact over regular SSP. But by years 4 to 5Q2, SSP Plus participants were earning about $100/month more.

• In total SSP Plus participants earned about $3200 more up to year 5Q2, and probably kept earning $100/month more – so that the employment services had more total impact than 3 years of cash supplements did

• Analysis suggests SSP Plus helped those who secured employment to stay in work for a longer time. SSP Plus is likely to have cost little more than regular SSP (the main impact on the services received was on: “Took part in job-search program such as job club or job-search”).

Canada’s Self-Sufficiency Project Plus

6

• New UI recipients from July to December 2009 (in deep recession)

were referred to a benefit eligibility interview which also offered an

“array” of services designed to enhance the quality of job search

• For the treatment group compared with control:

– the % exhausting regular UI benefits (i.e. unemployed for six months)

fell by 10 points (-16%)

– total earnings over 18 months increased by $2600 (+18%)

– employment was 25% in Q2 after random assignment, and still 14%

higher in Q6

• “the largest portion of the impacts is attributed to the effectiveness

of the services..” “conditional on employment, treatment group

recipients had higher earnings”.

Re-employment services in Nevada

7

• Controlled experiments have often found a large multi-year positive impact from employment services – though:– There are also findings of no/negative impact: women, in the

UK experiment; National Employment Action Plan in Ireland

– Declining returns may apply: plausibly impact is greatest when the alternative is no or only limited services

– Quality matters – the UI program in Nevada had more impact than UI programs in several other States.

– Counselling is not pure “work first” - it delivers assistance and can act as a gateway to more-expensive services

• Despite the caveats: an ongoing/repeated service offer to out-of-work clients – which is delivered by a well-managed PES – plausibly has a cost-effective lifetime impact on unemployment, employment and earnings

Conclusions about the impact

8

• Non-experimental evaluations may not be reliable

• Outcomes need to be tracked for all individuals who were assigned to treatment vs. control groups – this requires significant sample sizes

• Short-term impacts may or may not fade to zero in the following few years – or impacts may start small, but then become significant after two years. To limit the cost of tracking long-term outcomes, administrative data are needed.

• Observed impacts are influenced by the broader policy context, target group, jobseeker expectations, etc. The interpretation of findings in one specific context and their application to a different context requires some expertise and judgement.

Accurate evaluation is difficult

9

Slide 3: Dolton, P. and D. O’Niell (2002), “The Long-Run Effects of Unemployment Monitoring and Work-Search Programs: Experimental Evidence from the United Kingdom”, Journal of Labour Economics, Vol. 20, No. 2, Pt. 1.

Slide 4: Greenberg, D., K. Ashworth, A. Cebulla and R. Walker (2004), “Do Welfare-to-Work Programmes Work for Long?”, Fiscal Studies, vol. 25, no. 1.

Slide 5: Michalopoulos, C., D. Tattrie, C. Miller, P. Robins, P. Morris, D. Gyarmati, C. Redcross, K. Foley and R. Ford (2002), Making Work Pay: Final Report on the Self-Sufficiency Project for Long-Term Welfare Recipients, Social Research and Demonstration Corporation.

Slide 6: Michaelides, M. (2013), Are Reemployment Services Effective in Periods of High Unemployment? Experimental Evidence from the UI System, University of Cyprus, mimeo., December; and Michaelides, M. E. Poe-Yamagata, J. Benus, and D. Tirumalasetti (2012), Impact of the Reemployment Eligibility Initiative In Nevada, Impaq International.

Slide 7: McGuinness, S., P. O’Connell, E. Kelly and J. Walsh (2011), Activation in Ireland: An Evaluation of the National Employment Action Plan, ESRI Research Series 20.

References