A net impact analysis of active labour programmes in Hungary

Economics of Transition, Volume 5 (2), 453-484, 1997

A net impact analysis of active labour programmes in Hungary

Christopher J. O’Leary’

WE Upjohn Institute for Employment Research 300 South Westnedge Avenue Kalamazoo, Michigan 49007, USA Tel: (1 ) 616 343 5541 Fax: (1) 616 343 3308 E-mail: oleary @we.upjohninst.org

Abstract

This paper presents estimates of the impact on re-employment and earnings of the two most popular active labour programmes used during the economic transition in Hungary: retraining and public service employment (PSE). To adjust for non-random assignment of programme participants, net impacts were computed using matched pair samples and regression models. The evidence suggests retraining may improve the chance for re- employment, is unlikely to improve re-employment earnings, but may improve job durability. Net societal benefits could be improved by retraining relatively more males, older persons, and those with less education. PSE does not appear to provide a reliable path to a regular non-subsidized job, and may even lower re-employment earnings. PSE might best be viewed as an income transfer programme having the collateral benefit of maintaining basic work habits. The net societal impact of PSE could increase if it involved relatively more females and older persons.

JEL classification: J68, P23. Keywords: training, employment, policy, Hungary, unemployment.

1. Introduction

During the early phase of post-Socialist economic restructuring in Hungary, unemployment rose dramatically. To ease the hardship associated with worker dislocation and to maintain social stability, funding for a variety of active labour programmes (ALPs) was increased. This paper presents estimates of the impact on re- employment and earnings of the two most popular ALPs: retraining and public service employment (PSE). Since non-random assignment of programme applicants resulted in participant groups with characteristics significantly different from the general

454 A net impact analysis of active labour programmes in Hungary

population of unemployed persons, net impact estimates presented in this paper were computed using a variety of methods.

Estimates presented in this paper are based on a survey organized by an International Labour Office (ILO) group in the Hungarian Ministry of Labour which operated on a grant from the government of Japan. Involved in the survey were representatives of the Hungarian Ministry of Labour, the National Labour Centre in Hungary, the ILO, the W.E. Upjohn Institute for Employment Research, and the labour administrations in the Hungarian counties of Borsod-Abalij-ZemplCn, Hajdli-Bihar, and Somogy.

Design of the sample on which the survey was conducted began in July 1992. A previous survey using the sample design was conducted in November 1992. Results of that survey are summarized in Godfrey, Ldziir and O’Leary (1993). That paper reported a fundamental problem in evaluating the effect of the active programmes on re-employment-many of those interviewed were still involved in retraining or PSE. This paper reports on results of the second attempt to interview the sample. The second wave of interviews was carried out in November 1993, exactly one year after the first interviews.

Controlling for observable characteristics, retraining may have slightly improved the chances for re-employment in a non-subsidized job, but the gain in re-employment was probably not sufficient to justify the cost of retraining. However, since the durability of jobs appears to be better for those who were retrained, the long-term earnings impacts may be significant. Net societal benefits from retraining could be improved by targeting services to more males, older persons, those with fewer years of formal education, and those with no non-manual specialization. Public Service Employment (PSE) was a successful strategy to keep people out of unemployment, but it did not appear to be a cost-effective means of getting people re-employed in non-subsidized jobs. PSE is probably best viewed as an income transfer programme that has the side effect of preventing deterioration of basic work habits. In terms of re-employment, the net societal impact of PSE could be improved if it involved more older persons and females.

2. Sample design and selection

Rather than emphasizing statistical precision and power, the sample sizes for the ILO survey of labour market programme participants in Hungary were largely determined by the budget available and the time burden conducting the surveys would impose on the county labour office staff. Subject to these constraints the samples were made as large as possible. Other basic objectives were to have the sample sizes across counties be in proportion to the population and number of unemployed in the counties, and to have a subsample which would act as a comparison group for estimating programme impacts which was somewhat larger so as to maximize the statistical leverage in estimating impacts.

Just as the original survey conducted in 1992 had survey response rates over ninety per cent in each of the three counties, in November 1993 over ninety per cent of the previous respondents were contacted in each county. A review of the methods used to contact randomly selected clients, and the rules for suspending interviews is given for each of the three counties in Godfrey, Ldziir and O’Leary (1993). A statement describing the success achieved in obtaining a random sample for the survey is described in O’Leary (1993a).

Interviews for the survey were conducted in the three Hungarian counties of Borsod (Borsod-Abadj-Zempltn), Hajdd (Hajdli-Bihar), and Somogy. In these counties three categories of persons who used labour market programmes were surveyed: (1) persons

O’Leary 455

who registered as unemployed in June 1991, (2) persons who entered retraining in the second half of 1991, and (3) persons who participated in public service employment (PSE) in September 1991.

As summarized in Table 1, a total of 1,478 persons were interviewed for the survey in November 1993. This total is somewhat smaller than the 1,574 interviewed in November 1992, but the sample proportions across programmes and counties did not differ significantly between years nor from the sample design.

Table 1. Sample characteristics

Respondents by labour market programme and county, November 1993

Borsod Hajdd Somogy Total

Registered unemployed 262 203 139 604

Retraining 196 146 103 445

Public service employment 187 135 107 429

Totals 645 484 349 1,478

Respondents by labour market programme and county, November 1992


Registered unemployed 28 1 223 144 648

Retraining 207 159 108 474


Totals 684 524 366 1,574

Survey sample design by labour market programme and county


Registered unemployed 288 212 144 644

Retraining 216 159 108 483


Totals 720 530 360 1,610

3. Preliminary comparison of the samples

Comparing the exogenous characteristics of the three samples we see that those who entered retraining and those who participated in PSE are quite different from those in our sample of registered unemployed. As shown in Table 2, excluding county of residence, there are statistically significant differences in nine of the ten exogenous characteristics


when either the retraining or PSE sample is compared to the sample of registered unemployed. Compared to the sample of registered unemployed those in the retraining sample are significantly younger, more likely to be female, more educated, more specialized in professional and technical skills, much more likely to have worked in white collar jobs, less likely to have received unemployment insurance (UI) benefits since June 1991, less likely to have special problems in finding a job, and less likely to be unskilled.

The contrast between the PSE sample and the sample of registered unemployed is just as great, but the differences are generally in the opposite direction. Relative to the registered unemployed, PSE workers tend to be somewhat younger, more likely to be male, less educated, less specialized in either manual or technical skills, much less likely to have worked in white collar jobs, much less likely to have received UI since June 1991, more likely to have special problems in finding a job, and much more likely to be unskilled. Clearly, there are different selection criteria applied in referring registered unemployed to retraining and PSE. This selection-bias should not be ignored in evaluating the impact of the programmes. First, however, some further refinement of the sample should be made before examining programme impacts on re-employment and earnings.

In addition to comparing the samples in terms of exogenous variables, Table 2 also summarizes results on outcomes of interest. Compared to the 1992 survey, there was an increase in the percentage of respondents in a normal job within each of the three samples. The most dramatic increase was for persons in the retraining sample who increased their rate in a normal job by nearly 20 percentage points to 50.6 per cents2 Furthermore, while the percentage in a normal job for training participants was not different from the registered unemployed in 1992, the retraining participants had a statistically significant 19.2 percentage point higher re-employment rate in 1993 than the registered unemployed sample. For the PSE sample we see in the 1993 survey that while the percentage in a normal job remains well below that for the comparison sample, between years there was a significant increase in the percentage in a normal job.

Average monthly earnings on the normal job increased by about 18 per cent for both the retraining and comparison samples of registered unemployed, with the monthly earnings for training participants being significantly higher than the comparison group of registered unemployed in 1993. Earnings for former PSE workers now in a normal job rose from a much lower base of about 20 per cent, but were significantly lower than the comparison group in November 1993.

A broader measure of work is also reported on in Table 2. The percentages in any job in November 1993 were 31.4, 56.0, and 39.2 for the registered unemployed, retraining, and PSE samples respectively. This employment indicator may better summarize avoidance of hardship because it includes persons in supported work who are receiving incomes. It is reported here because it is related to the official measure of unemployment. However, since the ultimate aim of active measures is to get people into unsubsidized jobs, the proportion in normal jobs will be the main focus of this paper.

4. Identifying the samples for analysis

Table 3 summarizes restrictions placed on the samples before impact analysis was conducted. Among the 604 in the sample of registered unemployed interviewed in November of 1993, 15 had participated in either retraining or PSE before November 1992. Since we are interested in comparing samples of participants in retraining or PSE

O'Leary 457

with a sample of non-participants, for analysis, the group of 15 is removed from the sample of registered unemployed.

Table 2. Comparison of means of exogenous and endogenous variables across labour market programmes, 1993 survey respondents

Variable Registered Entered Public service unemployed retraining employment

Age (years) 34.0 25.0** 31.3**

Gender @er cent male) 55.1 43.4** 67.5**

Education (years) 9.8 11.8** 8.7**

Specialization (manual, per cent) 36.1 3 1 .O* 17.9**

Specialization (technical, per cent) 10.3 24.3** 6.5**

White collar worker (per cent) 14.9 49.2** 7.4**

Received UI since 6/91 @er cent) 79.0 71 .O** 38.9**

Never worked before 6/91 @er cent) 18.6 42.1 ** 18.5

Special problems finding job (per cent) 5.0 2.5** 7.7*

Unskilled worker @er cent) 31.9 8.3** 66.7**

Borsod county @er cent) 43.4 44.0 43.6

Hajdli county @er cent) 33.6 32.8 31.5

Somogy county (per cent) 23.0 23.2 24.9

Per cent in any job in November 93 35.8 56.0** 39.2

Per cent with a normal job in November 93 31.4 50.6** 14.0**

Per cent with a normal job in November 92 26.8 30.8 8.2**

92)

Monthly earnings on normal job in Nov 93 15,307 16.875* 13,355*

Difference inper cent with a normal job (93- 4.6**** 19.8**** ' 5.8****

Monthly earnings on normal job in Nov 92 12,567 13,861 10.653

Difference in monthly earnings on normal job 2,740**** 3,014**** 2,702**** (93-92)

Sample size 604 445 429

* Significantly different from full sample of registered unemployed at the 90 per cent

confidence level. ** Significantly different from full sample of registered unemployed at the 95 per cenr

confidence level. *** Difference between years significant at the 90 per cent confidence level. **** Difference between years significant at the 95 per cent confidence level.


Table 3. Activity of the November 1993 survey respondents

Registered Retraining PSE unemployed participant worker

Total 604 445 429

Exclusion from analysis' 15 0 36

Sample for analysis 589 445 393

In a normal job 185 225 59

Not in a normal job 404 220 334

In any job 209 249 132

Not in any job 380 196 26 1

Registered as unemployed 238 104 204

Receiving unemployment compensation 19 30 77

Group training 5 13 2 Individual training 1 Public service employment 14 10 71 Self-employment 3 Wage subsidy job 6 5 1 Investment subsidy job 4 4 1 Work sharing 2 Early retirement 47 3

In an active labour market programme

1. From the comparison sample of registered unemployed, 3 were excluded because they were involved in retraining in November 1992 and 12 were excluded because they participated in public service employment (PSE) in November 1992. There were no exclusions from the November 1993 sample of retraining participants since all had left training they participated in during or prior to November 1992. From the sample of 429 claimants who were in PSE in November 1992,36 were excluded from analysis because they had not left PSE between then and November 1993.

Since we are interested in determining the effect of retraining on labour market success, it is important that we restrict our sample of retrainees to those who have left retraining. In analysing the November 1992 survey this condition meant eliminating 106 of the 474 respondents. However, by November 1993 all 445 of the November 1993 respondents had completed the training which they entered in the second half of 1991.

In the earlier survey it was impossible to determine the proportion of PSE participants who had finished their involvement with the programme. However, application of this distinction was possible in the November 1993 survey. Among the 429 PSE respondents 393 had completed their involvement with the programme which began in 1991. This is the relevant sample to examine when estimating the proportion in a normal job.

Table 3 also provides a further summary of the activities of persons not in normal jobs on the survey date. For each sample the table lists the number of people registered

O’Leary 459

as unemployed, the number receiving unemployment compensation, and the numbers involved in various active labour market programmes.

Among those not in any job on the survey date, the vast majority appear to view the public employment service as a useful aid in gaining re-employment. For this group 63 per cent of registered unemployed, 53 per cent of retraining participants, and 78 per cent of persons with PSE experience were registered as unemployed with the public employment exchange. Given the small numbers receiving unemployment compensation, it would also seem that only a small fraction registered simply to meet continuing eligibility for benefits3 Table 3 also shows that a good number of persons not in normal jobs were involved with active labour market programmes. For the comparison sample of registered unemployed the most popular activity was early retirement, among the retraining participants the most popular was additional retraining closely followed by PSE, and for the PSE sample the most popular activity was further involvement in PSE.

While not reported in Table 3, it is also interesting to note that among those not in a normal job in November 1993, a significant proportion have held a normal job at some, time since November 1992. The numbers are 39 (or 9.7 per cent) of the 404 registered unemployed, 35 (or 15.9 per cent) of the 220 retraining completers, and 47 (or 14.1 per cent) of the 334 PSE workers who were not in a normal job in November 1993 had one at some time since November 1992.

Table 4 summarizes characteristics of the three samples that were used for analysis. It should be noted that the comparison group of 589 registered unemployed who did not use an ALP had no statistically significant differences in exogenous characteristics from the full sample of 604.

As shown in Table 4, comparing the new retraining and PSE samples .to the new comparison group sample of registered unemployed persons who did not participate in an ALP, we see the same pattern of results as is presented in Table 2 for the full subsamples-the retraining sample is more female, better educated, and more skilled than the comparison group, while the PSE sample is more male, less well educated, and less skilled than the comparison group. To summarize, the samples for analysis include 589 persons who registered as unemployed but did not participate in an ALP, 445 persons who completed a retraining course by November 1993, and 393 persons who finished participation in PSE.

The two indicators of labour market success used in conducting the impact analysis are: (1) now employed in a normal job, and (2) monthly earnings on the normal job. The first outcome is the best available measure of labour market success for making comparisons between the groups. It is an indicator of whether or not the person was employed on the survey date in November 1993.

For comparisons of this type there are problems with any measure of re-employment because participants in retraining and PSE have less time available for job search and less public and private resources devoted to job search since the time of registering as unemployed than persons who are registered as unemployed and not involved in an ALP. However, with the present sample this is less of a problem than for the November 1992 survey because of the longer total time available for search. Furthermore, at this later date, employment status may depend more on the quality of the job match than the time available for search since at this time the current job might no longer be the first post- unemployment job. Regarding the second measure of labour market success, monthly earnings, fortunately all persons employed in a normal job on the interview date reported the amount of average monthly earnings on their job.


Table 4. Comparison of means of exogenous and endogenous variables across labour market programmes, 1993 survey respondents

Variable Registered Completed No longer in unemployed retraining PSE

no ALP'

Age (Years)

Gender (per cent male)

Education (years)

Specialization (manual, per cent)

Specialization (technical, per cent)

White collar worker (per cent)

Received UI since 6/91 (per cent)

Never worked before 6/91 (per cenf)

Special problems finding job @er cenf)

Unskilled worker @er cent)

Borsod county

Hajdu county

Somgy county

Per cent in any job in November 93

Per cent with a normal job in November 93

Per cent with a normal job in November 92

Difference inper cent with a normal job (93-92)

Monthly earnings on normal job in Nov 93

Monthly earnings on normal job in Nov 92

Difference in monthly earnings on normal job (93-92)

34.3

55.5

9.8

36.5

10.4

14.9

79.3

17.7

4.8

31.8

43.1

33.6

23.3

35.5

31.4

27.2

4.2****

15,388

12,611

2,777****

25.0**

43.4**

11.8**

31.0*

24.3**

49.2**

71 .O**

42.1 ** 2.5'

8.3**

44.0

32.8

23.2

56.0**

50.6**

30.8

19.8****

16,875

13,861

3,014****

31.2**

67.4**

8.7**

17.0**

6.6**

8.1**

40.7**

18.4

8.2**

67.1 **

44.3

29.5

26.2

33.6

15.0**

8.9**

6.1 ****

13,200*

10,653

2,547****


* Significantly different from full sample of registered unemployed at the 90 per cent confidence level.

** Significantly different from full sample of registered unemployed at the 95 per cent confidence level.

*** Difference between years significant at the 90 per cent confidence level. **** Difference between years significant at the 95 per cent confidence level.

I . Not in retraining or PSE prior to November 30 1992.

O'Leary 46 1

5. Impact estimation methodology

The extreme sample selection involved in assigning registered unemployed to these programmes, means that special care must be taken in evaluating the impacts of retraining and PSE on labour market success. Impact estimates reported in this paper were computed in three different ways: (1) simple unadjusted comparison of means, (2) comparison of means using a matched pairs comparison group, and (3) regression adjusted impact estimates. The following is a brief description of each of the three procedures used to estimate programme impacts.

5.2. Unadjusted impact estimates In terms of clearly guiding policy, simple unadjusted impact estimates are usually the most influential because they are easy to understand. This is the main appeal of programme evaluation done using a classically designed experiment involving random assignment? When random assignment has been achieved, modelling of behaviour and complex econometric methods are not needed to estimate reliable programme impacts. With large samples randomly assigned to treatment and control groups, observable and unobservable characteristics of the two groups should not differ, on average, so that any difference in outcomes may be attributed to exposure to the programme. Programme impacts may be computed as the simple difference between means of the samples of programme participants and control group members on outcome measures of interest, or:

where E is the expectation operator yielding means of the random variables, y is an outcome of interest, and the index i denotes the sample of programme participants while] denotes the comparison sample. Tests of significance are done using t-statistics.

5.2. Impact estimates using a matched pairs comparison group In terms of observable characteristics, the comparison group of 589 persons who registered as unemployed but did not participate in an ALP differed significantly from both the 445 persons who completed a retraining course by November 1993, and the 393 persons who have finished participation in PSE. Therefore, it would not be surprising to observe different labour market success across the three groups even in the absence of ALPS. To put the assessment of retraining and PSE on a more even footing, separate synthetic comparison groups for the samples of retraining and PSE participants were formed using a matched pairs method~logy.~

The synthetic comparison groups used in the analysis reported on here were formed by comparing observations in the comparison group of 589 with those in the completed retraining and PSE samples using the standardized Mahalanobis distance measure:

2 dii = Sumk (Zik - Zjk)

where, the index i represents observations in either the retraining or PSE samples and the index j represents observations in the comparison group of 589, the index k runs over the 13 exogenous characteristics on which the observations are matched, and Z represents the standardized value of a characteristic where the mean and standard deviation of the characteristic is computed on the pooled sample of the 589 comparison group members and the members of the relevant ALP.

Using this distance measure, separate comparison groups were formed for the retraining and PSE groups. For example, for each of the 445 persons in the retraining


sample d , was computed for each of the 589 people in the comparison group. The person with the smallest dii from the comparison group was selected for inclusion in the new synthetic comparison group, with ties being resolved randomly and each person in the retraining sample being compared to all 589 in the comparison group.6 The same procedure was used to form a synthetic comparison group for the PSE sample.

After forming the new synthetic comparison groups of 445 for the retraining completers and 393 for the PSE sample, programme impact estimates were computed using a simple difference of means, with significance of impacts being judged by t-tests. It should be noted that because a single observation from the comparison sample may be chosen more than once for the synthetic comparison group the estimated standard error, computed in the usual way, for this group will be will be reduced. The t-tests for the matched pairs analysis therefore depend on weighted standard error estimates which give the upper bound on the possible standard error?

5.3. Regression adjusted impact estimates A natural method for assessing the impact of participation in a particular programme on labour market success when observable characteristics of participant and comparison group members are dramatically different is multivariate regression analysis.

For this study, ordinary least squares (OLS) estimation of the model:

yj = a. + alpi + blXIi + b2X2; + ...+ bl$iloi + ui, (3)

was done on the pooled sample of comparison group members and programme participants.' In the model y is the outcome of interest, a0 is the mean value of the outcome for comparison group members evaluated at the mean of all observable characteristics included in the regression, P is a dummy variable with a value of 1 for programme participation (either retraining or PSE) and 0 otherwise, al is the impact of the programme on the outcome for the programme participants evaluated at fhe mean of all observable characteristics, X I to Xl0 are observable characteristics measured as deviations from their mean values, and ui is a normally distributed mean zero error term.g

6. Re-employment success of participants in retraining

The following comparisons involve persons who participated in retraining programmes in the second half of 1991 and had completed the retraining course by the survey date in November 1993, with the comparison group being persons who were registered as unemployed in June 1991 and did not participate in retraining or PSE during that spell of unemployment.

The following description of the usual process of selecting candidates for participation in retraining was provided by a county labour programmes administrator- Dr. JInos Simk6 of Borsod-Abadj-ZemplCn county where the first regional retraining centre (ERAK) has been established:

'Unemployed persons interested in retraining are usually first informed about the availability of courses at the local employment centre, although announcements are frequently also made in local newspapers. Anyone who is unemployed can apply for retraining. Counsellors at local employment centres try to guide applicants into the most appropriate type of training. According to the law, the unemployed may be obliged to enter retraining, but this i s not generally applied in practice. Applicants undergo an

0 'Leary 463

aptitude test and a health examination which is either carried out by a physician and psychologist of the county labour centre, or in certain cases-such as at the regional retraining centres-at the retraining institution. With courses where there are too many applicants, there is a kind of ranking based on the psychology test results. The quality of these tests vary, some of them are very supeecial. Recently an attempt was made to encourage training institutions to use specialists to do deeper examinations to reduce drop-outs among retraining participants. In this field we are extremely happy about the methods used by the regional retraining centre. Afrer selecting the actual participants, we stop their unemployment compensation, because they receive a retraining subsidy during the course'.

This statement of the selection process for retraining conforms with the characteristics of the samples observed. A clear form of sample selection is the case where a course is over-subscribed and applicants are referred based on their rank in performance on psychological and physical examinations. Scores for these tests would be a useful characteristic in modelling sample selection. Unfortunately these results are not available. Clearly those individuals with a comparative advantage with training were selected for training; they should be expected to have higher re-employment rates after training than would a randomly selected sample.

6.1. Unadjusted impact estimates for retraining From Table 4 we see that on the survey date the percentage of people re-employed in a normal job was 19.2 points higher for retraining completers as compared to registered unemployed who never participated in an ALP. Furthermore, the difference is significant at the 95 per cent confidence level.

Persons who completed retraining and were employed in normal jobs also appear to have monthly earnings which are about HUF 1,500 higher than persons in the comparison group, but this difference is not statistically significant.

From November 1992 to November 1993 the percentage of persons holding a normal job increased by 4.2 percentage points among those in the comparison group but increased by 19.8 percentage points among those who completed retraining, thereby magnifying the difference in success between groups.

An outcome which was not reported in summaries of the earlier survey, the percentage in any job in November 1993, shows that training completers had better success than the comparison group even by this broader measure of labour market success. Any job also includes participation in public works or any other type of subsidized job. Fully 56 per cent of retraining completers were employed in any job in November 1993 while only 35.5 per cent of persons in the comparison held any employment position.

6.2. Impact estimates using a matched pairs comparison group fur retraining In an attempt to correct for the sample selection which resulted in the group of training participants being younger, more female, more educated and more specialized than persons in the comparison group, the matched pairs method was used to form a synthetic comparison group with similar characteristics. Examining means on the thirteen exogenous characteristics in Table 5 we see that the synthetic Comparison group looks much like the group of retraining participants in terms of observable characteristics. It is also the case that the re-employment rates are not statistically significantly different between the two groups. While not significant, the point estimate for those who did not participate in retraining shows a 1.2 per cent lower re-employment rate. This suggests


that most of the added re-employment success of those participating in retraining is due to the observable characteristics of those selected for retraining.

Average monthly earnings on the current normal job for the synthetic comparison group were somewhat lower-by HUF 2,052-than for the group of retraining completers. Again, this is probably due to the fact that those selected for retraining tend to be those registered unemployed with the highest potential productivity.

6.3. Regression adjusted impact estimates for retraining The regression adjusted impact estimates presented in Table 6 indicate that on the survey date people who completed retraining were about 6.4 per cent more likely to be re- employed in a normal job than were persons who were registered as unemployed and never participated in an ALP. This difference is significant at the 90 per cent confidence level. To produce these estimates, regressions were run on the pooled sample of 445 retraining completers and 589 comparison group members who registered as unemployed in June 1991 and had not participated in an ALP by November 1992.'' The point estimates should therefore be interpreted as the mean response for the retraining and comparison groups evaluated at the mean characteristics of the combined sample. That is, if the retrainees had the mean characteristics of the combined sample they would be about 6.4 per cent more likely to have a normal job at the survey date than the average person in the combined sample.

From this analysis persons who completed retraining also appear to have monthly earnings which are about HUF 500 higher than persons in the comparison group, but this difference is not statistically significant.

7. Re-employment success of participants in PSE

This analysis examines persons who participated in public service employment (PSE) programmes in September 1991, with the comparison group being persons who were registered as unemployed in June 1991 and did not participate in retraining or PSE during that spell of unemployment. The aim of PSE is mainly one of income transfer to the long-term unemployed while at the same time giving people regular work activity to arrest the deterioration of basic workplace skills. Secondary aims include contribution to the public welfare and the public infrastructure so as to enhance future re-employment possibilities. The categories of activities which may be undertaken under PSE contracts are few in number and are clearly specified in Hungarian employment law. The main types of PSE work are maintenance of public facilities and assistance to social welfare agencies.

The value of these activities is difficult to measure in market terms, the only real way being to measure the cost of inputs which is mainly a wage cost. While re-employment in a normal job is not the principal aim of PSE, this would be a favourable outcome, and it is one which is possible to objectively measure. Results of such an analysis are presented in this section.

Just as for retraining, the group of persons selected for PSE do not have the same characteristics as the average unemployed person. As indicated in Table 4, relative to the typical registered unemployed person, a PSE participant is more likely to be male, less educated and less likely to have formal job skills and credentials.

O’Leary 465

Table 5. Means of retraining and comparison groups for exogenous and endogenous variables. Comparison group formed using matched pairs method, 1993 survey respondents

Variable Retraining Comparison Impact group estimate

(1) (2) (1) - (2)

Age (years) 25.0 26.8 -1.8**


Mucation (years)

43.4 42.7 0.7

11.8 11.6 0.2

Specialization (manual, per cent) 31.0 31.7 -0.7

Specialization (technical, per cent) 24.3 23.8 0.5

White collar worker (per cent) 49.2 50.3 -1.1

Received UI since 6191 @er cent) 71 .O 70.3 0.7

Never worked before 6/91 (per cent) 42.1 40.1 2.0

Special problem finding job @er cent) 2.5 2.5 0.0

Unskilled worker (per cent) 8.3 9.9 -1.6

Borsod county (per cent) 44.0 44.0 0.0

Hajdi, county @er cent) 32.8 32.8 0.0


In any job (per cent) 56.0 49.4 6.6

Now has normal job (no subsidy, per cent) 50.6 49.4 1.2

Earnings on normal job (HUF) 16,875 14,823 2,052


*

** Difference between retraining participant and comparison group significant at the 90 per cent confidence level. Difference between retraining participant and comparison group significant at the 95 per cent confidence level.

a. Matched pairs are formed by selecting for each person in the training sample that person in the sample of unemployed who did not participate in an ALP the one most similar in the characteristics: age, gender, education, specialization (manual and technical), collar colour, UI recipiency, prior working history, special job-finding problems. and worker skill. Matches are made using the Mahalanobis distance measure. The distance between two observations is the sum of squared differences in characteristics. T e s are resolved randomly, To equally weight the characteristics in matching, distances are computed for standardized characteristics where the means and standard deviations are computed on the combined retraining and comparison group samples.


Table 6. Regression adjusted estimates of the impact of retraining for the 1993 survey (sfandard errors in parentheses)

Independent variable Dependent variable

Fraction with Fraction with a Level of any job normal job earnings

(HUF)

Control group at means

Impact of retraining at means

Age (yeam)

Gender (male=l)

Education (years)

Specialization (manual-1)

Specialization (technicakl)

Received UI since 6/91 (Yes=l)

Never worked before 6/91 (Yes=l)

Special problems finding job (Yes=l)

HajdG county (Yesrl)

Somogy county (Y-1)

Sample size

R2

F

0.41 1 ** (0.021 )

0.069** (0.036)

-0.005 ** (0.002)

-0.048 (0.03 1)

0.044** (0.010)

0.079* (0.042)

0.030 (0.065)

-0.050 (0.037)

-0.034 (0.041)

0.029 (0.081)

0.085** (0.034)

0.006 (0.038)

1,007

0.125

0.367** (0.021)

0.064* (0.035)

-0.004** (0.002)

-0.062** (0.030)

0.041 ** (0.01 0)

0.104** (0.041)

0.049 (0.064)

-0.047 (0.037)

-0.033 (0.040)

-0.042 (0.079)

0.089** (0.034)

-0.w (0.038)

1,007

0.124

16,105** (643)

493 (913)

208** (59)

4,388** (87 1)

I ,776** (305)

-633 (1,103)

1,239 (1,702)

1,182 (988)

-76 (1,124)

2,265 (2,787)

1,528 (962)

637 (1.109)

397

0.317

- 12.97** 12.79** 16.27**

* ** Parameter estimate significant at the 90 per cent confidence level.

Parameter estimate significant at the 95 per cent confidence level.

O’Leary 467

We therefore examine the labour market success of PSE participants using the same variety of techniques as was used for evaluating retraining. Following is a description of the usual process of selecting candidates for participation in PSE provided by a county labour programmes administrator-Dr. JBnos Simk6 of Borsod-Abatij-Zemplkn county: ‘It is local employment centres that refer unemployed persons to PSE. Howevel; it often happens that an employer selects someone from among the unemployed before referral. These requests are usually filled by a local employment centre, because it is important for local employment centres to reduce the number of idle unemployed and there are no special criteria for referral to PSE. The unemployed are obliged to accept PSE work, f i t conforms to their education and skills. Mostly unemployed with low education are sent to these jobs. Ifan unemployed person does not accept a PSE j o b suitable for him, he can be denied eligibility for unemployment compensation payments I.

There is clear sample selection in referral to PSE with the resulting sample of participants having characteristics completely different from those referred to retraining.

7.1. Unadjusted impact estimates for PSE From Table 4 we see that, on the survey date, people who participated in PSE were 16.4 per cent less likely to be re-employed in a normal job than were persons who were registered as unemployed and never participated in an ALP. Furthermore, this difference is significant at the 95 per cent confidence level. It should be noted that this unadjusted re-employment rate is 6.1 percentage points higher than observed for the same group in November 1992.

Persons who participated in PSE and were re-employed in normal jobs also appear to have monthly earnings which are about HUF 2,200 lower than persons in the comparison group, with this difference statistically significant at the 90 per cent confidence level.

7.2. Impact estimates using a matched pairs comparison group for PSE In an attempt to correct for the sample selection which resulted in the group of PSE participants being more male, less educated and less specialized than persons in the comparison group, the matched pairs method was used to form a synthetic comparison group with similar characteristics. Examining means of the thirteen exogenous characteristics in Table 7 we see that the synthetic comparison group looks much like the group of PSE participants in terms of observable characteristics. However, the rates of re-employment in a normal job are statistically significantly different between the two groups, with the point estimate for those who participated in PSE being 15.0 per cent lower than the comparison group. This differential is somewhat smaller than the unadjusted difference given in Table 4. Clearly comparing the labour market success of PSE participants with unemployed persons who have similar characteristics is more even handed. Even this comparison probably overestimates the real re-employment rate differential because there are probably unobserved factors such as motivation or personal contacts which explain why people who could be selected for PSE choose to do otherwise and enjoy better re-employment success.

Average monthly earnings on the current normal job for the synthetic comparison group were not significantly different from the group of PSE completers. This result is undoubtedly due to the fact that persons with low qualifications compete for jobs near the bottom of the earnings distribution.

7.3. Regression adjusted impact estimates for PSE Regression adjusted impact estimates presented in Table 8 indicate that on the survey date people who completed PSE were 19.0 per cent less likely to be re-employed in a normal job than were persons in the comparison group. This difference is significant at the 95 per cent confidence level.


To produce these estimates regressions were run on the pooled sample of 393 PSE completers and 589 comparison group members who registered as unemplo ed in June 1991 and had not participated in retraining or PSE by November 1992.’ , The point estimates should, therefore, be interpreted as the mean response for the PSE and comparison groups evaluated at the mean characteristics of the combined sample. That is, if the PSE participants had the mean characteristics of the combined sample they would be 19.0 per cent less likely to have a normal job at the survey date than the average person in the combined sample.

Similar regressions, also reported in Table 8, suggest that persons who completed PSE appear to have monthly earnings which are HUF 236 lower than persons in the comparison group, but this difference is not statistically significant.

Y

8. Subgroup analysis

There are at least two reasons to examine treatment impacts by population subgroup. One is to provide information to policy-makers who may consider targeting retraining or PSE to certain groups like those without a specialization or older unemployed persons. Another is to identify any possible biases in the effects-a programme that benefits only one gender or certain education level groups may not be considered good policy even if it is cost-effective. This section reports on programme impacts for sixteen subgroups defined by categorical variables for the following seven characteristics: age (three groups), gender, educational attainment, non-manual specialization, unemployment insurance (UI) benefit receipt, whether or not there was previous work experience, and county (three groups). The dummy variables actually used indicated the following: age 25 or less, age 26 to 40 or otherwise, if female, education 8 years or less, non-manual specialization or not, received UI since June 1991 or not, worked before June 1991 or not, registered in Hajdd county or not, registered in Borsod county or not.”

All subgroup treatment impacts were simultaneously estimated in a single regression model. The specification employed allows the treatment response for each subgroup to be estimated controlling for the influence of other subgroup characteristics. For example, the model allows estimation of treatment impacts associated with being female controlling for the fact that females are more likely to have more than 8 years education and less likely to have a non-manual specialization. The subgroup treatment impact estimates are reported in Table 9 for retraining and Table 10 for PSE. Suppressing subscripts and using matrix notation, the regression equation estimated can be written:

Y = a + PB + G C + GPD‘+ u (4)

where Y is the outcome measure re-employed in a normal job, a is the intercept, B, C, and D , are conformable parameter vectors, P is the indicator of participation in either retraining or PSE, G is the matrix of dummy variables which code for membership in a subgroup, and u is a mean zero normally distributed random error term. Equation (4) specifies a complete one-way interaction model. It allows simultaneous estimation of all subgroup treatment impacts, but imposes linear restrictions on their estimates. Treatment impacts for a particular subgroup are computed as the sum of the parameter estimate on the product of the subgroup dummy variable and the treatment indicator plus the sum of parameter estimates on the product of subgroup dummies and the treatment indicator multiplied by their respective population shares. In each computation, parameter estimates for the complement to the subgroup of interest are omitted.

O’Leary 469

Table 7. Means of PSE and comparison groups for exogenous and endogenous variables. Comparison group formed using matched pairs method, 1993 survey respondents

Variable PSE Comparison Impact group estimate

(1) (2) (1) - (2)

Age (years) 31.2 30.6 0.6


Education (years)

67.4 64.3 3.1

8.7 8.8 -0. I

Specialization (manual, per cent) 17.0 17.3 -0.3

Specialization (technical, per cent) 6.6 6.6 0.0

White coliar worker (per cent) 8.1 7.8 0.3

Received UI since 6/91 (per cent) 40.7 44.3 -3.6

Never worked before 6/91 (per cent) 18.4 17.2 1.2

Special problems finding job (per cent) 8.2 8.1 0.1

Unskilled worker (per cent) 67.1 62.7 4.4

Borsod county (per cent) 44.3 44.8 -0.5

Hajde county @er cent) 29.5 29.5 0.0


In any job (per cent) 33.6 30.0 3.6

Now has normal job (no subsidy,per cent) 15.0 30.0 -15.0**

Earnings on normal job (HUF) 13,200 14,249 -1,049


*

** Difference between retraining participant and comparison group significant at the 90 per cent confidence level. Difference between retraining participant and comparison group significant at the 95 per cent confidence level.

a. Matched pairs are formed by selecting for each person in the PSE sample that person in the sample of unemployed who did not participate in an ALP the one most similar in the characteristics: age, gender, education, specialization (manual and technical), collar colour, UI recipiency, prior working history, special job-finding problems, and worker skill. Matches are made using the Mahalanobis distance measure. The distance between two observations is the sum of squared differences in characteristics. Ties are resolved randomly. To equally weight the characteristics in matching, distances are computed for standardized characteristics where the means and standard deviations are computed on the combined PSE and comparison group samples.


Table 8. Regression adjusted estimates of the impact of PSE for the 1993 survey (standard errors in parentheses)

Independent variable Dependent variable Fraction with Fraction with a Level of

any job normal job earnings ( W F )

Control group at means

Impact of retraining at means

Age (years)

Gender (male-l)

Education (years)

Specialization (manual=l)

Specialization (technical=l)

Received UI Since 6/91 (Yes=l)

Never worked before 6/91 (Yes=l)

Special problems finding job (Yes=l)

Hajdli county (Yes=l)

Somogy county (Yesrl)

Sample size

R2

0.353** (0.020)

-0.019 (0.034)

-0.004** (0.002)

-0.048 (0.031)

0.039** (0.011)

0.063 (0.049)

0.020 (0.077)

-0.133** (0.034)

-0.038 (0.047)

0.013 (0.062)

0.131 ** (0.034)

0.053 (0.037)

967

0.116

0.305** (0.017)

-0.142** (0.030)

-0.003** (0.001)

-0.074** (0.028)

0.042** (0.010)

0.041 (0.043)

0.002 (0.068)

-0.086** (0.030)

0.026 (0.041)

0.007 (0.055)

0.087** (0.030)

0.008 (0.033)

967

0.168

17.50**

14,935** (57 1)

-236 (1,3 19)

137** (61)

4,056** ( 1,029)

1,070** (369)

(1,342)

1,673 (2,058)

1602 (1,123)

204 (1,383)

2,278 (2,692)

1,643 ( 1,074)

(1,366)

240

0.189

-230

-328

4.82**

* **

Parameter estimate significant at the 90 per cent confidence level. Parameter estimate significant at the 95 per cent confidence level.

The subgroup impact estimates may be considered to be regression adjusted in the sense that each subgroup impact is estimated while simultaneously ailowing impacts to vary across other subgroups considered. There is no formal attempt to control for sample selection in the subgroup impact analysis.

O’Leary 47 1

8.2. Subgroup analysis of retraining The sample of 445 persons who completed retraining was quite small to begin with, and further dividing it to do subgroup analysis yielded very small sample sizes. Therefore, the standard errors on the subgroup impact estimates were rather large, and while several individual impact estimates for subgroups were made with statistical precision there were no significant differences between subgroups of a particular category of characteristic. For the earnings impact outcome, because it only involves people who had a normal job sample, subgroup sample sizes become absolutely too small. Subgroup impacts are not examined for earnings, but only for re-employment in a normal job.

Results presented in Table 9 show that there were no statistically significant differences between subgroups of any particular category, but it is worthwhile to discuss the different tendencies. The effect of retraining on being in a normal job steadily increased with age and was most beneficial for the group of older workers (over 40). There was virtually no difference in impact by gender with men showing just over a one percentage point greater impact on their success rate than women. Those with eight or, less years of education had their success of re-employment in a normal job boosted an average of 22 percentage points by retraining while those with more education gained only an average of 7 percentage points. Similarly, those without a non-manual specialization benefited nearly three times more than those who entered retraining with a non-manual specialization. Those who received UI since June 199 1 and completed retraining got re-employed relatively more frequently than those who completed retraining but had not drawn UI benefits recently. Retraining was also more of a re- employment aid to those who had prior work experience. Among the three counties, retraining participants in Borsod county gained about twice as much success in re- employment from retraining than did participants from Hajdd and Somogy.

The most powerful and appealing results from this analysis are the suggestions that groups considered to be the most difficult to re-employ appear to have gained the greatest help in getting a normal job by retraining, these groups are: older workers, those with less education, and those without a manual trade.

8.2. Subgroup analysis of PSE In Table 10 we see that despite the fact that the PSE sample was only 445 persons in size, because the differences in the proportion employed in a normal job between the PSE and comparison groups are large, the subgroup programme impacts for PSE are nearly all estimated with statistical precision. However, because very few of those in PSE actually were in a normal job on the survey date, very few had earnings and subgroup sample sizes were too small to do an impact analysis on earnings.

PSE tended to help gain employment in a normal job more often for persons in the older age group (over 40), females, those with less education and those without a non- manual specialization. Working in PSE helped in gaining a normal job twice as much in Borsod county as it did in Hajdu and Somogy counties. There was a significant difference in the proportion in a normal job between the middle (26 to 40) and the older (over 40) age groups, and between those with less than eight years of schooling and those with more. The differences by gender, and between Borsod and the other two counties were nearly significant.


Table 9. Impacts of completing retraining o n whether or not currently in a normal job, by subgroup (standard errors in parentheses)

Subgroup Impact estimate Sample size Sample size retraining no ALP

Age 25 or less

Age 26 to 40

Age over 40

Female

Male

Education 8 years or less

0.0718 294 213 (0.05 12)

0.1363* 118 168 (0.0742)

0.2637** 33 204 (0.1027)

0.1121 ** 250 26 1 (0.0493)

0.1297** 192 326 (0.0475)

0.2188** 74 274 (0.0766)

Education more than 8 years 0.0706 371 315 (0.0432)

Non-manual specialization 0.0791* 307 374 (0.0459)

No non-manual specialization 0.201 3** 138 215 (0.0636)

Not received UI since 6/91

Received UI since 6/91

Worked before 6/91

0.0488 129 122 (0.0657)

0.1436** 316 467 (0.0429)

0.1370** 249 482 (0.0445)

Never worked before 6/91 0.0798 181 104 (0.0739)

Borsod county

Hajdu county

Somogy county

0.1592** 196 254 (0.0509)

0.0789 146 198 (0.0597)

0.0640 103 137 (0.0675)

* **

Impact estimate significant at the 90 per cent confidence level. Impact estimate significant at the 95 per cent confidence level.

O’Leary 47 3

Table 10. Impacts of participation in PSE on whether or not currently in a normal job, by subgroup (standard errors in parentheses)

Subgroup Impact estimate Sample size Sample size PSE no ALP

-0.1588** 159 213 Age 25 or less

Age 26 to 40

Age over 40

Female

Male

Education 8 years or less

Education more than 8 years

Non-manual specialization

No non-manual specialization

Not received UI since 6/91

Received UI since 6/91

Worked before 6/91

Never worked before 6/91

B o d county

Hajd~ county

Somogy county

(0.0506)

-0.1940** (0.0577)

-0.0870**** (0.0644)

-0.099 1 (0.0471)

-0.1 867** (0.0380)

-0.0743 (0.0498)

-0.2499*** (0.0592)

-0.1706** (0.0393)

-0.1047 (0.0766)

-0.1472** (0.0476)

-0.1544** (0.0384)

-0.1557** (0.0339)

-0.1339* (0.0802)

-0.1 131** (0.0427)

-0.2054* * (0.0514)

-0.1941 ** (0.0583)

149

85

127

262

27 1

122

326

67

233

160

320

72

1 74

116

103

168

204

26 1

326

274

315

374

21 5

122

467

482

104

254

198

137

* Impact estimate significant at the 90 per cent confidence level. ** Impact estimate significant at the 95 per cent confidence level. *** Difference from first subgroup listed for the characteristic significant at the 90 per cent

confidence level. **** Difference from age 26 to 40 significant at the 90 per cent confidence level.


9. The timing of re-employment

As mentioned in the introduction, analysis of a previous survey of the population studied here revealed a fundamental problem in evaluating the effect of the active programmes on re-employment-many of those interviewed were still involved in retraining or PSE. To illustrate this a table from O’Leary (1993a) is repeated here as Table 11. The table shows the timing of employment in the first normal job after entering each of the three programmes. Re-employment frequencies are arrayed by month from June 1991 to November 1992. There is a clear difference in the timing between those who simply registered as unemployed and those who completed retraining. Obviously training takes time. The bulk of first new normal jobs for the sample of persons who completed retraining occurred in the second half of the period, while for the registered unemployed sample the majority of first normal jobs occurred in the first half of the period. Fully 106 of the 474 retrainees interviewed in November 1992 had not completed their course at the time of the inter vie^.'^

Among PSE participants there is no clear pattern of timing in the transition to the first normal job as is revealed in Table 11 based on the November 1992 survey. It appears that the probability of taking a normal job does not depend on the length of time spent working on PSE. The timing of re-employment probably depends more on the random timing of when an opportunity for a regular job arises.

Table 12 presents information about the date on which the current normal job began for persons in the analytic sample from the November 1993 survey. The retraining and PSE samples show re-employment timing patterns similar to the earlier survey summarized in Table 11 .14 That is, the retraining participants tended to gain employment in normal jobs toward the end of the period observed, and there is no particular timing pattern for the PSE participants.

Perhaps the most interesting result about the timing of re-employment concerns the durability of new jobs. While people in retraining entered jobs later, because they spent time in retraining, they appear to hold the normal jobs they obtain longer than the comparison group of registered unemployed. Table 11 shows when people started their first normal job and Table 12 shows when people began their current normal job. The registered unemployed generally started their first normal job sooner, but many had left that job by the date of the second survey. The retraining participants show a very high rate of retaining their first normal job after retraining. For the PSE participants there was no particular pattern in the timing of obtaining or the retention of normal jobs.

As a final note to the timing of re-employment, Table 13 provides some information on the fraction of people not in a normal job on the November 1993 survey date who did spend some time in a normal job since November 1992. Among the registered unemployed, retraining participants, and PSE workers not currently in a normal job the percentages who had been in one were 9.7, 15.9, and 14.1 respectively. This provides further evidence on the usefulness of retraining and PSE in re-employing the unemployed, it also reveals something about the dynamic aspect of unemployment in Hungary. Measuring re-employment at a point in time fails to reveal the natural turnover which occurs among both jobs and the stock of unemployed.

10. Summary

In November 1992 surveys were conducted in the three Hungarian counties of Borsod- Abacj-ZemplCn, Hajd&Bihar, and Somogy as part of the first scientific attempt to

O’Leary 475

examine the impact of labour market programmes in post-Socialist Hungary. The surveys were organized by the International Labour Ofice (ILO) mission in the Hungarian Ministry of Labour which is financed by a grant from the government of Japan. An attempt to reinterview survey respondents was undertaken in November 1993. Results of the reinterview effort, which succeeded in contacting about 93 per cent of previous respondents, are reported on in this paper.

Table 11. Frequencies and cumulative re-employment rates in the first normal job after registering as unemployed based on the November 1992 survey

Registered unemployed Completed PSE participant No ALP retraining

Month Cumulative Cumulative Cumulative

Frequency @er cent) Frequency @er cent) Frequency (per cent)

June 1991

July 1991

August 1991

September 1991

October 1991

November 1991

December 1991

January 1992

February1992

March 1992

April 1992

May 1992

June 1992

July 1992

August 1992

September 1992

October 1992

November 1992

Total

15

16

18

15

5

11

6

12

6

17

9

7

2

4

6

1 1

4

1

165

9.1

18.8

29.7

38.8

41.8

48.5

52.1

59.4

63.0

73.3

78.8

83.0

84.2

86.7

90.3

97.0

99.4

100

1

1

1

2

1

3

1

7

4

8

4

9 19

10

9

19

14

3

116

0.9

1.7

2.6

4.3

5.2

7.8

8.6

14.7

18.1

25.0

28.4

36.2

52.6

61.2

69.0

85.3

97.4

100.0

0

2

3

0

5

3

2

5

3

3

3

1

3

0

2

3

1

2

41

0.0

4.9

12.2

12.2

24.4

31.7

36.6

48.8

56.1

63.4

70.7

73.2

80.5

80.5

85.4

92.7

95.1

100.0


Table 12. Frequencies and cumulative re-employment rates, by month in the current normal job’

Registered unemployed Completed retraining No longer in PSE no ALP

Month Cumulative Cumulative Cumulative

Frequency @er cent) Frequency @er cent) Frequency (per cent)

February 1991 March 1991 April 1991 May 1991 June 1991 July 1991 August 1991 September 1991 October 1991 November 1991 December 1991

January 1992 February 1992 March 1992 April 1992 May 1992 June 1992 July 1992 August 1992 September 1992 October 1992 November 1992 December 1992

January 1993 February 1993 March 1993 April 1993 May 1993 June 1993 July 1993 August 1993 September 1993 October 1993 November 1993

In a normal job

Not in a normal job

1 0 0 1 3 8 8 9 3 5 3

6 3 5 6 5 0 1 4 12 8 11 5

10 4 3 6 7 8 6 7 12 7 8

185

404

0.2 0.2 0.2 0.3 0.8 2.2 3.6 5.1 5.6 6.5 7.0

8.0 8.5 9.3 10.4 11.2 11.2 11.4 12.1 14.1 15.4 17.3 18.2

19.9 20.5 21.1 22.1 23.3 24.6 25.6 26.8 28.9 30.1 31.4

1 1 1

2 4 0 5 12 7 12 9 15 10 23 10

9 9 15 9 11 10 11 7 10 13 9

225

220

0.2 0.4 0.7

1.1 2.0 2.0 3.1 5.8 7.4 10.1 12.1 15.5 17.8 22.9 25.2

27.2 29.2 32.6 34.6 37.1 39.3 41.8 43.4 45.6 48.5 50.6

1 0 1 3 1 1 1 2

0 1 2 1 2 1 4 I 2 2 1 2

2 1 2 2 3 2 4 2 7 3 2

59

334

0.3 0.3 0.5 1.3 1.5 1.8 2.0 2.5

2.5 2.8 3.3 3.6 4.1 4.3 5.3 5.6 6.1 6.6 6.9 7.4

7.9 8.1 8.7 9.2 9.9 10.4 11.5 12.0 13.7 14.5 15.0

Total 589 445 393

1. Based on response to the question, ‘At present you are in a normal job, when did you start this job?’ in the November 1993 ILO survey.

O’Leary 477

Table 13. Experience with normal employment by persons not in a normal job in November 1993

Registered Retraining PSE unemployed participant worker

Not in a normal job in November 93 404 220 334

Per cent with a normal job since November 9.7% 15.9% 14.1% 92

Distribution of months in a normal job since November 92

1 month 2 months 3 months 4 months 5 months 6 months 7 months 8 months 9 months 10 months 11 months 12 months

2 6 5 6 5 3 2 3 3 3 0 1

4 2 5 7 2 4 1 2 1 2 1 4

5 7 6 6 2 2 9 2 2 3 2 1

Total with a normal job since November 92 39 35 47

The surveys were designed to investigate the impact of retraining and public service employment (PSE) on labour market success, by comparing outcomes for participants in these programmes to others who were registered as unemployed but did not participate in retraining or PSE before November 1992. Labour market success is measured b whether or not a person is re-employed in a normal job and by the earnings on that job,

The November 1993 sample included 604 people who were registered unemployed, 445 retraining participants, and 429 who participated in PSE. While the samples available for analysis from the three groups are relatively small, they were selected by random processes. The samples are also believed to be representative of the three populations from which they were drawn.

I Y

Table 14. Summary of impact estimates for participation in retraining

Estimation methodology In a normal job Monthly earnings (per cent) (Hungarian forints)

Unadjusted 19.2** 1,487 Matched pairs 1.2 2,052 Regression adjusted 6.3* 493

* **

Impact estimate significant at the 90per cent confidence level in a two-tail test. Impact estimate significant at the 95 per cent confidence level in a two-tail test.


Table 14 summarizes the impact estimates of retraining on the two outcomes: in a normal job on the survey date, and monthly earnings on the job. The table presents estimates from each of the three methodologies used. The unadjusted estimates indicate that on the survey date people who completed retraining were 19.2 percentage points more likely to be re-employed in a normal job than were persons who were registered as unemployed and never participated in an ALP. Retrainees were also estimated to have monthly earnings which are about HUF 1,500 higher than persons in the comparison group.

In an attempt to correct for the sample selection which resulted in the group of training participants being younger, more female, more educated and more .specialized than persons in the comparison group, two additional estimation methods were applied: (1) matched pairs which involved forming a synthetic comparison group with characteristics similar to the participant group, and (2) regression adjustment using observable characteristics as adjustment factors. The close agreement between results generated from the matched pairs and regression adjustment method suggests that the true adjusted training impact is in the range of 1 to 6 additional percentage points of re- employment, with a modest gain of from HUF 500 to HUF 2,000 in monthly earnings. That is, among similar individuals, retraining participants were only mildly more successful in gaining re-employment than were non-participants.

While the samples were too small to yield reliable estimates of subgroup impacts of retraining, the analysis indicated that retraining was most beneficial for the following subgroups: those over 40 years of age, males, those with eight or less years of education, those without non-manual specialization, and those who had some previous work experience.

Table 15 summarizes the impact estimates of public service employment (PSE) following the same format as used in Table 14 for retraining. Estimates are presented from each of the three different methodologies. The unadjusted estimates indicate that, on the survey date, people who participated in PSE were 16.4 per cenr less likely to be re-employed in a normal job than were persons who were registered as unemployed and never participated in an ALP, PSE participants were also estimated to have monthly earnings which are about HUF 2,200 lower than persons in the comparison group.

Table 15. Summary of impact estimates for participation in public service employment (PSE)

Estimation methodology In a normal job Monthly earnings (per cent) (Hungarian forints)

Unadjusted

Matched pairs

Regression ad.iusted

-16.4**

-15.0**

-16.2**

-2.188*

-1.049

-236

* **

Impact estimate significant at the 90 per cent confidence level in a two-tail test. Impact estimate significant at the 95 per cenr confidence level in a two-tail test.

In an attempt to correct for the sample selection which resulted in the group of PSE participants being more male, less educated and less skilled and specialized than persons in the comparison group, PSE net impacts were estimated using regression and matched pairs methods. The two methods both yielded an estimate of the PSE impact on the percentage of persons in a normal job of about 15 percentage points; the earnings impacts estimates were

0 'Leary 479

not statistically significant, but were considerably smaller than the unadjusted estimate. It appears that selection-bias is less of a problem in the evaluation of PSE than retraining. For the PSE, there is generally more consensus among the alternative methods. The results suggest that PSE participants were mildly more successful in gaining regular employment than is suggested by the unadjusted impact estimates.

Subgroup analysis of the PSE sample indicated that PSE tended to be followed by employment in a normal job more often for older people, females, people with fewer years of formal education and those without a non-manual specialization. Those selected for PSE were more likely to be rather young, male, less educated and without formal skills and credentials.

An analysis of the timing of re-employment indicated that use of the November 1993 survey data allowed for a better estimate of the effect of retraining on employment in a normal job than did a similar survey conducted a year earlier. It also revealed that retraining appears to lead to re-employment in normal jobs which last longer than normal jobs obtained by unemployed persons not receiving retraining.

Considering all of the evaluation results taken together, it appears that retraining may have slightly helped improve the chances for re-employment in a non-subsidized job. In terms of simple re-employment rates, the gain in re-employment was probably not sufficient to justify the cost of retraining. However, since the durability of jobs appears to be better for those who were retrained, the long-term earnings impacts may be significant. Net social benefits from retraining could be improved by retargeting services to more males, older persons, those with fewer years of formal education, and those with no non-manual specialization.

Overall, public service employment (PSE) appeared to have been a successful strategy to keep people out of unemployment, but it did not appear to be a cost-effective means of getting people reintegrated into normal non-subsidized jobs. PSE is probably best viewed as an income transfer programme that has the valuable side effect of reducing the deterioration of basic work habits. In terms of re-employment, the social impact of PSE could be improved if it were targeted to include more older persons and females.

Before ending, the summary of net impact estimates must include a few comments about the context of labour market behaviour studied. During the period of observation, June 1991 to November 1993, registered unemployment in Hungary rose from 186,000 to 640,000 and the unemployment rate increased threefold from 4 per cent to over 12 per cent. This environment was certain to have affected the chance for re-employment in a normal job. Re-employment is difficult to achieve even in an expanding economy. That we observed any positive net effect of retraining on employment in a normal job is surprising in the face of such rapidly rising unemployment.

It is also important to note that the samples on which inferences were drawn are quite small and include survey respondents from only three of twenty Hungarian counties. Furthermore, the non-random nature of assignment to programme participation meant that the original comparison group was of limited value.

Another pervasive factor which may have influenced the overall net impact estimates is the generous system of unemployment insurance (UI) operating in Hungary. Micklewright and Nagy (1994a) who reviewed the rules and effects of UI in Hungary report that in 1991 UI replaced between fifty and seventy per cent of prior wages for up to 24 months, that all eligible recipients qualified for at least 6 months benefits, and that about sixty per cent of beneficiaries qualified for close to the maximum entitled duration.

An indication of the influence UI had on behaviour is that the vast majority of all unemployed at the time of the study were registered with the public employment exchange.I6 Conversely, Godfrey (1 995) reports that in transition countries like Ukraine, where the unemployment benefit is meagre, only a tiny fraction of unemployed job-


seekers register with the public employment exchange. In addition to having virtually all jobless persons registered as unemployed in Hungary, during the typical month in 1991 an average of 76 per cent of registered unemployed collected UI benefits.

Not only entry to the unemployment register was influenced by the generous UI benefits system, Micklewright and Nagy (1994b) found that during the early 1990s once people began drawing UI in Hungary they tended to stop drawing a benefit at a slow rate, with many exiting just prior to benefit exhaustion. It should also be noted that there is a close programmatic interaction between UI and active labour programmes in Hungary. A stipend, which does not diminish the UI entitlement, is generally paid to those involved in retraining. If no job is available after retraining, which may last three to twelve months, full benefit entitlement may be re-established since eligibility was based on employment in the prior four years. Alternatively, time spent working on a PSE project is time on the job and may help establish UI eligibility for the generally low wage, low skill workers involved in PSE.

Using an imprecise measure of UI, a simple test was done to check if the present sample shows evidence that the generous and widely used system of UI in Hungary has an influence on re-employment rates following participation in active labour programmes. Table 4 shows that, 79 per cent of the registered unemployed sample, 71 per cent of the retraining participant sample, and 41 per cent of the PSE participant sample received UI benefits sometime after June 1991. The recipiency rate for the registered unemployed sample is on a par with the national average monthly figures for 1991. The higher recipiency rate among retrainees compared to PSE participants reflects the higher skill levels and labour force attachment among retraining participants.

Table 16. Influence of unemployment insurance on labour market outcomes' (standard errors in parentheses)

Outcome UI Retraining UI * Retraining

In a normal job -0.09 1 * -0.003 0.094 (0.049) (0.061) (0.071)

Re-employment earnings , 1,974 1,588 -1,563 (1,398) ( 1,644) (1,952)

UI PSE UI * PSE In a normal job -0.083* -0.138** -0.006

(0.042) (0.046) (0.059)

Re-employment earnings 1,818 71 -1,063 (1,271) (1,567) ( 2 3 12)

* **

Parameter estimate significant at the 90 per cent confidence level. Parameter estimate significant at the 95 per cent confidence level.

1. Parameter estimates from recomputing the regression models reported in Tables 6 and 8 while including the additional interaction variable UI received sometime since June 1991, multiplied by the programme participation indicator.

O'Leary 48 1

Tables 6 and 8 which present regression-adjusted estimates of programme impacts, suggest that recent UI receipt decreases the chance of getting into a normal job but increases re-employment earnings for those who find jobs. These results are consistent with a longer and more thorough job search by UI beneficiaries. Table 16 presents a summary of the results of adding to the models reported in Tables 6 and 8 the interaction between UI receipt and active labour programme participation. This allows a decomposition of the effects of active labour programmes between those with and those without recent UI benefit receipt.

The decomposition presented in Table 16 suggests that within the comparison group recent UI receipt tended to decrease re-employment by between eight and nine per cent, and increase monthly earnings by roughly HUF 1,900. For retraining, while the results are not statistically significant, any positive effect of retraining on the re-employment rate appears to have been entirely among those having recently drawn some UI benefits. The recent UI recipients who gained re-employment after retraining apparently did so at the cost of much lower earnings than the retrainees who did not draw UI recently. Recent UI receipt had a similar qualitative effect on PSE participants in the sample. That is, while those PSE participants without recent UI receipt were re-employed at a rate almost fourteen percentage points lower than the comparison group, PSE participants who recently drew UI gained re-employment at a rate on a par with the comparison group of registered unemployed.

In summary, unemployment insurance (UI) may positively influence entry to the unemployment register. It also probably prolongs the duration out of work for those who do not participate in retraining or PSE, but raises their re-employment earnings. Among retrainees, those who recently drew UI appeared more likely to gain re-employment, but at the cost of lower wages. Among PSE participants re-employment success for those who recently drew UI was on a par with the comparison group, while PSE participants who had not recently drawn UI experienced a significantly lower re-employment rate.

To get a better understanding of the effects of retraining and PSE in Hungary, it would be useful if some future evaluations could be based on larger samples perhaps involving random assignment of eligible persons to programmes and comparison groups. This would be a reasonable and equitable way to ration training courses and PSE slots when they are over-subscribed. The benefit in terms of understanding programme effects would be great.

Endnotes

1. Valuable suggestions for improving this paper were offered by Martin Godfrey, Gyorgy LWr, John Micklewright, Gyula Nagy, David Fretwell, Indermit Gill, Randall Eberts, Maria Frey, Kevin Hollenbeck, Kenneth Kline, an anonymous referee, and seminar participants at the W.E. Upjohn Institute for Employment Research, the Hungarian Ministry of Labour, and the World Bank. Funding for this work was provided by the International Labour Office/Japan Project in the Hungarian Ministry of Labour. The data were compiled by the National Labour Centre in Hungary. Richard Deibel and Kenneth Kline expertly performed the necessary estimation. Clerical assistance was provided by Claire Vogelsong and Ellen Maloney. Remaining errors are my own. Normal job means a job which is not subsidized in any way with money from the government's Employment Fund. In this paper we also examine the percentage in any job, which is a broader outcome measure.

2.


3.

4.

5.

6.

7.

8.

9.

10.

11.

12.

13.

Given the time periods involved, the majority of these people are probably receiving the new quasi-welfare benefit known as Unemployment Assistance rather than regular Unemployment Compensation. For examples of employment programmes evaluated using a classically designed field experiment see Decker and O’Leary (1995). See Fraker and Maynard (1987) for an interesting review and application of comparison group designs for evaluating employment-related programmes. That is, sampling was done with replacement. In neither the retraining nor PSE synthetic comparison group samples, did one observation from the full comparison group of 57 1 appear more than ten times. Weights for computing the standard error are one over the number of times an observation appears in the sample. This is equivalent to computing the standard error on a sample where each observation drawn appears only once. Using this upper bound on the standard error, we apply the weakest possible t-tests. Since the main dependent variable of interest-in a normal job-is binary, the regression model predicts the probability of re-employment. The OLS estimation is a linear probability model, which may yield biased estimates. Maddala (1982, Chapter 1) suggests using the logit estimation in such cases. All models reported in this paper were also estimated as logit models. Since re-employment probabilities for the training and comparison groups ranged from about 35 to 56 per cent, the limited range of the dependent variable did not cause severe bias in estimating parameters by OLS, indeed the OLS and logit estimates were nearly identical. While re-employment rates for PSE were much lower, still the logit and OLS estimates were nearly identical for the PSE models. Both results, logit and OLS, were reported in O’Leary (1995). In this application the regression model is a statement of an analysis of covariance methodology, where XI to Xi0 are the covariates. Mohr (1992, pp.83-87) discusses extending a regression model for programme impacts to include control variables. The potential covariates white collar worker and unskilled worker reported on in other tables were not used in the regression analysis because missing values would have dramatically reduced the sample sizes for the regressions. The obvious next procedure to adjust for differences across samples is to account for differences in unobservable characteristics. O’Leary (1995) reports on extensive attempts to do this using the methods of Heckman (1976). The effort failed essentially because no instruments were available which explained programme participation independent of re-employment success. A slightly smaller sample resulted because of missing values for some of the independent variables. The earnings equation was estimated only on those who were employed in a normal job on the survey date. A slightly smaller sample resulted because of missing values for some of the independent variables. The earnings equation was estimated only on those who were employed in a normal job on the survey date. For the subgroup analysis, Somogy is the omitted county indicator variable since a more extensive examination of the data by O’Leary (1995) revealed the greatest number of statistically significant cross-county differences in the sample to be between Somogy and the other two counties. A 1992 summary of follow-up surveys for the regional retraining centre (ERAK) at Miskolc found that 45 per cent of retrainees had been re-employed when surveyed one month after retraining, while 60 per cent were found to be re- employed when surveyed six months after retraining.

O’Leary 483

14.

15. 16.

Table 16 shows seven persons in normal jobs earlier than should be expected. Before June 1991 when they were registered as unemployed two persons appeared to have already obtained normal jobs which they still held in November 1993, and before September 1991, five persons in PSE appeared to have already obtained normal jobs which they still held in November 1993. Data errors were probably committed in these cases, but the observations were retained for analysis because of the small sample sizes and the supposition that these persons were indeed in normal jobs on the November 1993 survey date. A normal job is one not subsidized by money from the Employment Fund. O’Leary (1993b) reported that in the first quarter of 1992 an average of 459,000 were registered as unemployed with the public employment exchange compared with an estimate of total unemployed of 423,000 for the same period based on the Labour Force Survey done by the Hungarian Central Statistical Office. While the Labour Force Survey oversamples in urban areas where unemployment is lower, the two numbers suggest a high registration rate among unemployed.

References

Decker, P. T. and C. J. O’Leary (1995), ‘Evaluating Pooled Evidence from the Re- employment Bonus Experiments’, Journal of Human Resources, Vo1.30(3), pp.534- 50.

Fraker, T. and R. Maynard (1987), ‘The Adequacy of Comparison Group Designs for Evaluations of Employment-Related Programs’, Journal of Human Resources,

Godfrey, M., G. U d r , and C. J. O’Leary (1 993), Report on a Survey of Unemployment and Active Labour Market Programmes in Hungary, Budapest: International Labour OfficelJapan Project.

Godfrey, M. (1 995), ‘The Struggle Against Unemployment: Medium-Term Policy Options For Transitional Economies’, International Labour Review, Vol. 134( 1 ),

Heckman, J. (1976), ‘The Common Structure of Statistical Models of Truncation, Sample Selection, and Limited Dependent Variables and a Simple Estimator for such Models’, Annals of Economic and Social Measurement, Vo1.5, pp.475-92.

Maddala, G. S . (1983), Limited-Dependent and Qualitative Variables in Econometrics, Cambridge: Cambridge University Press.

Micklewright, J. and G. Nagy (1994a), ‘How Does the Hungarian Unemployment Insurance System Really Work?’, Economics of Transition, V01.2(2), pp.209-32.

Micklewright, J. and G. Nagy (1994b), ‘Flows To and From Insured Unemployment in Hungary’, European University Institute Working Paper in Economics, No. 9414 1.

Mohr, L. B. (1992), Impact Analysis for Program Evaluation, London: Sage. O’Leary, C. J. (1993a), An Impact Analysis of Labour Market Programs in Hungary

based on the November 1992 ILO Survey of Participants, Kalamazoo: W.E. Upjohn Institute for Employment Research.

O’Leary, C, J. (1993b), Innovation in Planning Hungarian Labour Market Programs, Labour Market Analysis and Employment Policies Working Paper No.61, Geneva: International Labor Office.

V01.22(2), pp.207-27.

pp.3-15.

4 84 A net impact analysis of active labour programmes in Hungary

O’Leary, C. J. (1993, An Impact Analysis of Employment Programs in Hungary, Staff Working Paper 95-30, Kalamazoo: W.E. Upjohn Institute for Employment Research.

Documents

A net impact analysis of active labour programmes in Hungary