2010 Improvement of the competitiveness of enterprises: analysis of net effects Prof. Jarosław...

2010

Improvement of the competitiveness of enterprises: analysis of net effects

Prof. Jarosław GórniakRafał Trzciński

26.02.2010

Objective of the evaluation study

The main objective of the study was to estimate and assess the real effects of Sectoral Operational Programme Improvement of the Competitiveness of Enterprises, years 2004-2006 (SOP-ICE). Two measures were covered by the study:

Measure 2.1 – Improvement of competitiveness of SMEs through advice,

Measure 2.3 – Improvement of competitiveness of SMEs through investments

Background of the problem

?

Factor I

Factor II

Problem: low

competitiveness of enterprises

SOP-ICE

Effects

The research problem

To what extent has the intervention contributed to the observed effects?

What is the net effect of the implemented programmes?

Net effect – the difference between what is observed and what would have been observed had the

intervention not been implemented

How to measure the net effect?

Revenues from sale:

€ 2 million € 1,5 million

Net effect =

Counterfactual

0,5 million

€ = ?

?

Counterfactual € = ?

Propensity score matching

Propensity score matching method helps to ensure that the treatment and control groups are similar in terms of observable characteristics (reduction of selection bias).

The propensity score is the probability of assignment to treatment (program, project, etc.) given a set of covariates X.

Propensity score is unknown and needs to be estimated, for instance, by using logistic regression.

The main assumption: control group shows what would have happened to participants if they had not participated in the programme (selection bias is eliminated by conditioning on a set of observable variables).

Propensity score matching in its simplest form

NON-PARTICIPANTSCONTROL GROUP

ps= 0,6ps= 0,5

ps= 0,8ps= 0,1

ps= 0,2

ps= 0,3ps= 0,2

ps= 0,01

ps= 0,4

ps= 0,9

TREATMENT GROUP (PARTICIPANTS)

ps= 0,8 ps= 0,3

ps= 0,9 ps= 0,4

ps= 0,1

Evaluation of the Sectoral Operational Programme Improvement of the Competitiveness of Enterprises

Evaluated programs

Measure 2.1 was aimed at increasing the competitiveness of Polish SMEs through facilitating their access to specialised advisory assistance.Budget: PLN 59,6 million (average level of co-

financing amounted to PLN 27,8 thousand). Measure 2.3 was aimed at improvement of

competitiveness of Polish SMEs through modernisation of their product and technological offer.Budget: PLN 1 489,6 million (average value of co-

financing amounted to PLN 505,4 thousand).

Data sets used in the evaluation PAED data sets (databases of all applicants; databases

containing information on the participation in other programs – Phare SME, SOP-HRD), 19 covariates such as age, revenue, size, legal form,

assets, the use of other funding etc. Central Statistical Office data sets collected in the so-called F-

01/I-01 form (Report on revenues, costs and financial results and the cost of fixed assets) 30 outcome variables such as net revenues from sale,

average paid employment, expenditures, costs, etc. In the first case the data were used for the selection of the

control group, In the second case the data were used to estimate the net

effect of the support.

Model review (Measure 2.3)

Variable Treated

(mean value)Untreated

(mean value)Control group(mean value)

The standardized difference in percent

(before matching)

The standardized difference in percent

(after matching)Total employment 73,3095 203,6780 70,2619 -1,48 0,03Employment of women 20,5097 171,3047 18,4324 -1,71 0,02Revenues 68032,5452 133853,2120 77556,1103 -5,27 -0,76Assets 39844,9119 158897,6398 31228,8509 -3,17 0,23Amount of deminimis 14170,8455 10479,6290 14917,1959 4,66 -0,94Age 10,6512 10,7992 10,6376 -1,77 0,16Percentage of women ,2714 ,3343 ,2745 -25,75 -1,26Employment growth before the programme

,6571 ,5367 ,6700 24,72 -2,66

Revenue growth before the programe

,7288 ,5981 ,7171 27,92 2,51

Assets growth before the programe

,7082 ,6067 ,7006 21,52 1,62

Number of contracts signed in Phare

1,2365 ,4825 1,3088 57,34 -5,50

Value of signed contracts in Phare

17417,7968 5771,3597 17886,0127 49,78 -2,00

Application in SOP-ICE 2.1 ,0776 ,0121 ,0700 32,06 3,74Contract in SOP-ICE 2.1 ,0518 ,0059 ,0453 27,67 3,90Contract in SOP-HRD 2.3 ,1406 ,0152 ,1194 48,10 8,12… … … … … …

Results

Effects of advisory program (Measure 2.1)

Net effect:PLN 2 942thousand

43,3%

Effects of advisory program (Measure 2.1)

Net effect:PLN 861thousand

Effects of advisory program (Measure 2.1)

79,2%

24,3%

Net effect:PLN 1 077thousand

Effects of advisory program (Measure 2.1)

Net effect:PLN 2 864thousand

30% 44,6%

Effects of advisory program (Measure 2.1)

29,4% 20,8%

Net effect:5 FTE

Effects of advisory program (Measure 2.1)

Net effect:PLN 335thousand

65,4% 41,9%

Effects of advisory program (Measure 2.1)

Net effect:PLN 542thousand

48,4% 85%

Effects of advisory program (Measure 2.1)

Net effect:PLN -72thousand

71,4%

18,4%

Effects of advisory program (Measure 2.1)

Net effect:PLN 1thousand

105,9% 100%

Effects of investment program (Measure 2.3)

Net effect:PLN 2 359thousand

47,6%

48,1%

Effects of investment program (Measure 2.3)

Net effect:PLN 1 391thousand

55,5%

70,1%

Effects of investment program (Measure 2.3)

Net effect:PLN 829thousand

42,4%

Effects of investment program (Measure 2.3)

Net effect:PLN 2 429thousand

46,9%

Effects of investment program (Measure 2.3)

Net effect:14 FTE

20,5%

34,1%

Effects of investment program (Measure 2.3)

59%

Effects of investment program (Measure 2.3)

Net effect:PLN 652thousand

74,7%

95,6%

Effects of investment program (Measure 2.3)

Net effect:PLN 223thousand

72,4%

303,7%

Effects of investment program (Measure 2.3)

9,5%

260,4%

The main conclusions

The main conclusions

SOP-ICE improved competitivness of participating SMEs.

The effects of advisory programme (2.1) clearly surpassed the effects of investment programme (2.3) at the time of evaluation.

Results assessment in relative terms differs from calculated in absolut terms – percent versus money.

Estimation of the net effect ≠ evaluation

Net effect measurement addresses effectivness of the programme and does it well.

Among others, it does not cover non-intended consequences of the programme – is fully goal based.

It has to be supplemented with insight addressing other evaluation criteria guided by the (re)constructed programme theory.

Thank you for your attention!

Polish Agency for Enterprise Development81/83 Pańska Street

00-834 Warsaw, Poland

Tel + 48 (22) 432 80 80Fax + 48 (22) 432 86 20

+ 48 (22) 432 84 04

Infoline: + 48 (22) 432 89 91/92/93www.parp.gov.pl