A new approach to the economic evaluation of public programmes

Presupuesto y Gasto Puacuteblico 662012 85-108 Secretariacutea de Estado de Presupuestos y Gastos

copy 2012 Instituto de Estudios Fiscales

A new approach to the economic evaluation of public programmes on business internationalisation The case of the Diagnostic Programme in Spain

JOSEacute M CANSINO

JAIME LOacutePEZ-MELENDO

MARIacuteA DEL P PABLO-ROMERO

ANTONIO SAacuteNCHEZ BRAZA

Universidad de Sevilla

Recibido Noviembre 2011 Aceptado Enero 2012

Abstract

The study proposes a new approach to the economic evaluation of public programmes on business internationalisashytion using a methodology based on statistical causal inference It represents a new perspective as compared to preshyvious approaches like surveys or cost-benefit analysis The paper considers the case of the Diagnostic Programme in Spain The methodology used is the selection of observables based on procedures such as subclassification matshyching or the estimation of the propensity score Consequently one response variable (earnings before financial exshypenses and taxes attributable to exports) and four pre-determined covariates (stage in the process of internationalisashytion size product and location) are considered After applying the above-mentioned procedures the analysis draws the conclusion that the propensity score-matching method is an adequate tool to evaluate the Diagnostic Programme

Key words public policies internationalisation statistical causal inference diagnostic programme

JEL Classification H43 H59 O24

Research Highlights

gt This paper discusses a new method to evaluate internationalisation programmes gt It considers the case of the Diagnostic Programme in Spain gt One response variable and four covariates are determined before implementing the methodology gt The analysis concludes that the propensity score can be used to evaluate this Programme

Resumen

El trabajo aborda el anaacutelisis de un programa de impulso a la internacionalizacioacuten de la empresa andaluza mediante el uso de la inferencia causal estadiacutestica Representa un nuevo enfoque respecto a otros estudios basados en encuestas o en la reshylacioacuten costebeneficio El meacutetodo de anaacutelisis que se propone es el de seleccioacuten de variables observables con los procedishymientos de subclasificacioacuten matching y propensity score En consecuencia se consideran una variable respuesta (gananshycias antes de gastos financieros y de impuestos atribuible a las exportaciones) y cuatro variables predefinidas (fase del

The authors acknowledge financial supporty SES-132 (Research Group in Econoic theory and Political Economy)

86 Joseacute M Cansino Jaime Loacutepez-Melendo Mordf del P Pablo-Romero y Antonio Saacutenchez Braza

proceso de internacionalizacioacuten tamantildeo producto y localizacioacuten) Aplicados estos procedimientos se concluye que los procedimientos de ldquomatchingrdquo y ldquopropensity scorerdquo se pueden utilizar para evaluar el Programa de Diagnoacutestico Ademaacutes se hace referencia a los datos que necesita la organizacioacuten que lo desarrolla para llevar a cabo la evaluacioacuten

Palabras clave poliacuteticas puacuteblicas internacionalizacioacuten inferencia causal estadiacutestica Programa de Diagnoacutestico

Clasificacioacuten JEL H43 H59 O24

1 Introduction

Internationalisation can be defined as a gradual and evolutionary process in which firms progressively increase their involvement in international business It is a beneficial process for both the company and the national or local economy Internationalisation may generate econoshymies of scale in local firms and promote the transfer of technology and management knowlshyedge thus boosting growth and employment (Austrade 2002) These benefits explain the imshyplementation of export promotion activities and programmes financed by public funds they will justify the costs associated with this public expenditure It is therefore interesting to know the extent to which those export promotion policies (EPPs) undertaken by governments are profitable or not ie their development needs to be assessed in order to see whether their applishycation is effective or can be improved through a new programme design Cansino (2003) conshysiders the background of the programme as a conflict between the effective application of the public funds and the implementation of policies determined by the interests of the performing agent Therefore it is convenient to carry out evaluations of these policies because on the whole the research developed during the 1980s the 1990s and into the new millennium reshymains somewhat inconclusive regarding the effectiveness of EPPs (Brewer 2009)

As pointed out by Seringhaus (1990) the evaluation of the impact of export promotion programmes on exports became a concern in the late 1970s Since then the literature on the economic evaluation of public programmes of internationalisation has been based on surveys which reflect the recipientsrsquo assessment and quantitative studies particularly cost-benefit analyses

Among the former ie those studies based on surveys the work of Albaum (1983) can be highlighted It underlined the fact that the programmes were generally unfavourably valshyued and revealed a lack of understanding between the government and the small businesses about the role and value of such programmes Seringhaus (1990) valued the information gathered through the surveys but cast doubts on its reliability Wilkinson amp Brouthers (2006) used surveys conducted with a four-item scale to measure the level of satisfaction among respondents The use of surveys to assess export promotion programmes has been questioned by many authors As noted by Brewer (2009) these surveys may be unsatisfacshytory due to various reasons such as the firmsrsquo reluctance to criticise a programme that in many cases had no cost for them the diversity of the respondentsrsquo opinions as evidenced by Crick amp Czinkota (1995) or the pressure from the programme providers (Seringhaus amp Rosson 1990) Thus as stated by Francis amp Collins-Dodd (2004) the use of these surveys can be criticised as lacking in objectivity

87 A new approach to the economic evaluation of public programmes

Among the quantitative studies it is worth mentioning the early work by Pointon (1978) which evaluated the impact in terms of additional exports generated by British export promotion programmes The work of Coughlin amp Cartwright (1987) which analyzed the elasticity of export promotion in the USA by estimating the effect of an extra unit of expense in terms of additional export units can also be considered Or the evaluation conducted by Marandu (1995) which analysed the effect of export promotion programmes directly on exshyport performance From a macroeconomic point of view some studies have attempted to esshytimate the relationship between aggregate export promotion spending and aggregate export performance In this sense the studies by Camino (1991) Armah amp Epperson (1997) and Richards et al (1997) are to be remarked

These studies have been criticised by several authors Seringhaus (1986) considered that the methods applied in them required a sceptical evaluation He inquired about the definition of costs or the assumption that all exports made in a specific year are related to the governmentrsquos promotional expenditure in the previous year According to Cadwell (1992) it is impossible to relate public export promotion activities to overall state exports or to the exports of those firms that have received support because many factors come into play at the firm level Also Genccediltuumlrk amp Kotabe (2001) highlighted the impossibility of linking the result of those efforts to a single eleshyment In the same way Gillespie amp Riddle (2004) justified their rejection of an analysis based on the relationship between total exports and the application of those programmes

Therefore the subjectivity of the responses in survey research is called into question and the limitations of quantitative analyses are also stressed According to Brewer (2009) the difficulties associated to research on these programmes and the researchersrsquo failure to generate a common opinion on the results have led to a decrease in the number of studies on this topic in recent years

This article explores the capability of statistical causal inference methods to evaluate public programmes for the internationalisation of export firms The proposed approach allows a significant improvement of the economic evaluation of such public programmes and is an useful instrument for policy makers interested in assessing the results of these policy measures These methods overcome many of the difficulties encountered during the economic evaluation of these programmes In contrast with the subjectivity of surveys they contribute to an objecshytive evaluation of the effects of public policies allowing unlike quantitative analyses to detershymine the increase in the export figures attributable to these programmes

These methods based on statistical causal inference have been largely and successshyfully applied in the fields of Medicine Criminology and Finance The evaluations of public training programmes developed by Card amp Sullivan (1988) and Manski amp Garfinkel (1992) for the USA Bonnall Fougegravere amp Seacuterandon (1997) for France Andrews Bradley amp Upward (1999) and Blundell et al (2004) for the UK Bergemann Fitzenberger amp Speckesser (2005) for Germany and Park et al (1996) for Canada deserve a special mention Mato (2002 2010) Arellano (2005 2010) Mato amp Cueto (2008 2009) and Cansino amp Saacutenchez Braza (2008 2009 2011) can be counted among the main works on the Spanish case

In this work the possibilities of a method based on the observation of covariates in a sample of firms are explored in a non-experimental context We discuss the methodrsquos capashy


bility to perform an economic evaluation of public programmes on business internationalisashytion through the case of the Diagnostic Programme designed to boost the internationalisation of small and medium-sized firms in Andalusia (Southern Spain)

In regard to the structure of the study section 2 after the introduction deals with the general aspects of the Diagnostic Programme It also describes the main elements of the dashytabase provided for the analysis Section 3 offers an overview of the statistical causal infershyence methodology applied to public policy evaluation After defining the analytical frameshywork section 4 considers the potential variables related to the Diagnostic Programme while section 5 examines the application of certain procedures to the Programme according to these variables Section 6 contains the lesson learns and Section 7 the main conclusions

2 The Diagnostic Programme

The Diagnostic Programme was devised by Extenda with the purpose of boosting the process of internationalisation of small and medium-sized firms in Andalusia and is one of the various plans designed for the internationalisation of Andalusian firms

Extenda is a body of the Andalusian Regional Government whose aim is to carry out different activities to promote the internationalisation of the regionrsquos economy In this sense several internationalisation plans have been developed since 1999 The first Plan (1999-2002) was followed by the 2003-2006 Plan and subsequently by the Plan for the Internationalisation of Andalusian Firms of 2007-2010 (Consejeriacutea de Turismo Comercio y Deporte 2006) 1

The 2007-2010 Plan considers a series of problems of the Andalusian export scenario among which the small number of export firms and the disparity of their export activities could be highlighted Other weaknesses are the high concentration of exports in European Union countries and the limited presence of Andalusian firms in emerging high-growth areas like Asia

One includes non-exporting firms with the capability to export As a rule they have a good product that could be sold abroad as a result of the companyrsquos experience and strength in the domestic market The firms in the second segment are already exporting their goods and services often in a standardised way Occasionally these firms have adjusted part of their production structure to export requirements (packaging labelling etc) However they lack a clear strategy of internationalisation Finally the third segment is formed by internashytionalised firms operating in a wide range of markets and often being physically present in other countries They have a strategy of internationalisation and an organisational structure that supports international business

From the above classification the 2007-2010 Plan emphasises the first two segments In the first case the objective is to push the firms to acquire the know-how necessary to unshydergo the process of internationalisation As for the second segment it is vital that the firms make a better market identification and that they broaden their range of products introduce their products into new markets and reconsider their structure in order to boost international


business This plan also prioritises actions that aim at adjusting the firmsrsquo potential to the acshytual demand as opposed to traditional internationalisation methods such as the attendance to fairs

In this framework the Diagnostic Programme is oriented to Andalusian firms in the preliminary or initial stages of the internationalisation process Extendarsquos programme conshysists in providing individualised assistance by an expert consultant specialised in internashytional development The planned duration for the development of this initiative is three months The process aims at helping identify or verify the firmsrsquo potential and at adopting decisions concerning the firmsrsquo supply market positioning target markets business segshyments and possible market access channels The process concludes with the drawing of an action plan that sets the internationalisation targets to be achieved in a given period of time

The only essential requirement for the participation of a firm in the programme is to have a head office local office manufacturing centre or service centre in Spain The comshypanyrsquos economic contribution to the programme amounts to 500 euro The average cost of the programme for the financing body is around 7000 euro

For the development of this work Extenda has kindly provided a database with informashytion on the companies participating in the programme Each company has been assigned an identification code in order to ensure data privacy The number of participating firms in the peshyriod 2002-2010 totals 593 The database specifies the type of activity developed by each comshypany although this is an internal reference that does not correspond with a standard classificashytion It also indicates the Andalusian province where the companyrsquos head office or work centre is located Finally data such as the firmsrsquo sales exports and staff figures between 2006 and 2008 are also included Figure 1 offers information about the firms involved in the programme

Figure 1 Firms benefited from the Diagnostic Programme (2002-2010)

0 20 40 60 80 100 120

2010

2009

2008

2007

2006

2005

2004

2003

2002 15

33

42

49

100

89

99

112

54

2002 2003 2004 2005 2006 2007 2008 2009 2010

Companies 15 33 42 49 100 89 99 112 54

Until May Source Extenda


3 Evaluation by using statistical causal inference

The use of statistical causal inference for the evaluation of public policies allows reshysearchers to estimate the causal effect of a public policy (the cause) on one or more variables of interest (the effect) Researchers try to isolate the effect of the policy on the variable of inshyterest by keeping other factors affecting the variable under control If conditions are not idenshytical the effects cannot be exclusively attributed to the implemented policy

The development of public policy evaluations has benefited from the use of causal inshyference 2 One of the results is the design of the Potential Outcome Model (POM) which alshylows comparing participants and non-participants in public programmes 3 A prolific develshyopment of the POM applied to the evaluation of training programmes can be found in Rubin 4 (1974 1978) This work leans on the Rubin Causal Model (RCM)

According to Rubin (1974) and Holland (1986) the definition of a treatment indicator Di as a binary variable for any firm that could potentially participate in the programme is reshyquired Di = 1 indicates that the firm has participated and Di = 0 indicates that it has not The indicator allows researchers to identify the status of the firms by distinguishing between treated and untreated

Once the indicator has been specified it is necessary to determine the response varishyable defined as the variable of interest ie the effect of the evaluated policy to be measured Considering the objectives of the Diagnostic Programme the proposed response variable or outcome (Y) is the export figure (earnings before financial expenses and taxes attributable to exports) defined as the net turnover derived from the supply of goods produced by Andalusian firms outside Spain over one economic cycle

However this variable needs to be clarified because the margins attributable to each monetary unit can substantially differ depending on the type of activity concerned This fact should be taken into account since it warns against the comparison of the export figures of different activities One response variable with a greater degree of homogeneity than profit attributable to exports is profit before interest and taxes This alternative outcome considers all the operating costs of the product including depreciation The main drawback is the diffishyculty in allocating operating costs especially indirect costs (overheads and where applicashyble depreciation expense)

Also the temporal aspect of the response variable must be taken into account Internashytionalisation is a dynamic process that requires efforts and training At the same time it is inshyfluenced by other factors such as the economic cycle or the economic background This imshyplies the need to contemplate a period of time longer than a year A reasonable scenario involves considering the average exports figure over the four or five years following the parshyticipation in the Diagnostic Programme

After selecting the most appropriate variable it is necessary to reckon that there will be potential responses associated to the firmrsquos status (concerning its participation or lack of parshyticipation in the programme) These potential responses will be denoted as Y0i (value of the response variable if the ith firm has not completed the programme) and Y1i (value of the reshysponse variable if the ith firm has completed the programme) The programmersquos causal effect


on the response variable is determined by the difference Y1i ndashY0i thus allowing the evaluashytion of the effectiveness of the programme

However it is impossible to observe both responses simultaneously because this is a counterfactual event For each company the only response that can be obtained is Yi deshyfined as

Yi = DiY1i + (1 ndash Di) Y0i (1)

According to Holland (1986) this fact known as the fundamental identification probshylem in causal studies makes it impossible to determine the individual causal effects of a programme The counterfactual nature of the potential responses reveals the need to search for a second best alternative This will involve estimating the average causal effect through a comparison between treated and non-treated firms

A high degree of homogeneity among the potential beneficiaries which implies that they give identical or similar responses would solve the problem However contrary to what happens in disciplines such as Physics or Chemistry potential responses in social experishyments tend to be heterogeneous because the same programme can stimulate very different reshysponses from the recipients This makes the calculation of a programmersquos individual causal effects especially difficult This is why calculating the average effect of a programme be-comes necessary in order to control the factors leading to this heterogeneity

The theoretical framework for the development of these evaluation methods derives from the above-mentioned Rubin Causal Model 5

Once the calculation of the average effect is considered in order to proceed with it the estimates need to be defined Thus the average treatment effect ATE is defined as the difshyference between the average value of the response variable of the companies that particishypated in the programme and the average value of the response variable of those that did not

E = E Y1 - 0 ) E Y 1) - ( 0 )aacute AT ( Y = ( E Y = (2)

= E Y 1) - ( 0D = E Y D = 0) = E Y( D = 1) - E Y ( D = 0)( 1

Alternatively the effect can be calculated only for the participating firms this would be the average treatment effect on the treated ATET In this case the calculation of the average effect would be restricted to firms that completed the programme

aacute E Y 1) = ( 1D = E Y 1) - ( 0D = E Y D = 1) (3)ˆ ATET = ( 1 -Y0

Once the estimates are defined a treatment group and a control group should be detershymined In random experiments firms are randomly assigned to the treatment and the control group The randomisation of the assignment makes it possible to compare treatment and conshytrol groups by ensuring the independence of the potential responses This independent status implies that the possible participation in the programme is not linked to the potential reshysponse of the participant


(Y1 Y0) D (4)

Although several attempts (Gertler 2004 Angrist amp Lavy 1999) have been made in this sense these experiments widely used in other fields can hardly be implemented in the case of the Social Sciences because of possible economic moral or ethical and time difficulshyties We must also consider potential drop-outs if the programme does not meet the firmrsquos expectations nor compensates the effort required for participation or when circumstances beyond the firmrsquos control prevent the company from completing the programme Another factor to be taken into account is the well-known ldquoHawthorne effectrdquo which is the improveshyment of an observed variable as a result of the motivational stimulus generated Stock amp Watson (2003) pointed out some drawbacks that can be encountered during practice and Cansino (2006) and Saacutenchez Braza (2006) mentioned others identified in training programmes

Given these restrictions the estimation of causal effects must be inferred from observashytional data in non-experimental contexts in order to allow the researchers to reproduce the setting of an experiment based on the so-called observational methods (Lalonde 1986)

The validity of the estimated average effect may diminish if the firms included in the treatment and the control group differ in characteristics other than those derived from the fact of their participation in the programme That is the validity may be affected by sample selection bias problems which result from using non-randomly selected samples (Heckman 1976) 6 Therefore these characteristics need to be controlled and the effects that they may produce on the values of the response variable must be taken into consideration Insofar as the features that are not associated to the participation in the programme can be perceived and the firms (participants and control) differ only in them these differences are susceptible to being controlled This is the basis for non-experimental methods of selection on observshyable variables

Selection on observables allows us to isolate the effect of a predetermined characterisshytic or covariate (or vector of covariates) while maintaining the independence between varishyable D (the indicator of participation) and response variable Y Thus it is possible to preserve the independence condition required in random experiments to allow the comparison beshytween the participants and the control group This condition can be expressed now as

(Y1 Y0) D | X (5)

Therefore the covariate (or vector of covariates) X needs to be defined The covariate X is a default variable with respect to D if for each ith company X1i = X0i ie if the value Xi does not depend on Di for each and every one of the observations considered According to Heckman and Hotz (1989) selection on observables is recommended when the independshyence of D and Y is due to the covariate (or vector of covariates) X which has influenced the individual selection process Thus by controlling X we give a solution to possible sample seshylection bias problems eliminating the dependency between D and Y


When the independence condition is not guaranteed it is possible to use an alternashytive method based on instrumental variables insofar as an adequate instrument to detershymine D is available In this case an instrument (one of the individualsrsquo characteristics) is required to induce exogenous selection for treatment (see Imbens and Angrist 1994) The instrument must be uncorrelated with Y and needs to satisfy several other requirements beshyfore being accepted as an instrumental variable 7 For the context of this study and considshyering the available data it is possible to affirm that none of the individualsrsquo characteristics has a determining influence on the selection for treatment and is at the same time uncorrelated with the response variable Y Consequently this method proves to be inadeshyquate

Finally when it is assumed that controlled and treated can differ in unobserved characteristics like for instance psychological ones the average effect can be measured with the differences-in-differences estimator (Card and Krueger 1994) In order to imshyplement this estimator it is necessary to have longitudinal data of the response variable available and referred to at least two different moments in time before and after the comshypletion of the programme For this reason the application of this estimator to the context of this work will not be feasible since for the evaluation of the Diagnostic Programme earnings attributable to exports are taken as the response variable Y and they are usually very small or even equal zero for most of the firms before the programme starts Moreshyover this methodology is mostly used when there are clear signs of the conclusive influshyence of certain psychological characteristics or aptitudes on the results something that is unlikely in this case

Hence the estimation of the causal effect of the Diagnostic Programme can be inferred from observational data by applying the selection on observables method However before describing the method the covariates of interest need to be defined

4 Definition of the covariates

Table 1 shows the four covariates considered all of which can affect the outcome of the programme Further access to appropriate micro-data has been taken into account before seshylecting these covariates The covariates are the stage of the process of internationalisation in which the company finds itself the size of the company the type of business activity develshyoped by it and more specifically the type of product that is or could be exported and fishynally the companyrsquos location


Table 1

Covariates

COVARIATES CATEGORIES

Stage in the process of internationalisation The company is completely unrelated to exports

The company has occasionally exported its goods The company usually exports its goods

Size Micro company Small company

Medium-sized company

Divisiongroupclass Class 1 Class 2

Product Kind of product Type 1 Type 2

Market positioning High segment

Medium segment Low segment

Location Province County

Area

Source Own elaboration

41 Stage in the process of internationalisation

The concept of internationalisation is intimately linked with the idea of process thereshyfore each company can be situated in a different stage of it The ability of a company to reshyspond to public incentives will largely depend on its progress in the process of internationalishysation

A company that has not started its process of internationalisation will be on less advanshytageous terms to carry out the action plan of the Diagnostic Programme A company that has initiated this process even if still in its initial stage will have more experience to meet the objectives of the plan agreed with the consultant

Accordingly this variable can take two values 0 for the firms that have not started the process and 1 for those that are at an early stage of it Alternatively three other values can be assigned to respectively the companies that are completely unrelated to the process of internationalisation the firms that have occasionally exported and have now and then particishypated in a fair or a trade mission and finally the firms that export their goods more or less systematically and apply their promotional tools with some regularity but are lacking a clear strategy on how to channel this process


42 Size

The influence of this variable is determined primarily by the firmrsquos capacity to commit resources Cavusgil amp Naor (1987) concluded that exporting firms tend to be larger than non-exporters Knight amp Liesch (2002) emphasised that small and medium-sized businesses often have fewer resources and capabilities and less market power Wilkinson amp Brouthers (2006) pointed out that the ability to identify appropriate partners and distributors should be a key element in the market strategy of small and medium-sized export companies

The process of internationalisation necessarily involves the commitment of human and financial resources the larger the size of the company the higher the capacity In fact an opshyportunity to start business abroad can be determined by the size of the firm

The more common criteria to classify companies according to their size are their numshyber of workers their sales figure their assets and in the case of listed firms their financial market capitalisation Often two or more criteria are used together in order to determine the size of the firm since the choice of only one characteristic could lead to an unrealistic classishyfication This fact determines the use of several criteria at one time especially in the accountshying corporate and tax areas

Regardless of the magnitudes established for the definition of the intervals five busishyness segments based on size can be distinguished micro small medium-sized and large firms and finally multinational firms or large corporate groups that operate in many counshytries and often in different market sectors

In the case of the Diagnostic Programme the subjective frame for which it is developed automatically excludes the two upper segments Certainly the programme is not designed to promote the internationalisation of a large company let alone a multinational In the first case because with rare exceptions the process of internationalisation is already under way and if it is not these firms have alternative routes to the use of such programmes In the case of multinationals because it is clear that the internationalisation of the companyrsquos activity is a quality or intrinsic component of the business developed

Therefore the firms can be classified into three segments depending on their size mishycro small and medium-sized firms Thus the default variable ldquosizerdquo can take three values

43 Product

The product determines both the structure of the company and its management Acshycording to the product certain aspects such as the companyrsquos assets technology location product marketing customers and suppliers and national and international competition can be defined Different products require different business management strategies both in the domestic and the international market

If the National Classification of Economical Activity (NCEA) mdashdivisiongroupclassmdash is applied an initial grouping must take place so that the reference is set with regard to a prodshyuct class which belongs to a group that in turn belongs to a specific division Once the assignshy


ment to a classgroupdivision is fulfilled the second grouping criterion for comparison purshyposes will be the particular type of product Finally in relation to aspects such as product quality or design which however do not affect extremely generic goods a third level of classishyfication will be determined by the productrsquos market positioning which can be split according to the traditional distinction into high medium and low segments

This means that the default variable can take a large number of values The (NCEA) identifies up to 98 divisions each of them comprising their own series of groups which in turn are divided into their respective classes of products

In summary the covariate can take a variety of values because the participating firms belong to various productive sectors and within these sectors the products that they offer are very heterogeneous in their characteristics and in their market positions

44 Location

The companyrsquos location affects mainly its transport expenditure The proximity to a main road or mdashespecially in the case of export firmsmdash a sea port is an unquestionable comshypetitive advantage

In terms of distribution the consideration of the province as the unit of analysis has one disadvantage in the case of Andalusia namely the large size of some of the Andalusian provshyinces (eg Seville Cordoba and Jaeacuten) and the fact that their territory comprises very heteroshygeneous areas For this reason the best delimitation of the companyrsquos location is the comarca (a Spanish territorial division similar to the British county) a variable that in pracshytice can take multiple values

5 Methods using selection on observables

Once the covariates are defined one of the methods using selection on observables can be implemented The procedures of subclassification matching and those based on a previshyous estimation of the propensity score mdashall of them reviewed by Imbens (2004)mdash are counted among those methods

51 Subclassification

In the subclassification method once the covariate X is identified its potential values must be specified Depending on these possible values the subclassification is then carried out by dividing the treatment and control groups into different subclasses

If X is a discrete variable there will be a subclass of firms for each value that X can take The number of subclasses is finite If X is a continuous variable it will be necessary to construct a finite number of intervals each constituting a subclass The subclassification of


1 2 3the population originates a set (X X X Xk) of k subclasses Thus n is the number of firms considered of which n1 are part of the treatment group (D = 1) and n0 constitute the control group (D = 0)

If k is the number of subclasses within the population the firms that make up each subshyclass are determined as follows

nj is the total number of firms that make up subclass j

n1 j is the number of firms in the treatment group that belong to subclass j

n0 j is the number of firms in the control group that belong to subclass j

In any case it is true that

j j jn + n = n1 0 (6)

1 + n2 + n kn 3 + + n = n (7)

njn is the weight of subclass j in relation to the total population and n1 j n1 is the weight

of the participating firms in relation to the total number of firms belonging to this subclass In any case

Y measures the average value of the response variable

Y j is the mean of the response of the participating firms belonging to subclass j1

Y j is the mean of the response of the firms in the control group belonging to subclass j0

Within each subclass both groups are formed by firms of similar characteristics This allows estimating the average treatment effect (ATE) adding in each subclass the differshyences in the observable mean values of the response variable These differences are weighted by the weight of each subclass Analytically this can be expressed as

k jj j n

aacute ATE -SUBCLASS = L ( Y1 - Y0 ) ( ) n (8)

j=1

And in relation to the participants only (ATET)

k jj j n

aacute ATET -SUBCLASS = L ( Y1 - Y0 ) ( )1

n (9)j=1 1

In regard to the variable ldquostage of internationalisationrdquo subclassification is possible in general terms because the variable can be split into two subclasses Alternatively three subshyclasses can be identified However this subclassification will be influenced by the inclusion of companies in the treatment or the control group that may belong to highly differentiated sectors and thus affect the values of the response variable


The same consideration can be made regarding the covariates ldquosizerdquo and ldquolocationrdquo In the first case a subclassification may be carried out in accordance to the companiesrsquo size into categories of micro small or medium-sized firms In the second case according to the chosen territorial division

In the case of the covariate ldquoproductrdquo given that the Diagnostic Programme has no secshytoral vocation the diversity of values that the covariate may take prevents the putting into practice of the subclassification method

52 Matching

In the process of matching a company from the control group is assigned to a particishypating company having covariates of the same (or approximate) value and vice versa a company from the treatment group is assigned to a company in the control group having covariates of the same (or approximate) value This technique allows matching firms with similar observable characteristics If the matching of firms includes all these default varishyables this procedure provides an unbiased estimate of the effects of the programmersquos impleshymentation Matching is a procedure of comparison even more stringent than subclassificashytion

Company i participating in the programme with a specific value Xi for the default varishyable is matched with company m from the control group with a Xm value for that same varishyable so that Xi = Xm or in case this is not possible Xi ~ Xm The exact match will not always be possible given that in this type of research the entities have many observable charactershyistics and exact pairs are difficult to find This will often lead to inaccurate matches and will demand the definition of the criteria of proximity in order to set the conditions of the match

Following the same procedure each company in the control group will be assigned a company from the treatment group that meets the condition of equality or similarity of the value of covariate X

The difference between the observed value of a company and the observed value of the company assigned as its match (Yi ndash Ym (i)) allows obtaining the estimator of the effect of the evaluated programme aacuteATE according to the expression

n1 aacuteATE -MATCHING = L ( i - m i ( ) )Y Y (10)n i=1

In the same way this procedure can be carried out for the participating firms ie taking into consideration the companies belonging to the treatment group only In this case the matching estimator will be aacuteATET Analytically it can be expressed as

1 n1

aacuteATET -MATCHING = L ( i - m i ( ) )Y Y n1 i=1 (11)


Generally speaking implementing the matching method in the evaluation of the Diagshynostic Programme is feasible because in most cases it is possible to identify companies that are similar to the participants but have not participated in the programme

When establishing the criteria of proximity the variable ldquoproductrdquo should be the first to be considered The best option will be to match firms exporting the same specific product and belonging to the same market segment If this is not possible the product should at least be similar The second criterion of proximity is the variable ldquostage in the process of internashytionalisationrdquo Considering the differences that may exist in relation to the use of the programme and the potential difference in the companiesrsquo profitability it is advisable to match firms in the same stage of internationalisation The third criterion is the variable ldquosizerdquo the optimum solution will be to match firms of similar size Finally companies from the same comarca (county) should be selected

If the sample matches meet all the requirements listed above it will be possible to evalshyuate the results of the implementation of the Diagnostic Programme Moreover the estimashytion of the average causal effect may be completed for both the participating firms and the firms integrated into the treatment and control groups

53 Propensity score methods

The operational performance of the subclassification and matching procedures may be limited in cases where the number of covariates is high Rosenbaum (1995) draws the attenshytion to how the use of subclassification with a large number of covariates may force the deshysign of an excessively large number of subclasses

If the number of variables is high the procedure called propensity score (in this case propensity to participate) or the estimation of the probability of being a beneficiary of the programme can be implemented Rosenbaum amp Rubin (1983) define the propensity score as the chance to participate in the performance (probability of) conditioned by the values taken by a vector of covariates X (X = X1 X2 X3 Xn)

According to Hahn (1998) the calculation of the propensity score given some obshyserved characteristics plays a crucial role in controlling the bias when obtaining the estimashytor of the programmersquos impact

The propensity score method proceeds as if there was only one one-dimensional covariate Thus the evaluation can gain operability and avoid the manipulation of the large number of covariates that vector X may include This probability can be expressed in the folshylowing terms

s(X) = P (D = 1|X) (12)

being s (X) the probability of participating in the programme conditioned on X ie the proshypensity score


Rosenbaum amp Rubin (1983) presented the independence condition by using the proshypensity score as

(Y1 Y0) D|s(X) (13)

On this basis all observations showing a similar (or close) propensity score also have a similar distribution of the vector of covariates X This way the values of the response varishyables of the firms in the treatment and control groups having a similar (or close) propensity score can be compared Thus the contaminant effect of the covariates is isolated making it possible to calculate the programmersquos average effect

Being the propensity score s(X) a function of X the probability depends on the assumpshytion about its distribution function which must be estimated from the sample data

s(X) = P(D = 1|X) = F(fX) (14)

where f is the vector of parameters associated to the vector X of covariates

According to the hypothesis about the shape of this distribution function different bishynary response-choice models can be specified Among all the possible options for a non-linshyear distribution the more often used Logit and Probit models are to be mentioned The anashylytical expression of the distribution function of this model is

-z 2fXe f 1 s( )X = P D ( =1 ) = F ( ) =X fX = f e 2 -o lt z lt o

fX -o1+ e 2 logit probit

(15)

There is not a defined criterion of selection to choose one or the other model for the esshytimation of the propensity score Usually the choice is made for purely operational reasons Therefore both models are estimated and in view of the values obtained the one presenting lower values according to the Akaike Schwarz and Hannan-Quinn information criteria as well as the highest value in the likelihood function will be the most efficient and the one chosen

According to what has been stated above the estimation of the effect of the evaluated programme on the basis of the propensity score should follow a two-step procedure First the propensity score s(X) is estimated Second the average effect is calculated by applying either of the above-described techniques namely subclassification or matching to the estishymated values of the propensity score

Once the values of the four covariates considered are fixed the propensity score methshyodology allows comparing the firms of the treatment and control groups having a similar or close scoring Then the calculation of the programmersquos average effect will be undertaken by applying the matching procedure according to the stated criteria of proximity For this secshyond stage of the process matching is chosen rather than subclassification because it is a stricter and therefore more appropriate procedure of comparison


From what has been exposed it is possible to conclude that the propensity score-matchshying method is a suitable procedure to evaluate the average causal effect of the Diagnostic Programme The reason for choosing the propensity score-matching method is that it proves to be the most adequate tool when the dimensionality of the observable characteristics is high as it happens in this case If the number of characteristics is small (for instance one or two covariates) subclassification and matching will be both straightforward and adequate However when the dimensionality of X is high these two methods may prove defective In the event of a great number of covariates subclassification is likely to create cells of X with either no participants or no control observations Similarly when covariates are many it may be hard to find for each observation in the treatment or control group an observation in the other group having similar covariate values On the contrary the propensity score-matching method is especially useful under such circumstances (a great number of observable characshyteristics) because it provides a natural weighting scheme that yields unbiased estimates of the treatmentrsquos impact (Dehejia amp Wabba 2002) thus reducing the dimensionality problem of the subclassification and matching procedures

6 Lesson Learns

The benefits of internationalisation certainly explain the implementation of export proshymotion programmes financed by public funds but it is interesting to know if these programmes are effective The evaluations conducted since the 1970s have been questioned for several reasons The subjective nature of survey-based assessments and the attempts to quantitatively measure these programmes by using a single variable mdashthe overall export volshyume although considered in different waysmdash have been particularly criticised The diversity of factors affecting the success or failure of a programme makes it impossible to link the programmersquos results directly to a single variable so these factors must be somehow taken into account and controlled

The use of statistical causal inference for the evaluation of public policies allows reshysearchers to estimate the causal effect of a public programme by keeping under control other factors affecting this variable A model that applies this technique is the Rubin Causal Model (RCM) which allows comparing participants and non-participants in public programmes through a treatment indicator Once the indicator is specified it is necessary to determine the variable that will measure the effect of the evaluated policy In the case discussed in this work the export figure stands as a good response variable But there are certain issues that also need to be taken into consideration This variable must be defined according to the type of activity concerned because the margin attributable to each monetary unit can differ subshystantially Also the temporal aspect of the response variable must be reckoned since Intershynationalisation is a dynamic process that requires efforts and training Therefore a period of four or five years following the firmrsquos participation in the programme should be considered

After selecting the most adequate variable there will be potential responses associated to the firmrsquos status But in social experiments like the evaluation of export promotion programmes potential responses can be very heterogeneous This is why it becomes necesshy


sary to calculate the average effect of a programme in a way that allows controlling the facshytors responsible for this heterogeneity The validity of the estimated average effect may diminish if the firms included in the treatment group and the control group differ in characshyteristics other than those derived from the fact of their participation in the programme For this reason the different characteristics that can affect the results must be controlled through a number of covariates As long as these differentiating characteristics can be observed it will be possible to control them This is the basis of the selection on observables method

This work considers four covariates capable of affecting the outcome of the Diagnostic Programme The selection of these covariates depends on the available micro-data and the characteristics of the evaluated programme However covariates such as the stage of the process of internationalisation in which the company finds itself the size of the company the type of business activity it develops and its location are generally considered as adequate

In order to estimate the average effect of a programme with a selection on observables method it is possible to use procedures such as subclassification matching or those based on the previous estimation of the propensity score Subclassification is carried out by dividing the treatment and control groups into different subclasses and matching allows forming pairs of firms with similar observable characteristics But the effectiveness these two proceshydures may be limited when the number of covariates is high and this is exactly the case of export promotion programmes

When the dimensionality of the observable characteristics is high the propensity score-matching method is especially useful Thus the propensity score-matching can be conshysidered a suitable procedure to evaluate the average causal effect of the Diagnostic Programme and in general any export promotion programme because it provides a natural weighting scheme that yields unbiased estimates of the treatment impact reducing the dimensionality problem associated to the subclassification or matching methods

7 Conclusions

Literature on the economic evaluation of public programmes of internationalisation has been traditionally based on surveys that requested the recipientsrsquo evaluation or on quantitashytive studies particularly cost-benefit analyses Experts have valued the amount of informashytion obtained from surveys but they have also cast doubts on their reliability and have pointed out that survey-based evaluations may be unsatisfactory due to various reasons such as the firmsrsquo reluctance to criticise programmes that often have no cost for them

The economic evaluation of public programmes on business internationalisation is an interesting issue for policy makers and may benefit from the application of methods based on statistical causal inference to non-experimental contexts This approach has been successshyfully developed in many disciplines therefore its use in the evaluation of public programmes will surely contribute to the scientific literature in this field


For the case of the Diagnostic Programme this work proposes the export figure deshyfined as the net turnover derived from the supply of goods produced by Andalusian firms outside Spain as the response variable

Four covariates are also proposed the stage in the process of internationalisation in which the company finds itself the companyrsquos size the activity developed by it and more specifically the type of product that is or could be exported and finally the firmrsquos location Further access to appropriate micro-data has been taken into account before selecting these covariates

Due to the number of covariates in this case this work proposes the propensity score-matching method as the most suitable procedure to evaluate the average causal effect of the Diagnostic Programme through an unbiased estimator of ATE and ATET

Hopefully further researches will let us estimate the values of ATE and ATET from apshypropriate databases

Notes

1 A new plan has been designed for 2010-2013 (Consejeriacutea de Economiacutea Innovacioacuten y Ciencia 2010)

2 See Cox (1992) on the theoretical approach of causality and its use in randomised experiments Other authors such as Dawid (1979 2000) Holland (1986) Heckman (1990) and Pearl (2000) also discuss the meaning of causality in such an environment Finally in the specific case of training programmes we have referred to the seminal papers of Rubin (1974) and Heckman amp Hotz (1989)

3 Cameron amp Trivedi (2005) expose the advantages of POM as compared to other alternative models

4 The first references considered by Rubin were Neyman (1923 1935) and Fisher (1928 1935)

5 See Rubin (1974 1978) on the original references to this issue

6 According to Heckman (1976) sample selection bias problems may arise in practice for two reasons First there may be a self-selection of the individuals being analysed Second the researchersrsquo decisions may also have an influence on the sample selection

7 According to Angrist et al (1996) a variable is an instrumental variable for the causal effect of D on Y if its average effect on D is not zero if it satisfies both the exclusion restriction and the monotonicity assumption if it is randomly (or ignorantly) assigned and if the SUTVA (Stable Unit Treatment Value Assumption) holds

References

Albaum G (1983) ldquoEffectiveness of government export assistance for US smaller sized manufactushyrers Some further evidencerdquo International Marketing Review 1(1) 68-73

Andrews M Bradley S amp Upward R (1999) ldquoEstimating youth training wage differentials during and after trainingrdquo Oxford Economic Papers 51(3) 517-544

Angrist J D Imbens G W amp Rubin D B (1996) ldquoIdentification of causal effects using instrumenshytal variablesrdquo Journal of the American Statistical Association 91(434) 444-455


Angrist J D amp Lavy V (1999) ldquoUsing Maimonidesrsquo rule to estimate the effect of class size on schoshylastic achievementrdquo The Quarterly Journal of Economics 114(2) 533-575

Arellano A (2005) ldquoDo training programmes get the unemployed back to work A look at the Spanish experiencerdquo Working Paper 05-25 Economic Series 05 April Universidad Carlos III de Madrid

Arellano A (2010) ldquoDo training programmes get the unemployed back to work A look at the Spashynish experiencerdquo Revista de Economiacutea Aplicada 53(18) 39-65

Armah B K Jr amp Epperson J E (1997) ldquoExport demand for US orange juice impacts of US export promotion programmesrdquo Agribusiness 13 1-6

Austrade (2002) Knowing and Growing the Exporter Community Sydney Australian Trade Commisshysion

Bergemann A Fitzenberger B amp Speckesser S (2005) ldquoEvaluating the dynamic employment efshyfects of training programmes in East Germany using conditional difference-in-differencesrdquo Institushyte for the Study of Labor (IZA) working paper nordm 1848 November 2005

Blundell R Costa M Meghir C amp Van Reenen J (2004) ldquoEvaluating the employment impact of a mandatory job search programmerdquo Journal of the European Economic Association 2(4) 569-606

Bonnall L Fougegravere D amp Seacuterandon A (1997) ldquoEvaluating the impact of French employment policies on individual labour market historiesrdquo Review of Economic Studies 64(4) 683-713

Brewer P (2009) ldquoAustraliarsquos export promotion programme is it effectiverdquo Australia Journal of Management 34(1) 125-142

Cadwell C (1992) ldquoState Export Promotion and Small Businessrdquo Research Summary Number 125 Small business administration Washington DC United States

Cameron A C amp Trivedi P K (2005) Microeconometrics Methods and applications New York Cambridge University Press

Camino D (1991) ldquoExport promotion policies in Spain and other EEC countries systems and perforshymancerdquo in Seringhaus F H R amp Rosson P (Eds) Export Development and Promotion The Role of Public Organizations Kluwer Academic Publishers Norwall MA 119-44

Cansino J M (2003) ldquoTres Soluciones para la Provisioacuten Puacuteblica Ineficienterdquo Revista Espantildeola de Control Externo vol V nordm 14 175-207

Cansino J M (2006) ldquoPropuestas metodoloacutegicas para la evaluacioacuten de programas puacuteblicos de formashycioacuten Una revisioacuten criacuteticardquo La calidad del Gobierno Evaluacioacuten econoacutemica de las poliacuteticas puacuteblishycas Madrid Delta Publicaciones Universitarias pp 117-155

Cansino J M amp Saacutenchez Braza A (2008) ldquoAverage effect of training programmes on the time neeshyded to find a job The case of the training programmes schools in the south of Spain (Seville 1997-1999)rdquo Documentos de Trabajo de Funcas (text available at wwwfuncases)

Cansino J M amp Saacutenchez Braza A (2009) ldquoEvaluacioacuten del programa de Escuelas Taller y Casas de Oficios a partir de su efecto sobre el tiempo de buacutesqueda del primer empleo El caso de Sevillardquo Estudios de Economiacutea Aplicada 27 (1) 277 (22 pages open section)

Cansino J M amp Saacutenchez Braza A (2011) ldquoEffectiveness of public training programmes reducing the time needed to find a jobrdquo Estudios de Economiacutea Aplicada 29 (1) 391 (26 pages open section)


Card D amp Krueger A B (1994) ldquoMinimum wages and employment A case study of the fast-food industry in New Jersey and Pennsylvaniardquo American Economic Association 84 (4) 772-793

Card D amp Sullivan D (1988) ldquoMeasuring the effects of subsidized training programmes on moveshyments in and out of employmentrdquo Econometrica 56(3) 497-530

Cavusgil S T amp Naor J (1987) ldquoFirm and management characteristics as discriminators of export marketing activityrdquo Journal of Business Research 15(3) 221-35

Consejeriacutea de Economiacutea Innovacioacuten y Ciencia (2010) IV Plan de Internacionalizacioacuten de la Econoshymiacutea Andaluza Sevilla Junta de Andaluciacutea httpwwwjuntadeandaluciaesexportdrupalPlan_ Estratxgico_Internacionalizacixn_2010-2013_-version_final_IMPRENTAPDF (last accessed 10 February 2011)

Consejeriacutea de Turismo Comercio y Deporte (2006) Plan estrateacutegico para la internacionalizacioacuten de la empresa andaluza 2007-2010 Sevilla Junta de Andaluciacutea httpwwwextendaeswebexportsishytesextendaextendaplan-internacionaliacionPlan_Internacionalizaci_n_2007_2010pdf (last acshycessed 10 February 2011)

Coughlin C C amp Cartwright P A (1987) ldquoAn examination of state foreign export promotion and manufacturing exportsrdquo Journal of Regional Science 27(3) 439-49

Crick D amp Czinkota M R (1995) ldquoExport assistance another look at whether we are supporting the best programmesrdquo International Marketing Review 12(3) 61-72

Dawid A P (1979) ldquoConditional independence in statistical theoryrdquo Journal of the Royal Statistical Society Series B (Statistics Methodological) 41(1) 1-31

Dawid A P (2000) ldquoCausal inference without counterfactualsrdquo Journal of the American Statistical Association 95(2) 407-448

Dehejia R H and Wahba S (2002) ldquoPropensity score-matching methods for nonexperimental causal studiesrdquo The Review of Economics and Statistics 84(1) 151-161

Fisher R A (1928) ldquoThe general sampling distribution of the multiple correlation coefficientrdquo Proshyceedings of the Royal Society of London Series A Containing Papers of a Mathematical and Physishycal Character 121 654-673

Fisher R A (1935) The design of experiments Edinburgh Oliver and Boyd

Francis J amp Collins-Dodd C (2004) ldquoImpact of export promotion programmes on firm competenshycies strategies and performancerdquo International Marketing Review 21(45) 474-495

Genccediltuumlrk E F amp Kotabe M (2001) ldquoThe effect of export assistance programme usage on export pershyformance a contingency explanationrdquo Journal of International Marketing 9(2) 51-72

Gertler P (2004) ldquoDo conditional cash transfers improve child health Evidence from PROGRESArsquos control randomized experimentrdquo American Economic Review Papers and Proceedings 94(2) 336-341

Gillespie K amp Riddle L (2004) ldquoExport promotion organization emergence and development A call to researchrdquo International Marketing Review 21(45) 462-73

Hahn J (1998) ldquoOn the role of the propensity score in efficient semiparametric estimation of average treatment effectsrdquo Econometrica 66(2) 315-331

Heckman J J (1979) ldquoSample selection bias as a specification errorrdquo Econometrica 47(1) 153-61


Heckman J J (1990) ldquoVarieties of Selection Biasrdquo American Economic Review Papers and Proceeshydings of the Hundred and Second Annual Meeting of the American Economic Association 80(2) 313-338

Heckman J J amp Hotz V J (1989) ldquoChoosing Among Alternative Nonexperimental Methods for Estimating the Impact of Social Programmes The Case of Manpower Trainingrdquo Journal of the American Statistical Association 84(408) 862-874

Holland P W (1986) ldquoStatistics and causal inference (with discussion)rdquo Journal of de American Stashytistical Association 81 945-970

Imbens G W (2004) ldquoNonparametric estimation of average treatment effects under exogeneity A reshyviewrdquo Review of Economics and Statistics 8(1) 4-29

Imbens G W amp Angrist J D (1994) ldquoIdentification and estimation of local average treatment efshyfectsrdquo Econometrica 62(2) 467-475

Knight G A amp Liesch P W (2002) ldquoInformation internalisation in internationalising the firmrdquo Journal of Business Research 55 981-995

Lalonde R J (1986) ldquoEvaluating the econometric evaluation of training programmes with experishymental datardquo American Economic Review 76(4) 604-620

Manski C F amp Garfinkel I (1992) Evaluating welfare and training programmes Cambridge Masshysachusetts Harvard University Press

Marandu E E (1995) ldquoImpact of export promotion on export performance a Tanzanian studyrdquo Jourshynal of Global Marketing 9(12) 9-39

Mato F J (2002) La formacioacuten para el empleo una evaluacioacuten cuasi-experimental Madrid Civitas

Mato F J (2010) ldquoLa formacioacuten continua en Espantildea desde una perspectiva comparada Balance y propuestas de mejorardquo Papeles de Economiacutea Espantildeola 124 266-280

Mato F J amp Cueto B (2008) ldquoEfectos de las poliacuteticas de formacioacuten a desempleadosrdquo Revista de Economiacutea Aplicada 46(16) 61-83

Mato F J amp Cueto B (2009) ldquoA nonexperimental evaluation of training programmes regional evishydence for Spainrdquo Annals of Regional Science 43(2) 415-433

Neyman J (1923) ldquoOn the application of probability theory to agricultural experiments Essay on principlesrdquo Re-edited in Statistical Science (with discussion) 1990 5(4) 465-472

Neyman J (1935) ldquoStatistical problems in agricultural experimentationrdquo Supplement to the Journal of the Royal Statistical Society 2 107-180

Park N Power B Riddell W C amp Wong G (1996) ldquoAn Assessment of the Impact of Governshyment-Sponsored Trainingrdquo Canadian Journal of Economics 29 (Special Issue Part I) S93-S98

Pearl J (2000) Causality Models reasoning and inference Cambridge Cambridge University Press

Pointon T (1978) ldquoMeasuring the gains from government export promotionrdquo Journal of European Marketing 12(6) 451-462

Richards T J Van Ispelen P amp Kagan A (1997) ldquoA two-stage analysis of the effectiveness of proshymotion programmes for US applesrdquo American Journal of Agricultural Economics 79(3) 825-37

Rosenbaum P R (1995) Observational studies New York Springer-Verlag


Rosenbaum P R amp Rubin D B (1983) ldquoThe central role of the propensity score in observational stushydies for causal effectsrdquo Biometrica 70 41-55

Rubin D B (1974) ldquoEstimating causal effects of treatments in randomized and nonrandomized stushydiesrdquo Journal of Educational Psychology 66(5) 688-701

Rubin D B (1978) ldquoBayesian inference for causal effects The role of randomizationrdquo Annals of Stashytistics 6(1) 34-58

Saacutenchez Braza A (2006) ldquoLa evaluacioacuten aplicada a programas puacuteblicos de formacioacuten La utilizacioacuten de meacutetodos observacionales el estimador de diferencias en diferenciasrdquo La calidad del Gobierno Evaluacioacuten econoacutemica de las poliacuteticas puacuteblicas Madrid Delta Publicaciones Universitarias pp 339-367

Seringhaus F H R (1986) ldquoThe impact of government export marketing assistancerdquo International Marketing Review 3(2) 55-60

Seringhaus F H R (1990) ldquoProgramme impact evaluation Application to export promotionsrdquo Evashyluation and Programme Planning 13 251-265

Seringhaus F H R amp Rosson P J (1990) Government Export Promotion A Global Perspective London Routledge

Stock J amp Watson M (2003) Introduction to Econometrics 1st ed Boston Addison Wesley

Wilkinson T amp Brouthers L E (2006) ldquoTrade promotion and SME export performancerdquo Internatioshynal Business Review 15 233-252


proceso de internacionalizacioacuten tamantildeo producto y localizacioacuten) Aplicados estos procedimientos se concluye que los procedimientos de ldquomatchingrdquo y ldquopropensity scorerdquo se pueden utilizar para evaluar el Programa de Diagnoacutestico Ademaacutes se hace referencia a los datos que necesita la organizacioacuten que lo desarrolla para llevar a cabo la evaluacioacuten

Palabras clave poliacuteticas puacuteblicas internacionalizacioacuten inferencia causal estadiacutestica Programa de Diagnoacutestico

Clasificacioacuten JEL H43 H59 O24

1 Introduction

Internationalisation can be defined as a gradual and evolutionary process in which firms progressively increase their involvement in international business It is a beneficial process for both the company and the national or local economy Internationalisation may generate econoshymies of scale in local firms and promote the transfer of technology and management knowlshyedge thus boosting growth and employment (Austrade 2002) These benefits explain the imshyplementation of export promotion activities and programmes financed by public funds they will justify the costs associated with this public expenditure It is therefore interesting to know the extent to which those export promotion policies (EPPs) undertaken by governments are profitable or not ie their development needs to be assessed in order to see whether their applishycation is effective or can be improved through a new programme design Cansino (2003) conshysiders the background of the programme as a conflict between the effective application of the public funds and the implementation of policies determined by the interests of the performing agent Therefore it is convenient to carry out evaluations of these policies because on the whole the research developed during the 1980s the 1990s and into the new millennium reshymains somewhat inconclusive regarding the effectiveness of EPPs (Brewer 2009)

As pointed out by Seringhaus (1990) the evaluation of the impact of export promotion programmes on exports became a concern in the late 1970s Since then the literature on the economic evaluation of public programmes of internationalisation has been based on surveys which reflect the recipientsrsquo assessment and quantitative studies particularly cost-benefit analyses

Among the former ie those studies based on surveys the work of Albaum (1983) can be highlighted It underlined the fact that the programmes were generally unfavourably valshyued and revealed a lack of understanding between the government and the small businesses about the role and value of such programmes Seringhaus (1990) valued the information gathered through the surveys but cast doubts on its reliability Wilkinson amp Brouthers (2006) used surveys conducted with a four-item scale to measure the level of satisfaction among respondents The use of surveys to assess export promotion programmes has been questioned by many authors As noted by Brewer (2009) these surveys may be unsatisfacshytory due to various reasons such as the firmsrsquo reluctance to criticise a programme that in many cases had no cost for them the diversity of the respondentsrsquo opinions as evidenced by Crick amp Czinkota (1995) or the pressure from the programme providers (Seringhaus amp Rosson 1990) Thus as stated by Francis amp Collins-Dodd (2004) the use of these surveys can be criticised as lacking in objectivity























0 20 40 60 80 100 120

2010

2009

2008

2007

2006

2005

2004

2003

2002 15

33

42

49

100

89

99

112

54

2002 2003 2004 2005 2006 2007 2008 2009 2010

Companies 15 33 42 49 100 89 99 112 54




















= E Y 1) - ( 0D = E Y D = 0) = E Y( D = 1) - E Y ( D = 0)( 1





(Y1 Y0) D (4)





(Y1 Y0) D | X (5)









Table 1

Covariates











Area







42 Size







43 Product







44 Location















j j jn + n = n1 0 (6)

1 + n2 + n kn 3 + + n = n (7)







k jj j n


j=1


k jj j n


n (9)j=1 1





52 Matching







1 n1











s(X) = P (D = 1|X) (12)




(Y1 Y0) D|s(X) (13)



s(X) = P(D = 1|X) = F(fX) (14)





(15)






6 Lesson Learns









7 Conclusions








Notes








References






























































































0 20 40 60 80 100 120

2010

2009

2008

2007

2006

2005

2004

2003

2002 15

33

42

49

100

89

99

112

54

2002 2003 2004 2005 2006 2007 2008 2009 2010

Companies 15 33 42 49 100 89 99 112 54




















= E Y 1) - ( 0D = E Y D = 0) = E Y( D = 1) - E Y ( D = 0)( 1





(Y1 Y0) D (4)





(Y1 Y0) D | X (5)









Table 1

Covariates











Area







42 Size







43 Product







44 Location















j j jn + n = n1 0 (6)

1 + n2 + n kn 3 + + n = n (7)







k jj j n


j=1


k jj j n


n (9)j=1 1





52 Matching







1 n1











s(X) = P (D = 1|X) (12)




(Y1 Y0) D|s(X) (13)



s(X) = P(D = 1|X) = F(fX) (14)





(15)






6 Lesson Learns









7 Conclusions








Notes








References























































































0 20 40 60 80 100 120

2010

2009

2008

2007

2006

2005

2004

2003

2002 15

33

42

49

100

89

99

112

54

2002 2003 2004 2005 2006 2007 2008 2009 2010

Companies 15 33 42 49 100 89 99 112 54




















= E Y 1) - ( 0D = E Y D = 0) = E Y( D = 1) - E Y ( D = 0)( 1





(Y1 Y0) D (4)





(Y1 Y0) D | X (5)









Table 1

Covariates











Area







42 Size







43 Product







44 Location















j j jn + n = n1 0 (6)

1 + n2 + n kn 3 + + n = n (7)







k jj j n


j=1


k jj j n


n (9)j=1 1





52 Matching







1 n1











s(X) = P (D = 1|X) (12)




(Y1 Y0) D|s(X) (13)



s(X) = P(D = 1|X) = F(fX) (14)





(15)






6 Lesson Learns









7 Conclusions








Notes








References














































































0 20 40 60 80 100 120

2010

2009

2008

2007

2006

2005

2004

2003

2002 15

33

42

49

100

89

99

112

54

2002 2003 2004 2005 2006 2007 2008 2009 2010

Companies 15 33 42 49 100 89 99 112 54




















= E Y 1) - ( 0D = E Y D = 0) = E Y( D = 1) - E Y ( D = 0)( 1





(Y1 Y0) D (4)





(Y1 Y0) D | X (5)









Table 1

Covariates











Area







42 Size







43 Product







44 Location















j j jn + n = n1 0 (6)

1 + n2 + n kn 3 + + n = n (7)







k jj j n


j=1


k jj j n


n (9)j=1 1





52 Matching







1 n1











s(X) = P (D = 1|X) (12)




(Y1 Y0) D|s(X) (13)



s(X) = P(D = 1|X) = F(fX) (14)





(15)






6 Lesson Learns









7 Conclusions








Notes








References


























































































= E Y 1) - ( 0D = E Y D = 0) = E Y( D = 1) - E Y ( D = 0)( 1





(Y1 Y0) D (4)





(Y1 Y0) D | X (5)









Table 1

Covariates











Area







42 Size







43 Product







44 Location















j j jn + n = n1 0 (6)

1 + n2 + n kn 3 + + n = n (7)







k jj j n


j=1


k jj j n


n (9)j=1 1





52 Matching







1 n1











s(X) = P (D = 1|X) (12)




(Y1 Y0) D|s(X) (13)



s(X) = P(D = 1|X) = F(fX) (14)





(15)






6 Lesson Learns









7 Conclusions








Notes








References

















































































= E Y 1) - ( 0D = E Y D = 0) = E Y( D = 1) - E Y ( D = 0)( 1





(Y1 Y0) D (4)





(Y1 Y0) D | X (5)









Table 1

Covariates











Area







42 Size







43 Product







44 Location















j j jn + n = n1 0 (6)

1 + n2 + n kn 3 + + n = n (7)







k jj j n


j=1


k jj j n


n (9)j=1 1





52 Matching







1 n1











s(X) = P (D = 1|X) (12)




(Y1 Y0) D|s(X) (13)



s(X) = P(D = 1|X) = F(fX) (14)





(15)






6 Lesson Learns









7 Conclusions








Notes








References









































































(Y1 Y0) D (4)





(Y1 Y0) D | X (5)









Table 1

Covariates











Area







42 Size







43 Product







44 Location















j j jn + n = n1 0 (6)

1 + n2 + n kn 3 + + n = n (7)







k jj j n


j=1


k jj j n


n (9)j=1 1





52 Matching







1 n1











s(X) = P (D = 1|X) (12)




(Y1 Y0) D|s(X) (13)



s(X) = P(D = 1|X) = F(fX) (14)





(15)






6 Lesson Learns









7 Conclusions








Notes








References















































































Table 1

Covariates











Area







42 Size







43 Product







44 Location















j j jn + n = n1 0 (6)

1 + n2 + n kn 3 + + n = n (7)







k jj j n


j=1


k jj j n


n (9)j=1 1





52 Matching







1 n1











s(X) = P (D = 1|X) (12)




(Y1 Y0) D|s(X) (13)



s(X) = P(D = 1|X) = F(fX) (14)





(15)






6 Lesson Learns









7 Conclusions








Notes








References









































































Table 1

Covariates











Area







42 Size







43 Product







44 Location















j j jn + n = n1 0 (6)

1 + n2 + n kn 3 + + n = n (7)







k jj j n


j=1


k jj j n


n (9)j=1 1





52 Matching







1 n1











s(X) = P (D = 1|X) (12)




(Y1 Y0) D|s(X) (13)



s(X) = P(D = 1|X) = F(fX) (14)





(15)






6 Lesson Learns









7 Conclusions








Notes








References









































































42 Size







43 Product







44 Location















j j jn + n = n1 0 (6)

1 + n2 + n kn 3 + + n = n (7)







k jj j n


j=1


k jj j n


n (9)j=1 1





52 Matching







1 n1











s(X) = P (D = 1|X) (12)




(Y1 Y0) D|s(X) (13)



s(X) = P(D = 1|X) = F(fX) (14)





(15)






6 Lesson Learns









7 Conclusions








Notes








References












































































44 Location















j j jn + n = n1 0 (6)

1 + n2 + n kn 3 + + n = n (7)







k jj j n


j=1


k jj j n


n (9)j=1 1





52 Matching







1 n1











s(X) = P (D = 1|X) (12)




(Y1 Y0) D|s(X) (13)



s(X) = P(D = 1|X) = F(fX) (14)





(15)






6 Lesson Learns









7 Conclusions








Notes








References















































































j j jn + n = n1 0 (6)

1 + n2 + n kn 3 + + n = n (7)







k jj j n


j=1


k jj j n


n (9)j=1 1





52 Matching







1 n1











s(X) = P (D = 1|X) (12)




(Y1 Y0) D|s(X) (13)



s(X) = P(D = 1|X) = F(fX) (14)





(15)






6 Lesson Learns









7 Conclusions








Notes








References











































































52 Matching







1 n1











s(X) = P (D = 1|X) (12)




(Y1 Y0) D|s(X) (13)



s(X) = P(D = 1|X) = F(fX) (14)





(15)






6 Lesson Learns









7 Conclusions








Notes








References

















































































s(X) = P (D = 1|X) (12)




(Y1 Y0) D|s(X) (13)



s(X) = P(D = 1|X) = F(fX) (14)





(15)






6 Lesson Learns









7 Conclusions








Notes








References










































































(Y1 Y0) D|s(X) (13)



s(X) = P(D = 1|X) = F(fX) (14)





(15)






6 Lesson Learns









7 Conclusions








Notes








References










































































6 Lesson Learns









7 Conclusions








Notes








References













































































7 Conclusions








Notes








References













































































Notes








References







































































































































































































































Documents

A new approach to the economic evaluation of public programmes