11
Interfaces with Other Disciplines Productivity change using growth accounting and frontier-based approaches – Evidence from a Monte Carlo analysis Dimitris Giraleas , Ali Emrouznejad, Emmanuel Thanassoulis Operations and Information Management, Aston Business School, Aston University, Birmingham, UK article info Article history: Received 29 July 2011 Accepted 5 May 2012 Available online 26 May 2012 Keywords: Data envelopment analysis Productivity and competitiveness Monte Carlo analysis Stochastic frontier analysis Growth accounting abstract This study presents some quantitative evidence from a number of simulation experiments on the accu- racy of the productivity growth estimates derived from growth accounting (GA) and frontier-based meth- ods (namely data envelopment analysis-, corrected ordinary least squares-, and stochastic frontier analysis-based malmquist indices) under various conditions. These include the presence of technical inef- ficiency, measurement error, misspecification of the production function (for the GA and parametric approaches) and increased input and price volatility from one period to the next. The study finds that the frontier-based methods usually outperform GA, but the overall performance varies by experiment. Parametric approaches generally perform best when there is no functional form misspecification, but their accuracy greatly diminishes otherwise. The results also show that the deterministic approaches per- form adequately even under conditions of (modest) measurement error and when measurement error becomes larger, the accuracy of all approaches (including stochastic approaches) deteriorates rapidly, to the point that their estimates could be considered unreliable for policy purposes. Ó 2012 Elsevier B.V. All rights reserved. 1. Introduction The study of productivity is a very important topic. The UK Office of National Statistics (ONS, 2007) states that: ‘Statistics relating to productivity are vital to understanding the economy and how it changes’. It also states that: ‘it is crucial that both experts and the general public can depend on the accuracy and relevance of ONS productivity measures’. The Organisation for Economic and Social Development (OECD) also states that one of its major aims is to improve the measurement of productivity growth. 1 The pursuit of productivity growth and productivity conver- gence is also one of the central goals of the European Union (EU). Probably the main instruments to achieve those goals are the so- called Structural funds, which are distributed based on Gross Domestic Product (GDP) per capital differentials between the var- ious EU regions. Changes in GDP per capital are also used as simple measures of productivity growth and although probably sufficient for setting policy at this stage, a more refined productivity indica- tor is required to evaluate the effects of the funds and the degree of convergence. The issue of converge is critical, since the underlying aim of the Structural funds is to increase GDP by providing the rel- atively poorer regions with the tools to achieve the productivity/ efficiency potential of the more advanced regions, rather than rais- ing GDP simply through factor accumulation (i.e. increasing input quantities). More complex approaches that seek to estimate Total Factor Productivity growth (TFP) can provide the required granularity of information, by examining the sources of GDP growth that are not due to such factor accumulation. The EU seems to support the development and use of such approaches, given the emphasis the Directorate General for Economic and Financial Affairs (DG-ECFIN) has placed on the EU KLEMS project (EU KLEMS, 2008), an EU-wide research project that aims to provide estimates of aggregate TFP growth in the EU member states together with the data necessary for the estimation. The DG-ECFIN (Koszerek et al., 2007) states that the productivity indicators provided by EU KLEMS are ‘essential for understanding recent EU productivity trends’, ‘fundamental in assess- ing progress with the Lisbon Strategy’, ‘can complement the ’’Structural Indicators’’ Programme’, and ‘provide an additional data source for refining the potential growth rate estimates used in the EU’s budgetary surveillance process’. Productivity growth in the EU KLEMS database is estimated based on growth accounting (GA). GA is an index number-based methodology for measuring productivity growth which is based in the early work of Tinbergen (1942) and independently, Solow (1957) and is the method of choice when measuring aggregate (i.e. country- or sector-wide) productivity growth for most interested agents, namely statistical agencies (national and inter- national), central banks and government bodies (see for example 0377-2217/$ - see front matter Ó 2012 Elsevier B.V. All rights reserved. http://dx.doi.org/10.1016/j.ejor.2012.05.015 Corresponding author. E-mail address: [email protected] (D. Giraleas). 1 http://www.oecd.org/about/0,3347,en_2825_30453906_1_1_1_1_1,00.html, accessed 14 January 2011. European Journal of Operational Research 222 (2012) 673–683 Contents lists available at SciVerse ScienceDirect European Journal of Operational Research journal homepage: www.elsevier.com/locate/ejor

Productivity Change Using Growth Accounting and Frontier-based Approaches – Evidence From a Monte Carlo Analysis

Embed Size (px)

DESCRIPTION

produtividade

Citation preview

  • tilo

    naingh

    Stochastic frontier analysisGrowth accounting

    quawthpmeindior,inp

    their accuracy greatly diminishes otherwise. The results also show that the deterministic approaches per-

    importes thaing thcial thccuracy

    tor is required to evaluate the effects of the funds and the degree ofconvergence. The issue of converge is critical, since the underlyingaim of the Structural funds is to increase GDP by providing the rel-atively poorer regions with the tools to achieve the productivity/

    surveillance process.Productivity growth in the EU KLEMS database is estimated

    based on growth accounting (GA). GA is an index number-basedmethodology for measuring productivity growth which is basedin the early work of Tinbergen (1942) and independently, Solow(1957) and is the method of choice when measuring aggregate(i.e. country- or sector-wide) productivity growth for mostinterested agents, namely statistical agencies (national and inter-national), central banks and government bodies (see for example

    Corresponding author.E-mail address: [email protected] (D. Giraleas).

    1 http://www.oecd.org/about/0,3347,en_2825_30453906_1_1_1_1_1,00.html,

    European Journal of Operational Research 222 (2012) 673683

    Contents lists available at

    European Journal of O

    w.eaccessed 14 January 2011.productivity measures. The Organisation for Economic and SocialDevelopment (OECD) also states that one of its major aims is toimprove the measurement of productivity growth.1

    The pursuit of productivity growth and productivity conver-gence is also one of the central goals of the European Union (EU).Probably the main instruments to achieve those goals are the so-called Structural funds, which are distributed based on GrossDomestic Product (GDP) per capital differentials between the var-ious EU regions. Changes in GDP per capital are also used as simplemeasures of productivity growth and although probably sufcientfor setting policy at this stage, a more rened productivity indica-

    development and use of such approaches, given the emphasis theDirectorate General for Economic and Financial Affairs (DG-ECFIN)has placed on the EU KLEMS project (EU KLEMS, 2008), an EU-wideresearch project that aims to provide estimates of aggregate TFPgrowth in the EU member states together with the data necessaryfor the estimation. The DG-ECFIN (Koszerek et al., 2007) states thatthe productivity indicators provided by EU KLEMS are essential forunderstanding recent EU productivity trends, fundamental in assess-ing progress with the Lisbon Strategy, can complement the StructuralIndicators Programme, and provide an additional data source forrening the potential growth rate estimates used in the EUs budgetary1. Introduction

    The study of productivity is a veryof National Statistics (ONS, 2007) staproductivity are vital to understandchanges. It also states that: it is crugeneral public can depend on the a0377-2217/$ - see front matter 2012 Elsevier B.V. Ahttp://dx.doi.org/10.1016/j.ejor.2012.05.015form adequately even under conditions of (modest) measurement error and when measurement errorbecomes larger, the accuracy of all approaches (including stochastic approaches) deteriorates rapidly,to the point that their estimates could be considered unreliable for policy purposes.

    2012 Elsevier B.V. All rights reserved.

    tant topic. The UKOfcet: Statistics relating toe economy and how itat both experts and theand relevance of ONS

    efciency potential of the more advanced regions, rather than rais-ing GDP simply through factor accumulation (i.e. increasing inputquantities).

    More complex approaches that seek to estimate Total FactorProductivity growth (TFP) can provide the required granularity ofinformation, by examining the sources of GDP growth that are notdue to such factor accumulation. The EU seems to support theProductivity and competitivenessMonte Carlo analysis

    the frontier-based methods usually outperform GA, but the overall performance varies by experiment.Parametric approaches generally perform best when there is no functional form misspecication, butInterfaces with Other Disciplines

    Productivity change using growth accounapproaches Evidence from a Monte Car

    Dimitris Giraleas , Ali Emrouznejad, Emmanuel ThaOperations and Information Management, Aston Business School, Aston University, Birm

    a r t i c l e i n f o

    Article history:Received 29 July 2011Accepted 5 May 2012Available online 26 May 2012

    Keywords:Data envelopment analysis

    a b s t r a c t

    This study presents someracy of the productivity groods (namely data enveloanalysis-based malmquistciency, measurement errapproaches) and increased

    journal homepage: wwll rights reserved.ng and frontier-basedanalysis

    ssoulisam, UK

    ntitative evidence from a number of simulation experiments on the accu-estimates derived from growth accounting (GA) and frontier-based meth-nt analysis-, corrected ordinary least squares-, and stochastic frontierces) under various conditions. These include the presence of technical inef-misspecication of the production function (for the GA and parametricut and price volatility from one period to the next. The study nds that

    SciVerse ScienceDirect

    perational Research

    lsevier .com/locate /e jor

  • 674 D. Giraleas et al. / European Journal of Operthe US Bureau of Labour Statistics technical note on multifactorproductivity2 and ONS, 2007). A major factor in the widespreadadoption of GA is the fact that estimates can be (relatively) easilyproduced using country- or sector-specic National Accounts data,without recourse to information from outside the country or the sec-tor examined; on the other hand, GA requires the adoption of a num-ber of simplistic (potentially unrealistic) assumptions, most notablythose relying on the existence of perfect competition, which couldlead to unreliable estimates.

    Given the stated need for accurate productivity growth esti-mates, the rst aim of this study is to assess the impact on theaccuracy of the GA estimates when some of the assumptions cen-tral to the notion of perfect competition are violated. This isachieved by undertaking a number of simulation experiments,which utilise randomly generated data for which the parametersof interest (most importantly productivity change) are known apriori; when GA (or any other productivity change measurementapproach) is applied to the same dataset, a measure of the overallaccuracy of the approach can be devised by comparing the esti-mate of productivity change to its true value.

    Frontier-based methods offer an attractive alternative for themeasurement of aggregate productivity change. Unlike the moretraditional GA methods, they allow for the production to occur in-side the frontier, thereby explicitly allowing for inefciency in theproduction process and relaxing the stringent assumptions re-quired when using growth accounting methods. In addition, fron-tier-based methods also allow for the decomposition ofproductivity growth, which could be of great interest to the usersof productivity change estimates.

    There are a number of applications of frontier based methodsfor the measurement of aggregate productivity growth in the aca-demic literature. Fre et al. (1994) was one of the rst studies thatutilised Data Envelopment Analysis (DEA), the more widely-usednon-parametric frontier based approach, to construct Malmquistindices of productivity growth; the approach has since beenadopted in numerous other studies (for a comprehensive list ofapplications of DEA-based Malmquist indices see Fried et al.,2008 and Del Gatto et al., 2008). Kumbhakar and Lovell (2000)introduced another way to construct a Malmquist index of produc-tivity growth that relies on parametric frontier models, such asCorrected Ordinary Least Squares (COLS) and Stochastic FrontierAnalysis (SFA); such models have also been widely used in the lit-erature (see Sharma et al., 2007, for a list of sample studies).

    However, despite the adoption of such frontier-based methodsin the academic literature and the theoretical advantages offeredby frontier-based methods compared to the more traditional GAapproach, there has been limited research on quantifying whetherthese advantages translate into improved accuracy of the resultingproductivity change estimates and under which conditions onefrontier-based approach is more accurate than another. As such,the second aim of this study is to employ the aforementioned sim-ulation experiments to provide quantitative evidence on the accu-racy of the more widely adopted frontier-based approaches,namely DEA-, COLS- and SFA-based Malmquist indices, under anumber of conditions that violate the assumptions made underperfect competition.

    In more detail, this research aims to examine the accuracy ofboth GA and frontier-based productivity change estimates:

    when technical inefciency, in various degrees of severity, ispresent,

    when inputs and prices are volatile from one period to the next, when the production function is misspecied, and nally2 http://www.bls.gov/mfp/mprtech.pdf, accessed 14 January 2011. when the factors of production are measured inaccurately(again in various degrees of severity).

    2. Methodology of the current research

    2.1. Productivity measurement approaches considered

    Each simulation experiment examines the performance of thefollowing approaches:

    GA, DEA-based circular Malmquist indices, COLS-based Malmquist indices, and SFA-based Malmquist indices, (only when measurement noiseis included in the experiment).

    All frontier-based approaches examined in this analysis rely onthe notion of what has come to be known as the Malmquist pro-ductivity index (Diewert, 1992), which has been used extensivelyin both the parametric (see for example Kumbhakar and Lovell,2000) and the non-parametric (see for example Thanassoulis,2001) settings. Furthermore, the productivity index produced byGA can be considered as a special case of the Malmquist productiv-ity index (see OECD, 2001).

    Given the nature of the approaches considered, all of the analyseswe perform focus on the production side of the economic process.

    2.1.1. Growth accountingGrowth Accounting (GA) is an index number-based approach

    that relies on the neo-classical production framework, and seeksto estimate the rate of productivity change residually, i.e. by exam-ining how much of an observed rate of change of a units outputcan be explained by the rate of change of the combined inputs usedin the production process. There are many modications that couldbe applied to the more general GA setting (Balk, 2008; Del Gattoet al., 2008); however, most applications still utilise traditionalgrowth accounting methods, as described in OECD (2001) (seefor example OMahony and Timmer, 2009).

    GA postulates the existence of a production technology that canbe represented parametrically by a production function relatinggross output (Y), to primary inputs labour (L) and capital services(K) as well as intermediate inputs such as material, services or en-ergy (M).

    Y FK; L;M 1If gross output is measured net of intermediate inputs, i.e. using aGross Value Added (GVA) measure, (1) becomes:

    YGVA FK; L 2GA assumes that productivity changes (TFP) are Hicks-neutral type,i.e. they correspond to an outward shift of the production function,such that:

    YGVA FK; L TFP 3A number of assumptions are required to parameterise (3), namelythat:

    the production function is Cobb-Douglas and exhibits constantreturns to scale;

    each assessed unit minimises the costs of inputs for any desiredlevel of output and can adjust the level of primary inputs that itutilises at any moment and without additional costs;

    input markets are perfectly competitive and all production hap-

    ational Research 222 (2012) 673683pens on the frontier; all relevant inputs and outputs are taken into account and mea-sured without error.

  • OperFor a more detailed discussion on the assumptions required forGA, see Annex 3 of the OECD manual (OECD, 2001).

    If the above assumptions hold, once the Cobb-Douglas produc-tion function is differentiated with respect to time, the rate ofchange in output is equal to the sum of the weighted average ofthe change in inputs and the change in productivity. The inputweights are the output elasticities of each factor of production; un-der perfect competition conditions, the marginal revenue gener-ated by each factor is equal to its price and, as such, the outputelasticity of each factor is equal to its share in the total value ofproduction.

    Therefore, productivity change is estimated by:

    d ln TFPGAidt

    d ln yidt

    SLid ln Lidt

    SKid lnKidt

    4

    where SLi is the average share of labour in periods t and t 1, SLi isthe average share of capital in t and t-1 given by:

    SLi wLitLitpitYit

    wLit1Lit1

    pit1Yit1

    =2 5

    SKi wK;GAit KitpitYit

    wK;GAit1 Kit1

    pit1Yit1

    !=2 6

    It should be noted that the price of capital is not observable; there-fore analyses that utilise GA usually rely on an imputed price of cap-ital. This is discussed in more detail in Section 2.2 (Price data). Theuse of arithmetic averages for the input shares was adopted for con-sistency with the EU KLEMS methodology. An alternative optionwould be to use geometric averages; this has been examined insome of the initial simulation experiments, but it had almost no im-pact on the resulting estimates. As such, results using geometricaverages of shares are not reported in this paper.

    As is apparent from the above, one of the major advantages ofGA is that it does not require any information outside of the as-sessed unit to estimate productivity growth. To do so however,the analysis must adopt a number of restrictive assumptions asnoted above and also have access to price information, which isnecessary to parameterise the aggregate production function.

    2.1.2. DEA-based circular Malmquist indexThe most common non-parametric approach for productivity

    measurement utilises Data Envelopment Analysis (DEA) to con-struct Malmqusit indices of productivity change. This approachwas rst proposed by Caves et al. (1982) and later rened byFre et al. (1994).

    This study utilises the notion of a circular Malmquist-type index(thereafter referred to as circular Malmquist), as rst proposed byPastor and Lovell (2005) and rened by Portela and Thanassoulis(2010).

    The circular Malmquist index is based on the observation that,although a measure of distance between two multidimensionalpoints observed at two different time periods can be satisfactorycalculated directly, as per the traditional Malmquist index, a sim-ilar measure of distance can be calculated indirectly, by comparingthe multidimensional points of the two periods relative to a com-mon reference point, or in this case, to a common frontier. Thiscommon frontier is dened as the meta-frontier and since it enve-lopes all data points from all periods, it allows for the creation of aMalmquist-type index which is circular. To draw this meta-frontier, one must assume that convexity holds for all data pointsacross different time-periods. This actually translates to theassumption that what was technologically feasible in a given time

    D. Giraleas et al. / European Journal ofperiod will always be feasible in any future time period, a standardassumption in so-called sequential technology (Tulkens andVanden Eeckaut, 1995). Distances can then be measured usingthe standard DEA models.

    The main advantages of the circular Malmquist index relative tothe traditional (Fre et al., 1994) Malmquist index are the ease ofcomputation and the ability to accommodate unbalanced paneldata. For a more detailed discussion, see Portela and Thanassoulis(2010).

    2.1.3. Corrected OLSCorrected OLS is a deterministic, parametric approach and one

    of the numerous ways that have been suggested to correct theinconsistency of the OLS-derived constant term of the regressionwhen technical inefciency is present in the production process.

    Two different COLS model specications were tested withinthese simulations. Both are based on a pooled regression model(i.e. all observations are included in the same model with nounit-specic effect). The rst model assumes a Cobb-Douglas func-tional form and is used for those experiments where the data isgenerated using the Cobb-Douglas production function. In moredetail, the functional form used is:

    Yit Lait Kbit tcexp eit

    7where eit are the estimated OLS residuals. The standard logarithmictransformation converts (7) into:

    lnYit a ln Lit b ln Kit ct eit: 8It should be noted that the above specication matches perfectlythe data generating process, when measurement error is not in-cluded in the experiments.

    The second COLS model specication assumes a translog func-tional form and is used, together with the Cobb-Douglas functionalform specication, for those simulation experiments where thedata is generated using the piecewise-linear production function.The translog COLS model is given by:

    lnYit ai bL ln Lit bK lnKit ctt 12bLLln Lit2

    12bKKlnKit2

    12cttt

    2 bKL lnKit ln Lit cKt lnKitt cLt ln Litt eit 9

    Inefciency estimates are derived by:

    uit eit max eit 10

    Productivity change is calculated based on the same formula asused for the calculation of true productivity change, substitutingthe true parameters with the various parametric estimates (seeSection 2.2 and Kumbhakar and Lovell (2000)). So, productivitychange is given by:

    d ln TFPCOLSit =dt d ln ECCOLSit =dt d ln TCCOLSit =dt d ln SECCOLSit =dt11

    where

    d ln ECCOLSit =dt uit1 uit 12

    and

    d ln TCCOLSit =dt c 13for the Cobb-Douglas function and

    d ln TCCOLSit =dt ct tctt cKt lnKit cLt ln Lit 14for the translog function.ECCOLSit is the COLS-estimated efciency

    COLS COLS

    ational Research 222 (2012) 673683 675change and TCit is the COLS-estimated technical change. SECitis the COLS-estimated scale efciency change and is given by:

  • uitP 0) and vit represents measurement error v it N 0;r2v . The

    perinefciency component is estimated based on the JMLS (Jondrowet al., 1982) estimator.

    Two different distributions for the inefciency component aretested:

    the exponential distribution, uit Exp(ru) the half-normal distribution, uit N 0;r2u

    When the data is generated using the Cobb-Douglas production

    function, the exponential Cobb-Douglas SFA model is perfectlyspecied, since the data generation process also generates the inef-ciency values from an exponential distribution. The estimatesfrom the half-normal distribution are included in the experimentsto examine the impact of misspecication in the inefciency distri-bution to the SFA productivity change estimates. ProductivitySEit eit 1Xj

    ej;iteitDxj;t;t1 15

    where eit is the sum of all input elasticities (i.e. returns to scale mea-sure), ej,it is the elasticity of input j with respect to output andDxj,it,t1 is the difference in the quantity of input j from periodt 1 to t. For the Cobb-Douglas function, input elasticities are:

    eL;it aeK;it b

    16

    while for the translog function, input elasticities are given by:

    eL;it bL bLLln Lit bKL lnKit cLtteK;it bK bKKlnKit bKL ln Lit cKtt

    17

    2.1.4. Stochastic frontier analysisThe pre-eminent parametric frontier-based approach is Sto-

    chastic Frontier Analysis, which was developed independently byAigner et al. (1977) and by Meeusen and van Den Broeck (1977).The approach relies on the notion that the observed deviation fromthe frontier could be due to both genuine inefciency and randomeffects, including measurement error. SFA attempts to disentanglethose random effects by decomposing the residual of the paramet-ric formulation of the production process into noise (random error)and inefciency.

    As is the case with the COLS approach, two separate SFA modelspecications are used: a Cobb-Douglas functional form is em-ployed for those experiments where the data are generatedthrough a Cobb-Douglas production function, and both a Cobb-Douglas and translog functional form for those experiments wherethe data are generated through a piecewise linear production func-tion. The models are very similar to those used under COLS; in fact,the only difference lies in the specication of the residual.

    In more detail, the Cobb-Douglas model is given by:

    lnYit a ln Lit b ln Kit ct v it uit 18

    whereas the translog model is given by:

    lnYit ai bL ln Lit bK lnKit ctt 12bLLln Lit2

    12bKKlnKit2

    12cttt

    2 bKL lnKit ln Lit cKt lnKitt cLt ln Litt v it uit 19

    where uit represents the inefciency component (and as such

    676 D. Giraleas et al. / European Journal of Ochange is measured in exactly the same way as with COLS, i.e.using Eqs. (11)(17).2.2. Data generating process

    Since the focus on this analysis is on the production side of theeconomic process, information on inputs and output(s) is sufcientfor the estimation of productivity change under the frontier-basedapproaches. However, GA also requires information on prices forboth inputs and output(s) in order to parameterise the productionfunction (see Section 2.1), so price information that is consistentwith the quantities of inputs used and outputs produced by eachassessed unit also needs to be generated.

    Given that the analysis includes both parametric and non-para-metric approaches, the choice of the production function used togenerate the output values for the simulations can have a signi-cant impact on the accuracy of the resulting estimates. If the func-tional form adopted by the parametric approaches matches thefunctional form of the underlying production function, it is ex-pected that the resulting parametric-based estimates would bemore accurate relative to when functional form misspecicationis present. In addition, GA implicitly assumes that the underlyingproduction function is Cobb-Douglas, and as such it would alsobe pertinent for the analysis to examine what happens when thisassumption is violated. To assess the effects of such functionalform misspecication on the overall accuracy of the estimates,two sets of simulations are undertaken: the rst set adopts aCobb-Douglas production function and the second set adopts apiecewise-linear production function. In all cases, the productionfunction assumes that two inputs are used to produce a single out-put, which is also the norm when measuring aggregate productiv-ity change with value added as the output and labour and capitalquantities as the inputs.

    For the rst set of simulations, which utilise a Cobb-Douglasproduction function, output is given by:

    Yit LaitKbittc expmit uit 20where Yit is the output of unit i in time t, Lit is the labour input ofunit i in time t, Kit is the capital input of unit i in time t, mit is mea-surement error (noise) and uit is the technical efciency of unit i intime t. An element of technical change is also included in the formof the time trend t. Output elasticities are given by the parameters aand b for labour and capital, while c represents technical change.The values for the elasticity parameters in these experiments areset to a = b = 0.5 and c = 0.02, i.e. the experiments assume constantreturns to scale. Measurement error is normally distributed withzero mean and variance that changes according to aims of each sim-ulation experiment (some simulation experiments assume no mea-surement error, while others assume varying degrees ofmeasurement error).

    For the second set of simulations, which utilise a piecewise-lin-ear production function, output is given by:

    yi

    1:84Li 0:13Ki for Li=Ki 0:3 and Li=Ki 0:73 and Li=Ki 1:03 and Li=Ki 1:14 and Li=Ki 2:01 and Li=Ki 3:22

    8>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>:

    21

    The labour coefcients, the number of pieces (or facets) and thebreakpoints in the above function were randomly generated, while

    ational Research 222 (2012) 673683the capital coefcients where calculated such that the abovefunction would be convex in K and L (as in all input/output

  • correspondences belong to a convex set), monotonic, continuousand display constant returns to scale.3

    The yi parameter represents clean output, i.e. before the effectsof inefciency, technical change and possible measurement errorare included. The output value used in the simulation experimentsincludes all those elements and is given by:

    yit yitTEitTCt expv it 22where TE represents technical efciency and is given by

    D. Giraleas et al. / European Journal of Operit

    TEit expuit: 23

    TCt represents technical change and is a function of time (t) and aconstant c and is given by:

    TCt tc; 24and vit represents measurement error, which is normally distrib-uted with zero mean and variance that changes according to theaims of each simulation experiment.

    All simulations rely on a panel dataset of 20 units, observedover ve periods (i.e. the total number of observations is 100).4

    The input sets for all units in the rst period are randomly generatedfollowing a uniform distribution U[0,1]; in subsequent periods, theyare scaled by a random, normally-distributed number; the defaultassumption is that this scaling factor follows N(0,0.10), but this pa-per also examines the condition of increased volatility, by setting thestandard deviation to 0.25. It should be mentioned here that thesame scaling factor is used to generate input prices (as discussedin the following section).

    Efciency is also randomly generated and follows the exponen-tial distribution. Two cases, corresponding to different levels ofaverage inefciency are examined:

    for the average levels of inefciency experiments, the inef-ciency term follows Exp (1/7), which results in an average inef-ciency of approximately 12%.

    for the higher levels of inefciency experiments, the inef-ciency term follows Exp (1/2), which results in an average inef-ciency of approximately 32%.

    The denition of productivity change used for this analysis re-lies on the notion of what has come to be known as the Malmquistproductivity index. The Malmquist productivity index is the prod-uct of the index of efciency change ECit, scale efciency changeSEit and technical change TCit (otherwise known as technologicalchange or frontier shift). Taking logs and differentiating acrosstime provides the denition of productivity change across time:

    d ln TFPtrueit =dt d ln ECit=dt d ln SEit=dt d ln TCit=dt 25All of the simulation experiments assume constant returns to scaleand thus the change in scale efciency measure can be ignored.5

    Thus, following Kumbhakar and Lovell (2000), the expression in(25) can be rewritten as:

    d ln TFPtrueit =dt d ln ECit=dt d ln TCit=dt uit1 uit c 26

    3 The function represents a production process under constant returns to scalesince when input values are doubled, so does the total output, for all pieces or facetsof the function.

    4 We used 20 units because it is quite difcult to nd or create complete, highquality datasets of comparable units (be it countries or industries) when examiningaggregate productivity growth. We used a relatively short timeframe (5 periods),because the assumption of a production function that is stable in parameters over a

    longer timeframe might be considered as unrealistic.

    5 Scale efciency for all units is always equal to one, and thus scale efciencychange is equal to zero.Since all parameters in the right-hand side of the above equationsare known in the generated dataset the calculation of true produc-tivity change (in the context of the generated dataset) is trivial.

    2.2.1. Price dataTo generate price information for the experiments consistent

    with each production function, the analysis relies on micro-eco-nomic production theory and assumes that each producer attemptsto minimise costs (Samuelson, 1947), i.e.:

    min Ci wLi LwKi Ks:t: yi f Li;Ki 27assuming that producers utilise two inputs, capital and labour, withprices wKi and w

    Li respectively to produce a given level of a single

    output. To explore the optimal solution for (27), the Lagrangianform is required, i.e.:

    E wLi LwKi K kyi f Li;Ki 28where k is the Lagrangian multiplier. Combining the rst order con-ditions resulting from solving Eq. (28) yields:

    wLiwKi

    @f Li ;Ki

    @L@f Li ;Ki

    @K

    29

    which provides the structural relationship that links input prices toproduction characteristics. Note that due to the duality theory, thesame relationship applies even if the producer is assumed to be out-put maximising.

    For the simulation experiments that assume the Cobb-Douglasfunction specied in (20), Eq. (29) becomes:

    lnLitKit

    ln aw

    Kit

    bwLit

    30

    where a is the output elasticity of labour, b is the output elasticity ofcapital, wKit and w

    Lit are the prices of capital and labour respectively

    for unit i in time t.For the simulation experiments that assume the piecewise-lin-

    ear function specied in (21), Eq. (29) becomes:

    wLiwKi

    ajbj

    31

    where aj is the output elasticity of labour for the jth piece of thepiecewise linear function, bj is the output elasticity of capital forthe jth piece of the piecewise linear function and wKit and w

    Lit are

    the prices of capital and labour respectively for unit i in time t.Note that this analysis and (29) specically assume that alloca-

    tive inefciency, is either zero or time-invariant for each assessedunit.

    Given the above, input prices are generated using the followingapproach:

    First, prices for labour that are unique for each unit are gener-ated for the rst period of the analysis as random draws froma uniform distribution (U(0,0.1]).

    These values are then scaled by a random, normally-distributednumber to generate values for the subsequent periods, similarto the approach used for the generation of the input quantities.As with the input quantities, the experiments test for two dif-ferent scaling factors, one that follows N(0,0.10) and anotherthat follows N(0,0.25), which represents environments withhigher price volatility.

    Eq. (30) or (31) are then used to calculate the true price of cap-ital input, depending on whether the simulation experiments

    ational Research 222 (2012) 673683 677assume a Cobb-Douglas or a piecewise linear production func-tion, respectively. Note that the true price of capital is notobservable by the researcher and, as such, it is not used directly

  • perational Research 222 (2012) 673683in the simulation experiments (this is discussed in more detailbelow).

    Next, output prices are generated by equating total revenues tototal costs:

    pitYit wKitKit wLitLit 32

    pit wKitKit wLitLit

    Yit33

    where pit is the price of output of unit i in period t and Yit is the ef-

    cient level of output of unit i in period t. By using the efcient levelof output in equations (32) and (33), the analysis explicitly assumesthat only the producers that operate on the frontier are able to fullyrecover their total costs; this way, the effects of technical inef-ciency can be linked to total costs and revenues. Note that the abovedoes not mean that an efcient producer achieves zero prots.Rather, the price of capital includes an element commonly referredto as the user cost of capital, which ensures that an efcient pro-ducer receives an appropriate return on the capital invested (i.e.achieves a normal level of prots). By extension, an inefcient pro-ducer will receive a lower return of capital. Also note that this anal-ysis assumes that no producer has sufcient market power togenerate above normal prots.

    The data generation process described above is fully consistentwith the economic theory of production, but unfortunatelyproduces data, and specically data for the price of capital inputs,that are not available in the majority of real life applications andcertainly within the bounds of the National Accounts data. In themajority of real-life situations, the user cost of capital, which is acomponent necessary for the calculation of the price of capital, isnot observable; as such the real price of capital cannot be measuredwith certainty. So to calculate the full price of capital,most GA appli-cations (see for example OECD, 2001 and OMahony and Timmer,2009) adopt an endogenous user cost of capital, which is calculatedresidually. This is achieved by setting capital compensation (i.e. thecost of capital) to be equal to Value Added (which is equivalent torevenue in the setting of these simulations) minus the labourcompensation (i.e. the cost of labour). Since the quantity of capitalcan be estimated using national account data, the price of capitalbased on an endogenous user cost of capital can be derived by:

    wK;GAit pitYit wLitLit

    Kit34

    So, to ensure that the constructed data used for the simulations aresimilar to what is available in real-life applications, the analysis also

    uses this GA-adjusted price of capital wK;GAit

    to generate GA pro-

    ductivity change estimates. This modication ensures that inputshares add up to one and thus allows the use of GA in such a waythat is consistent with EU KLEMS and the methodology proposedby the OECD.

    Finally, in a very few cases in the simulations, the cost of labourcould exceed total revenue, and as such the GA-adjusted price ofcapital is negative. Although negative capital prices are not incon-sistent with theory (Berndt and Fuss, 1986), they are incompatiblewith the standard GA framework, since they result in negative cap-ital shares. To avoid this, the analysis follows the EU KLEMS prac-tice of setting all instances of negative prices to zero.

    2.3. The simulation experiments

    As mentioned in the introduction, the aim of this analysis is to

    678 D. Giraleas et al. / European Journal of Oassess the impacts to the accuracy of the produced productivitychange estimates from GA and a number of frontier-based ap-proaches when the underlying data do not adhere to the perfectcompetition assumptions. To do so, elements of both technicalinefciency and measurement error (noise) are gradually intro-duced to the production function used to generate the simulatedoutput. More specically, different experiments are undertakenassuming two different levels of technical inefciency: averagelevels (uit Exp(1/7)) and higher levels (uit Exp(1/2)) and threedifferent levels of noise: zero noise (mit = 0 for all i and t), modestnoise relative to inefciency (mit N(0,0.05)) and extensive noiserelative to inefciency (mit N(0,0.2)).

    This paper also examines the impact of functional form misspe-cication in the estimates derived from the parametric approachesand GA; as noted in Section 2.2, this is achieved by subdividing thesimulation experiments into two sets. Those in the rst set (de-noted as S1) use the data generating process that assumes aCobb-Douglas production function, while those in the second set(denoted as S2) use the data generating process that assumes apiecewise linear production function.

    Lastly, this paper examines the effects of input and price volatil-ity from one period to the next; this is achieved by generating anew set of data for each simulation experiment which is basedon a more volatile scaling factor (default scaling factor is randomlygenerated and follows N(0,0.10), while the more volatile scalingfactor follows N(0,0.25)). The way all of the above parameters en-ter into the production function is described in detail in Section 2.2.As a reminder, all data generated come from production functionsthat display constant returns to scale and also include and elementof time-invariant technical change (which corresponds to approx-imately 1% annual increase in output). The simulation experimentsundertaken are detailed in Table 1.

    It should also be mentioned that the original analysis also testedwhether the inclusion of fully efcient units would have any im-pact on the summary accuracy measures6; the analysis found thatthe accuracy measures from the simulations which included fullyefcient units are almost indistinguishable from the base case andthus these results are not reported in this paper.

    2.4. Measures of accuracy

    The productivity change estimates produced by each approachare compared to the true rate of productivity change (derived by(26)). Three different measures (MAD, MSE, TMAD as dened be-low) are employed to judge the accuracy of the estimates undereach approach:

    The mean absolute deviation (MAD) of productivity change isgiven by:

    MAD Xn;5

    i1;t1TFPTRUEit TFPESTit =N 35

    where TFPTRUEit is true productivity change and TFPESTit is the esti-

    mated productivity change derived from the approach under exam-ination. The MAD measure provides a robust central estimate of theoverall accuracy of each approach regardless of the sign of the devi-ation between the true and the estimated value. Lower MAD scoresrepresent better overall accuracy.

    The mean square error (MSE) of productivity change is givenby:

    MSE Xn;5

    i1;t1TFPTRUEit TFPESTit 2

    =N 36

    The MSE measure plays a complementary role to the MAD measure,since it gives more weight to larger deviations and thus provides a6 The data generation methodology implemented for these simulations ensuresthat no unit is fully, i.e. 100%, technically efcient.

  • better picture of extreme deviation. Lower MSE scores representbetter overall accuracy.

    The mean absolute deviation of the 25th percentile (topMAD or TMAD) of productivity change, which is the MAD of thetop 25% of the deviations TFPTRUE TFPEST

    . In other words, the

    S1 experiments (the exception is experiment S1.4, whichincludes both relatively high technical inefciency andmeasure-ment error levels, where the overall accuracy of all approachesconsidered does not change when volatility is increased).

    As for the performance of DEA, the analysis raises three majorpoints:

    DEA is the most accurate approach based on the MAD measurewhen technical inefciency is found at relatively high levels inthe data that also do not include any measurement error. Thisis a rather surprising result, since as was mentioned above,the COLS model that is also assessed in the relevant experiment(S1.2) is perfectly specied, given that the S1.2 data are con-structed using a Cobb-Douglas functional form and contain nomeasurement error. And indeed, the COLS approach is moreaccurate than DEA in this experiment based on the TMAD mea-sure and equally accurate based on the MSE measure, whichsuggests that the performance of COLS improves for the unitsthat occupy outlying positions in the dataset.

    The accuracy of the DEA-based estimates decreases at a lowerpace relative to the accuracy of the other deterministicapproaches when inputs and input prices become more volatilefrom one period to the next, in the experiments that do notinclude any measurement error (i.e. S1.1 and S1.2).

    Table 1Simulation experiments.

    Experiment Productionfunction

    Technicalinefciency

    Noise Input and priceVolatilityassumptions

    S1.1a Cobb-Douglas

    average zero default volatility

    S1.2a Cobb-Douglas

    higher zero default volatility

    S1.3a Cobb-Douglas

    average extensive default volatility

    D. Giraleas et al. / European Journal of Operational Research 222 (2012) 673683 679it it

    analysis calculates the absolute deviation of all observations andthen takes into account only the top 25% of those, in order to cal-culate the TMAD measure. This results in a measure that is quitesimilar to the MSE measure, with the notable exception that theunits are not squared (absolute deviations rather than squareddeviations); as such, TMAD is easier to interpret than MSE and pro-vides a clearer indication of the maximum deviation. As is the casewith both MAD and MSE measures, lower TMAD scores representmore accurate estimates.

    In addition to calculating the above measures, the analysis alsoincluded statistical testing to determine whether the pair-wise dif-ferences in those measures between approaches are statisticallysignicant, for all combinations.7 Both standard pair-wise Studentst-tests (assuming unequal variance) and the signed-rank tests wereused for this purpose.

    3. Results

    Tables 29 provide a summary of the three main accuracy mea-sures for all of the assessed approaches, as well as the relativeaccuracy rankings of each approach, taking into account the resultsof the statistical tests for the difference in mean accuracyestimates.8

    3.1. S1 simulation experiments

    In general, the analysis found that the most accurate approachesin the simulation experiments that adopted a Cobb-Douglas pro-duction function are the parametric approaches, i.e. COLS whenmeasurement error was not included in the analysis and SFA whenmeasurement error was present (with one exception discussed be-low). This is not an unexpected result, since the parametric modelsthat are ranked highest in each experiment are perfectly specied,in that they utilise the same functional form as the adopted produc-tion function and, in the case of the best-performing SFA models,assume the correct distribution for the inefciency term.

    Regarding the two non-parametric, determinist approaches, theoverall performance of GA was perhaps surprisingly robust, even ifthe approach displayed the worst (or joint worst accuracy) in themajority of the experiments. In most cases, the difference in accu-racy scores between GA and DEAwas quite small and for the exper-iments that included measurement error, the difference wasstatistically insignicant. The analysis however identied someconditions where the accuracy of the GA quickly deteriorates:

    As technical inefciency becomes more prevalent in the datathat include no measurement error, the accuracy of the GA esti-mates rapidly deteriorates. In the S1.2 experiment, GA wasranked last, while both COLS and DEA were assessed to be sub-stantially more accurate.

    When volatility in inputs and input prices increases from oneperiod to the next, the accuracy of the GA deteriorates at a fasterrate that the other approaches. This is the case in almost all of the

    7 For example, the average MAD score of the DEA estimates over all simulation runsin a single experiment is tested against the average MAD score of the GA, COLS andSFA (where applicable) estimates.

    8 In order to put the various MAD and TMAD measures into context, note that the

    data generation process adopted, both for the Cobb-Douglas and the piecewise-linearfunction, results in an average true productivity change of 2% p.a. but with a standarddeviation of approximately 20%.S1.4a Cobb-Douglas

    average modest default volatility

    S2.1a Piece-wiselinear

    average zero default volatility

    S2.2a Piece-wiselinear

    higher zero default volatility

    S2.3a Piece-wiselinear

    average extensive default volatility

    S2.4a Piece-wiselinear

    average modest default volatility

    S2.5a Piece-wiselinear

    higher extensive default volatility

    S2.6a Piece-wiselinear

    higher modest default volatility

    S1.1b Cobb-Douglas

    average zero higher volatility

    S1.2b Cobb-Douglas

    higher zero higher volatility

    S1.3b Cobb-Douglas

    average extensive higher volatility

    S1.4b Cobb-Douglas

    average modest higher volatility

    S2.1b Piece-wiselinear

    average zero higher volatility

    S2.2b Piece-wiselinear

    higher zero higher volatility

    S2.3b Piece-wiselinear

    average extensive higher volatility

    S2.4b Piece-wiselinear

    average modest higher volatility

    S2.5b Piece-wiselinear

    higher extensive higher volatilityS2.6b Piece-wiselinear

    higher modest higher volatility

  • GA

    004

    21

    10

    2279

    perTable 2Summary accuracy results for the S1 experiments, default volatility.

    Experiment Measure

    S1.1a: CRS CD, 12% average inefciency, no noise MAD (%)MSETMAD (%)

    S1.2a: CRS CD, 32% average inefciency, no noise MAD (%)MSETMAD (%)

    S1.3a: CRS CD, 12% average inefciency, noise N(0,0.2) MAD (%)MSE

    680 D. Giraleas et al. / European Journal of OIn addition to the points made above, some more general com-ments can be made when considering the analysis as a whole:

    When technical inefciency is modest, there is no measurementerror and the input levels and prices between subsequent peri-ods are relatively stable (S1.1 experiment), all approaches pro-vide quite accurate estimates of true productivity change.

    Increased volatility in inputs and prices in subsequent periodsadversely affects accuracy of all approaches, when no measure-ment error is included in the constructed dataset. The DEA esti-mates are the least affected, while the GA estimates are themost affected. Interestingly, when measurement error is intro-duced in the analysis, the increased volatility appears to have

    Table 3Accuracy rankings for the S1 experiments, default volatility.a

    Experiment Measure G

    S1.1a: CRS CD, 12% average inefciency, no noise MAD 3MSE 3TMAD 2

    S1.2a: CRS CD, 32% average inefciency, no noise MAD 3MSE 3TMAD 3

    S1.3a: CRS CD, 12% average inefciency, noise N(0,0.2) MAD 4MSE 4TMAD 4

    S1.4a: CRS CD, 12% average inefciency, noise N(0,0.05) MAD 4MSE 4TMAD 4

    a The rankings take into consideration the results of the statistical tests for the differ

    Table 4Summary accuracy results for the S1 experiments, increased volatility.

    Experiment Measure GA

    S1.1b: CRS CD, 12% average inefciency, no noise MAD (%) 2MSE 1TMAD (%) 11

    S1.2b: CRS CD, 32% average inefciency, no noise MAD (%) 7MSE 12TMAD (%) 30

    S1.3b: CRS CD, 12% average inefciency, noise N(0,0.2) MAD (%) 22MSE 82TMAD (%) 63

    S1.4b: CRS CD, 12% average inefciency, noise N(0,0.05) MAD (%) 6MSE 6TMAD (%) 18

    TMAD (%) 62

    S1.4a: CRS CD, 12% average inefciency, noise N(0,0.05) MAD (%) 5MSE 5TMAD (%) 16COLS DEA SFA (exponential) SFA (half-normal)

    .90 0.40 0.70

    .21 0.04 0.2

    .10 1.30 4.10

    .80 1.70 1.20

    .57 0.62 0.46

    .10 4.80 5.90

    .50 22.40 22.50 12.50 13.70

    .52 78.87 79.08 27.8 32.13

    ational Research 222 (2012) 673683very little impact on the accuracy of the deterministicapproaches and almost no impact at all on the accuracy of thestochastic approaches.

    When measurement error is present, the SFA approaches pro-vide the most accurate estimates. However, when measurementerror is more severe, even the best performing SFA model dem-onstrates quite large deviations from the true productivitychange values (MAD scores of approximately 12.5%). In addi-tion, when measurement error is more moderate, the gains inaccuracy achieved by the SFA models are quite modest com-pared to the deterministic approaches (e.g. GA and DEA MADscores are 5.8%, while the best performing SFA model has aMAD score of 5% in S1.3).

    A COLS DEA SFA (exponential) SFA (half-normal)

    1 21 21 2

    2 11 11 2

    4 4 1 24 4 1 24 4 1 2

    3 4 1 23 4 1 23 4 1 2

    ence in mean accuracy scores.

    COLS DEA SFA (exponential) SFA (half-normal)

    .50 0.80 1.20

    .7 0.16 0.5

    .80 2.70 6.10

    .60 3.00 2.20

    .92 1.92 1.99

    .60 9.30 12.70

    .90 22.50 23.10 12.50 13.90

    .76 79.75 83.16 28.11 33.43

    .90 62.50 63.20 40.00 42.00

    .30 5.80 6.00 5.00 5.40

    .53 5.17 5.66 3.96 4.67

    .5 15.9 16.7 14.1 15.2

    .00 61.70 61.40 40.00 41.30

    .80 5.70 5.80 5.00 5.40

    .3 5.06 5.34 4.04 4.64

    .30 15.90 16.20 14.20 15.10

  • Table 5Accuracy rankings for the S1 experiments, increased volatility.a

    Experiment Measure GA COLS DEA SFA (exponential) SFA (half-normal)

    S1.1b: CRS CD, 12% average inefciency, no noise MAD (%) 3 1 2MSE 3 1 2TMAD (%) 3 1 2

    S1.2b: CRS CD, 32% average inefciency, no noise MAD (%) 3 2 1MSE 3 1 2TMAD (%) 3 1 2

    S1.3b: CRS CD, 12% average inefciency, noise N(0,0.2) MAD (%) 4 4 4 1 2MSE 4 4 4 1 2TMAD (%) 4 4 4 1 2

    S1.4b: CRS CD, 12% average inefciency, noise N(0,0.05) MAD (%) 5 3 4 1 2MSE 5 3 4 1 2TMAD (%) 5 3 4 1 2

    a The rankings also take into consideration the results of the statistical tests for the difference in mean accuracy scores.

    Table 6Summary accuracy results for the S2 experiments, default volatility.

    Experiment Measure GA COLS COLS (translog) DEA SFA SFA (translog) SFA (half-normal)

    S2.1a: CRS PWL, 12% average inefciency, no noise MAD (%) 0.9 2.4 2.9 0.8MSE 0.20 1.08 70.31 0.22TMAD (%) 4.1 8.2 21.6 4.1

    S2.2a: CRS PWL, 32% average inefciency, no noise MAD (%) 2.2 2.4 4.9 1.1MSE 1.09 1.20 132.12 0.39TMAD (%) 8.8 8.8 32.1 5.4

    S2.3a: CRS PWL, 12% average inefciency, noise N(0,0.05) MAD (%) 5.8 6.3 6.5 5.8 7.1 6.1 6.4MSE 5.33 6.24 7.81 5.23 9.11 6.12 6.96TMAD (%) 16.2 17.6 20.6 16.0 21.6 17.7 18.5

    S2.4a: CRS PWL, 12% average inefciency, noise N(0,0.2) MAD (%) 22.9 23.0 23.4 22.8 14.3 15.4 16.6MSE 86.21 86.93 95.96 85.38 38.41 43.70 49.76TMAD (%) 67.7 67.8 71.3 67.4 48.8 51.4 52.2

    S2.5a: CRS PWL, 32% average inefciency, noise N(0,0.2) MAD (%) 23.6 23.6 24.8 23.4 21.5 22.2 22.9MSE 91.33 91.50 148.54 89.94 76.76 82.49 86.35TMAD (%) 69.1 69.5 82.0 68.7 63.6 66.4 67.4

    S2.6a: CRS PWL, 32% average inefciency, noise N(0,0.05) MAD (%) 6.2 6.2 8.7 5.8 7.1 6.5 6.2MSE 6.09 6.12 287.07 5.33 8.56 6.77 6.16TMAD (%) 17.5 17.4 47.2 16.3 20.4 18.6 17.4

    Table 7Accuracy rankings for the S2 experiments, default volatility.a

    Experiment Measure GA COLS COLS (translog) DEA SFA SFA (translog) SFA (half-normal)

    S2.1a: CRS PWL, 12% average inefciency, no noise MAD (%) 1 3 4 1MSE 1 3 4 1TMAD (%) 1 3 4 1

    S2.2a: CRS PWL, 32% average inefciency, no noise MAD (%) 2 3 4 1MSE 2 3 4 1TMAD 2 2 4 1

    S2.3a: CRS PWL, 12% average inefciency, noise N(0,0.05) MAD 1 5 5 1 7 3 5MSE 2 4 6 1 7 3 4TMAD 1 4 6 1 7 4 4

    S2.4a: CRS PWL, 12% average inefciency, noise N(0,0.2) MAD 5 5 7 4 1 2 3MSE 5 5 7 4 1 2 2TMAD 5 5 7 5 1 2 2

    S2.5a: CRS PWL, 32% average inefciency, noise N(0,0.2) MAD 5 5 7 3 1 2 3MSE 5 5 7 3 1 2 3TMAD 5 5 7 3 1 2 3

    S2.6a: CRS PWL, 32% average inefciency, noise N(0,0.05) MAD 3 3 7 1 6 5 3MSE 3 3 7 1 6 5 3TMAD 3 3 7 1 6 5 3

    a The rankings take into consideration the results of the statistical tests for the difference in mean accuracy scores.

    D. Giraleas et al. / European Journal of Operational Research 222 (2012) 673683 681

  • .8

    perTable 8Summary accuracy results for the S2 experiments, increased volatility.

    Experiment Measure GA

    S2.1b: CRS PWL, 12% average inefciency, no noise MAD (%) 2.5MSE 1.77TMAD (%) 12.4

    S2.2b: CRS PWL, 32% average inefciency, no noise MAD (%) 5. 6MSE 8.22TMAD (%) 25.6

    S2.3b: CRS PWL, 12% average inefciency, noise N(0,0.05) MAD (%) 6.3MSE 6.54TMAD (%) 18.7

    S2.4b: CRS PWL, 12% average inefciency, noise N(0,0.2) MAD (%) 23.5MSE 90.67TMAD (%) 68.9

    S2.5b: CRS PWL, 32% average inefciency, noise N(0,0.2) MAD (%) 23.9MSE 94.84TMAD (%) 72.1

    S2.6b: CRS PWL, 32% average inefciency, noise N(0,0.05) MAD (%) 8.4

    682 D. Giraleas et al. / European Journal of O3.2. Summary results for the S2 simulation experiments

    In the S2 experiments (those that use a piecewise linear produc-tion function for the data generating process), it is the non-para-metric approaches that are generally assessed as more accurate,with the exception of the simulations that assume extensivenoise. This was not unexpected, given that the underlying produc-tion function (piecewise linear) is not a perfect match to the func-tional form adopted by the parametric approaches. Theexperiments however demonstrate that the effect of this functionalform misspecication can be quite severe. For example, when nomeasurement error is present, the COLS Cobb-Douglas specica-tion displays MAD scores that are at least twice as large as thosedisplayed by the DEA estimates and the discrepancy in MSE scoresis signicantly larger (at least three times higher).

    Furthermore, the overall accuracy of the COLS specication thatadopts a translog functional form is even worse; in almost allexperiments, the COLS translog specication was ranked last interms of overall accuracy. The SFA translog models however doperform relatively better, compared to their COLS counterparts,

    MSE 13.00TMAD (%) 28.5

    Table 9Accuracy rankings for the S2 experiments, increased volatility.a

    Experiment Measure GA C

    S2.1b: CRS PWL, 12% average inefciency, no noise MAD 2MSE 2 3TMAD 2 3

    S2.2b: CRS PWL, 32% average inefciency, no noise MAD 2 3MSE 2 3TMAD 3 2

    S2.3b: CRS PWL, 12% average inefciency, noise N(0,0.05) MAD 2 4MSE 2 4TMAD 2 4

    S2.4b: CRS PWL, 12% average inefciency, noise N(0,0.2) MAD 5 6MSE 5 6TMAD 4 6

    S2.5b: CRS PWL, 32% average inefciency, noise N(0,0.2) MAD 4 4MSE 4 4TMAD 4 4

    S2.6b: CRS PWL, 32% average inefciency, noise N(0,0.05) MAD 2 3MSE 3 3TMAD 3 3

    a The rankings also take into consideration the results of the statistical tests for the dCOLS COLS (translog) DEA SFA SFA (translog) SFA (half-normal)

    6.8 9.0 1.68.64 20.57 0.96

    23.4 35.5 7.1

    17.4 2.29.00 73.56 1.62

    24.2 63.6 10.8

    8.9 11.4 5.9 8.9 11.4 8.613.40 27.97 5.47 14.59 29.43 13.0827.6 38.8 16.5 29.0 39.8 27.6

    24.4 26.6 23.3 14.7 19.3 17.397.77 116.72 89.43 41.05 67.99 54.3371.1 78.3 68.4 50.5 61.7 54.5

    23.9 31.8 22.9 21.7 28.1 23.394.30 177.16 87.49 78.90 140.91 89.7071.4 95.4 69.1 65.8 87.6 70.0

    9.1 19.1 6.2 9.8 14.6 9.0

    ational Research 222 (2012) 673683especially in the simulations that assume modest noise levels. Inthe extensive noise simulations do the Cobb-Douglas SFA modelsperform relatively better, but the difference in MAD scores is quitesmall (albeit statistically signicant), relative to the overall accu-racy of the estimates.

    Another important issue revealed by the S2 simulations is therelative underperformance of the SFA models under conditions ofmodest noise; in both such experiments (S2.3 and S2.6), GA andDEA, both deterministic approaches, perform better that thestochastic models. In addition, the Cobb-Douglas SFA models that(incorrectly) assume that the inefciency is half-normally distrib-uted is more accurate than the correctly specied Cobb-DouglasSFA exponential model, again under conditions of modest noise.Only when measurement error is more severe (the standard devia-tion is increased from 0.05 to 0.20), is the correctly specied SFA(exponential) model deemed to be more accurate. Its should benoted however that the overall accuracy of all approaches examinedunder the extensive noise conditions is relatively low (the betterperforming SFA models display MAD scores of 14.3% and 21.5% inexperiments S2.4a and S2.5a respectively). This was also the casefor the S1 experiments and suggests that additional research

    13.72 81.15 6.56 16.34 49.48 13.7127.7 66.1 18.9 29.6 53.1 28.2

    OLS COLS (translog) DEA SFA SFA (translog) SFA (half-normal)

    14 14 1

    4 14 14 1

    6 1 4 6 46 1 4 6 46 1 4 6 4

    7 4 1 3 27 4 1 3 27 4 1 3 2

    7 2 1 6 47 2 1 6 47 2 1 6 4

    7 1 5 6 37 1 5 6 37 1 5 6 3

    ifference in mean accuracy scores.

  • would be required to identify approaches that can produce robustestimates under these seemingly adverse conditions.

    It should also be mentioned that the analysis encountered somedifculties when estimating the SFA models using datasets that in-cluded relatively large technical inefciency levels. The problemwas that for some datasets, the SFA models failed to converge to

    rather under some specic circumstances, a specic approach islikely to be more accurate than its counterparts. The analysis alsodemonstrates that frontier-based approaches can usually produceat least as accurate, and in most cases more accurate, productivitychange estimates than the more traditional GA approach. And gi-

    D. Giraleas et al. / European Journal of Operational Research 222 (2012) 673683 683a maximum likelihood solution; this was usually due to persistentnon-concavity in the log-likelihood, which in turn may have beendue to the combined effects of misspecication and large inef-ciency levels. Although the analysis circumvented this issue by dis-carding the problematic datasets, this would not be a possiblesolution in a productivity measurement exercise utilising real data.

    Overall, the S2 experiments revealed that GA and DEA providereasonably accurate estimates under various conditions. In thecase of GA, this is a somewhat surprising result, given that the ap-proach does not acknowledge the presence of technical inef-ciency, which is a not inconsiderable component of productivitychange in these experiments. However, DEA is revealed to be themore accurate approach of the two, and, one could argue, the moreaccurate approach overall, in the S2 experiments.

    Another advantage of the DEA-derived estimates is their appar-ent robustness under conditions of increased volatility in inputsand prices. The S2 simulations showed that increased volatility re-duces the accuracy of all estimates, but to a different degree foreach approach; the same experiments also showed that the DEA-based estimates are in the majority of cases the ones that are leastaffected.

    4. Summary and recommendations

    The major ndings of this simulation study on the accuracy ofthe productivity change estimates are summarised below:

    Deterministic approaches perform adequately even under con-ditions of (modest) measurement error.

    When measurement error becomes larger, the accuracy of allapproaches (including SFA) deteriorates rapidly, to the pointthat their estimates could be considered unreliable for policypurposes.

    Functional formmisspecication has a severely negative impacton the accuracy of all parametric approaches.

    The SFA models that adopt a translog specication appear to bemore accurate in general than the Cobb-Douglas SFA modelswhen the underlying (true) production function is piecewiselinear. The opposite is true for the COLS models; the Cobb-Douglas COLS models are more accurate than their translogcounterparts in such circumstances.

    Misspecication of the inefciency distribution in the SFA mod-els does not appear to have a signicant effect on the overallaccuracy of said approach.

    Increased volatility in inputs and prices from one period to thenext adversely affects the accuracy of all approaches, in almostall experiments. The DEA estimates are the least affected, whilethe GA estimates are the most affected.

    This analysis demonstrates that no productivity change mea-surement approach has an absolute advantage over another, butven that high quality databases on measures of economic growth,employment creation and capital formation are becoming increas-ingly available (EU KLEMS, 2008), the adoption of frontier-basedapproaches when measuring aggregate productivity growth canonly improve our understanding of this elusive and complex topic.

    References

    Aigner, D., Lovell, C.A.K., Schmidt, P., 1977. Formulation and estimation of stochasticfrontier production function models. Journal of Econometrics 6, 2137.

    Balk, B., 2008. Measuring productivity change without neoclassical assumptions: aconceptual analysis. In: ERIM Report Series Reference No. ERS-2008-077-MKT.

    Berndt, E.R., Fuss, M.A., 1986. Productivity measurement with adjustments forvariations in capacity utilization and other forms of temporary equilibrium.Journal of Econometrics 33, 729.

    Caves, D.W., Christensen, L.R., Diewert, W.E., 1982. The economic theory of indexnumbers and the measurement of input, output, and productivity.Econometrica 50, 13931414.

    Del Gatto, M., Di Liberto, A., Petraglia, C., 2008. Measuring productivity. In: Centrefor North South Economic Research, University of Cagliari and Sassari, Sardinia.

    Diewert, W.E., 1992. The measurement of productivity. Bulletin of EconomicResearch 44, 163198.

    EU KLEMS, 2008. EU KLEMS Database, see Marcel Timmer, Mary OMahony & Bartvan Ark, The EU KLEMS Growth and Productivity Accounts: An Overview,University of Groningen & University of Birmingham.

    Fre, R., Grosskopf, S., Norris, M., Zhang, Z., 1994. Productivity growth, technicalprogress, and efciency change in industrialized countries. American EconomicReview 84, 6683.

    Fried, H.O., Lovell, C.A.K., Schmidt, S.S., 2008. The Measurement of ProductiveEfciency and Productivity Growth. Oxford University Press.

    Jondrow, J., Knox Lovell, C.A., Materov, I.S., Schmidt, P., 1982. On the estimation oftechnical inefciency in the stochastic frontier production function model.Journal of Econometrics 19, 233238.

    Koszerek, D., Havik, K., Mc Morrow, K., Rger, W., Schnborn, F., 2007. An overviewof the EU KLEMS growth and productivity accounts. In: Economic Paper No. 290,European Commission.

    Kumbhakar, S.C., Lovell, C.A.K., 2000. Stochastic Frontier Analysis. CambridgeUniversity Press.

    Meeusen, W., van Den Broeck, J., 1977. Efciency estimation from Cobb-Douglasproduction functions with composed error. International Economic Review 18,435444.

    OMahony, M., Timmer, M.P., 2009. Output, input and productivity measures at theindustry level: the EU KLEMS database. Economic Journal 119, 374403.

    OECD, 2001. Measuring productivity: measurement of aggregate and industry-levelproductivity growth. In: OECD.

    ONS, 2007. The ONS Productivity Handbook: A Statistical Overview and Guide. In:Camus, D. (Ed.), Palgrave Macmillan.

    Pastor, J.T., Lovell, C.A.K., 2005. A global Malmquist productivity index. EconomicsLetters 88, 266271.

    Portela, M.C.A.S., Thanassoulis, E., 2010. Malmquist-type indices in the presence ofnegative data: an application to bank branches. Journal of Banking & Finance 34,14721483.

    Samuelson, P.A., 1947. Foundations of Economic Analysis. Harvard University Press.Sharma, S.C., Sylwester, K., Margono, H., 2007. Decomposition of total factor

    productivity growth in US states. The Quarterly Review of Economics andFinance 47, 215241.

    Solow, R.M., 1957. Technical change and the aggregate production function. TheReview of Economics and Statistics 39, 312320.

    Thanassoulis, E., 2001. Introduction to the Theory and Application of DataEnvelopment Analysis: A Foundation Text with Integrated Software. Springer.

    Tinbergen, J., 1942. Theorie der Langfristigen Wirtschaftsentwicklung (on thetheory of long-term economic growth). Weltwirtschaftliches Archiv 55, 511549.

    Tulkens, H., Vanden Eeckaut, P., 1995. Non-parametric efciency, progress andregress measures for panel data: methodological aspects. European Journal ofOperational Research 80, 474499.

    Productivity change using growth accounting and frontier-based approaches Evidence from a Monte Carlo analysis1 Introduction2 Methodology of the current research2.1 Productivity measurement approaches considered2.1.1 Growth accounting2.1.2 DEA-based circular Malmquist index2.1.3 Corrected OLS2.1.4 Stochastic frontier analysis

    2.2 Data generating process2.2.1 Price data

    2.3 The simulation experiments2.4 Measures of accuracy

    3 Results3.1 S1 simulation experiments3.2 Summary results for the S2 simulation experiments

    4 Summary and recommendationsReferences