Influence of Contextual Variables: An Application to Agricultural Research

Evaluation in Brazil

Geraldo da Silva e Souza Geraldo da Silva e Souza Eliane Gonçalves GomesEliane Gonçalves Gomes

Roberta Blass StaubRoberta Blass Staub

Introduction Embrapa monitors, since 1996, the production

processes of each of its 37 research centers• Nonparametric production model: DEA

o Technical and cost efficiency of each research center

o Identification of benchmarks o Identification of exogenous factors (contextual

variables) potentially associated with efficiency

Problem: Define a proper data generating process allowing two stage statiscal inference

2

Objectives

Explore the probabilistic interpretations of FDH and the FDH conditional to define a proper data generating process for efficiency measurements

Use the ratio of conditional FDH to FDH, and two stage inference, to assess statistical significance of covariates of interest

3

FDH as an alternative to DEA FDH does not assume convexity for the

underlying technology. Consistent under convexity

• Free disposability of inputs

• FDH : input orientation

1 1 1

, , , , 1, 0,1 , 1...n n n

p lj j j j j j

j j j

x y R y y x x j n

1 1 1

ˆ , ; , , 1, 0,1n n n

j j j j j jj j j

x y Min y y x x

1... 1...ˆ ,

ij

j n i p i

xx y Min Max

x

1... 1...

ˆ ,ij

j n i l i

yx y Max Min

y

4

FDH: Probabilistic Interpretation Production process described by a joint

probability distribution. One is concerned with dominance

Technical measure of efficiency is defined by

Empirical estimate relative to set

, Pr , Pr PrH x y ob X x Y y ob X x Y y ob Y y

, inf ; , 0 inf ; 0x y H x y F x y

1

1

,ˆ ,

n

j jjn

jj

I X x Y yx y

I Y y

5

Condtional FDH A vector Z of contextual variables in Rk affects

the production process

Interest is in the conditional distribution of (X,Y) given Z=z

• Technical efficiency conditional to Z=z

• Empirical Estimator (continous covariate)

, inf ; , 0 inf ; , 0x y z H x y z F x y z

1

1

,ˆ ,

n

j j jj

n

j jj

I X x Y y K z z hx y z

I Y y K z z h

6

For multivariate Z• Joint kernel: product of univariate Epanechnikov

kernels

• Bandwidth: Minimizes aproximated integrated mean square error

• Technical efficiency depends on the kernel only through the bandwidth h

• Influence of Z: Daraio and Simar (2007) suggest nonparametric regression using as response variable

1...,

ˆ ,j j

i iji p

j y y z z hx y z Min Max x x

ˆ ,

ˆ ,

j j j

j

j j

x y zq z

x y

7

1 1

k fjt jt j jtjt ff

R q z c R q z R z

Application Panel Data: (xjt, yjt, zjt), j=1...n, t=1...T T small relative to n Arellano and Bond (1991) model on ranks Dynamic model allowing serial correlation GMM

Model assumes panel random effects It is not robust relative to the presence of

second order serial correlation8

Embrapa’s Production Model Output: Weighted average of 28 variables

falling into 4 categories• Scientific production

• Technical publications

• Development of technologies, products and processes.

• Diffusion of technologies and image Inputs: Vector of dimension 3 : capital, labor

and operational costs 37 research centers Period: 1999-2006

9

Contextual variables (covariates) Intensity of Partnerships (negative association

would imply unwanted competiton among research centers)

Income generation effort

Processes improvements

Administrative changes

Size (three levels)

Type (three levels)

10

Graphical Analysis

Evolution of FDH and DEA-BCC suggests differences

Evolution of FDH conditional and FDH suggests differences

11

.2.4

.6.8

1.2

.4.6

.81

.2.4

.6.8

1.2

.4.6

.81

.2.4

.6.8

1.2

.4.6

.81

0 2 4 6 8 0 2 4 6 8 0 2 4 6 8 0 2 4 6 8 0 2 4 6 8

0 2 4 6 8 0 2 4 6 8

1 2 3 4 5 6 7

8 9 10 11 12 13 14

15 16 17 18 19 20 21

22 23 24 25 26 27 28

29 30 31 32 33 34 35

36 37

dea_bcc fdh

time

Graphs by unit_dmu

12

.2.4

.6.8

1.2

.4.6

.81

.2.4

.6.8

1.2

.4.6

.81

.2.4

.6.8

1.2

.4.6

.81

0 2 4 6 8 0 2 4 6 8 0 2 4 6 8 0 2 4 6 8 0 2 4 6 8

0 2 4 6 8 0 2 4 6 8

1 2 3 4 5 6 7

8 9 10 11 12 13 14

15 16 17 18 19 20 21

22 23 24 25 26 27 28

29 30 31 32 33 34 35

36 37

fdh_cond fdh

time

Graphs by unit_dmu

13

Goodness of Fit : Dynamic Panel Instruments: Second order differences of rank

efficiencies, first order differences of continous contextual variables, time and categorical dummy variables.

No indication of second order serial correlation (p =11%)

Sargan’s specification test is not significant (p =76% )

14

Variable Coefficient Standard Error z P>|z| [95% Confidence Interval]

Lag1 0.0377 0.2152 0.18 0.861 -0.3841 0.4595

Lag2 -0.2694 0.0905 -2.98 0.003 -0.4468 -0.0920

Z1 (mproc) -0.0108 0.0418 -0.26 0.796 -0.0928 0.0712

Z2 (rec) -0.2011 0.0977 -2.06 0.040 -0.3929 -0.0096

Z3 (par) 0.0025 0.0453 0.05 0.956 -0.0863 0.0913

Z4 (adm) -0.5931 1.4980 -0.40 0.692 -3.5292 2.3429

Z5 (type2) 31.7611 102.2497 0.31 0.756 -168.6446 232.1668

Z6 (type3) -83.7362 153.0349 -0.55 0.584 -383.6790 216.2067

Z7 (medium) 23.7291 75.5381 0.31 0.753 -124.3228 171.7810

Z8 (large) 46.7976 94.9387 0.49 0.622 -139.2788 232.8741

Intercept 32.3361 46.9948 0.69 0.491 -59.7719 124.4442

15

Final Considerations The effects size and type are not statistically

significant with joint p-values of 84% and 86% respectively

Processes improvements, financial resources acquisition and management change have negative signs. But only financial acquisition resources is statistically significant. Case of favorable covariates

The intensity of partnerships is detrimental to the production process but it is not statistically significant

The lag 2 negative and statistically significant component of the response provides indication of an effort for improvement. Two periods are necessary for that to be achieved 16

Arellano, M., Bond, S., 1991. Some tests of specification for panel data: Monte Carlo evidence and an application to employment equations. Review of Economic Studies 58(2), 277-297.

Arellano, M., Bover O., 1995. Another look at the instrumental variable estimation of the error-components models. Journal of Econometrics 68, 29-51.

Blundell, R., Bond, S., 1998. Initial conditions and moment restrictions in dynamic panel-data models. Journal of Econometrics 87, 115-143.Conover, W.J., 1998. Practical nonparametric statistics. Wiley, New York.

Cooper, W.W., Seiford, L.M., Tone, K. 2000. Data Envelopment Analysis: a comprehensive text with models, applications, references and DEA-Solver software. Kluwer Academic Publishers. Boston.

Daraio, C., Simar, L., 2007. Advanced Robust and Nonparametric Methods in Efficiency Analysis. Springer, New York.

References

17

Deprins, D., Simar, L., Tulkens, H. 1984. Measuring labor inefficiency in post offices. In: Marchand, M., Pestieau, P., Tulkens, H. (Eds.), The Performance of Public Enterprises: concepts and measurements. North-Holland, Amsterdam, pp. 243-267.

Embrapa, 2006. Manual dos indicadores de avaliação de desempenho das unidades descentralizadas da Embrapa: Metas quantitativas - Versão para ano base 2007. Superintendência de Pesquisa e Desenvolvimento, Brasília.

Greene, W.H., 2007. Econometric Analysis. Prentice Hall, New Jersey.

Kerstens, K., Eeckaut, P.V., 1999. Estimating returns to scale using non-parametric deterministic technologies: A new method based on goodness-of-fit. European Journal of Operational Research 113, 206-214.

Kotz, N., Johnson, L., 1989. Thurstone’s theory of comparative judgment. Encyclopedia of Statistical Sciences 9, 237-239.

Podinovski, V.V., 2004. On the linearisation of reference technologies for testing returns to scale in FDH models. European Journal of Operational Research 152, 800-802. 18

Saaty, T.L., 1994. The Fundamentals of Decision Making and Priority Theory with the Analytic Hierarchy Process. RWS Publication, Pittsburgh.

Seiford, L.M., Thrall, R.M., 1990. Recent developments in DEA, the mathematical programming approach to frontier analysis. Journal of Econometrics 46, 7-38.

Silverman, B.W., 1986. Density Estimation for Statistics and Data Analysis. Chapman and Hall, London.

Simar, L., Wilson, P.W., 2007. Estimation and inference in two-stage, semi-parametric models of production processes. Journal of Econometrics 136 (1), 31-64.Soleimani-damaneh, M., Jahanshahloo, G.R., Reshadi, M., 2006. On the estimation of returns-to-scale in FDH models. European Journal of Operational Research 174, 1055-1059.

Soleimani-damaneh, M., Mostafaee, A., 2009. Stability of the classification of returns to scale in FDH models. European Journal of Operational Research 196, 1223-1228.

19

Souza, G.S., 2006. Significância de efeitos técnicos na eficiência de produção da pesquisa agropecuária brasileira. Revista Brasileira de Economia 60 (1), 91-117.

Souza, G.S., Alves, E., Avila, A.F.D., 1999. Technical efficiency in agricultural research. Scientometrics 46, 141-160.

Souza, G.S., Gomes, E.G., Magalhães, M.C., Avila, A.F.D., 2007. Economic efficiency of Embrapa’s research centers and the influence of contextual variables. Pesquisa Operacional 27, 15-26.

Torgerson, W.S., 1958. Theory and methods of scaling. Wiley, New York.

Wilson, P.W., 2008. FEAR: A software package for frontier efficiency analysis with R. Socio-Economic Planning Sciences 42, 247-254.

20