Clean Energy Regulators Initiative - Role of Efficiency Analysis

Dr. Konstantin Petrov, DNV KEMA

4 November 2013

Introduction to Network Regulation Module 2: Role of Efficiency Analysis

Introduction to Network Regulation

4 November 2013

Agenda

1. Introduction to Efficiency Analysis

2. Methods for Efficiency Assessment

3. Application of Efficiency Results

4. International Examples


4 November 2013

Introduction to Efficiency Analysis

Usually, competition forces companies to

operate in an efficient way

But, in areas where competition does not work

(e.g. natural monopolies - transmission,

distribution networks) regulation is needed to

limit excessive pricing and to set incentives for

efficient performance

In cost-based regulatory schemes, a fixed rate of

return compensates the companies and little

incentives to minimise costs are provided

Incentive regulatory schemes are explicitly

designed to provide incentives for cost-efficiency

Incentive regulation is based on benchmarking

which regulators use to assess efficiency of

regulated companies and to set targets

Why measure efficiency?

Cap Regulation

Actual Cost

Current price level

Current price + Inflation

Current price + Inflation – productivity growth

Efficiency gains

time

Influenced by company

Influenced by company

Set by regulator Base price

for next

regulatory

period

3


4 November 2013


Efficiency characterises the productivity of a company compared with the productivity of other

companies.

What is efficiency?

EfficiencyA = OutputsA

InputsA +

“Correction for

Environment”

Distribution Company A

e.g. # employees, fuel,

operational costs,

Input Factors

e.g. # customers, delivered

energy (kWh), peak load (kW)

Output Factors

e.g. firm size, network topology, climate, topography,

terrain, task complexity

Environmental Factors

4


4 November 2013


Inefficiency is a deviation from the optimal point on the production or cost frontier.

Two main sources for this deviation: technological deficits and problems due to a non-optimal

allocation of resources into production

Sources for efficiency changes:

- Technological change (frontier shift): change in production technology within the sector

- Efficiency change (catch-up): change in efficiency of production

• Change in the scale of production (scale efficiency)

• Pure technical efficiency change

- Allocative efficiency

• Input mix allocative efficiency: producing same outputs with different mix of inputs

• Output mix allocative efficiency: producing different level of outputs with same mix of

inputs

- Changes in operating environment

Why are companies inefficient?

5


4 November 2013

Agenda





6


4 November 2013

Model Application

Methods for Efficiency Assessment

Steps in Benchmarking Analysis

Choice of

Approach

for Efficiency

Measurement

Choice of

Model

Parameters

Step 1 Step 2 Step 3 Step 4 Step 5

Model Specification

Choice of

Sample Size

and Data

Collection

Model Run Result

Validation

1) DEA

2) SFA

3) OLS

4) COLS

5) Partial

1) Model Orientation

2) Constant or Variable Returns to Scale

3) Definition of Inputs/ Outputs

1) National versus International

2) Comparison Criteria

3) Data Validation

1) Application of Alternative

Approaches

2) Sensitivity Analysis

Related to Input and Output

Parameters

3) Check for Outliers

Be

nc

hm

ark

ing

Pro

ce

ss

D

ec

isio

n S

eq

ue

nc

e

7


4 November 2013


Benchmarking (efficiency performance assessment) is applied based on a variety of

methods ranging from basic indicators to more complex measures

Methods differ in the standard of comparison

Benchmarking influences the allowed revenue of companies and the price level

reliability of inefficiency scores and the method chosen is crucial for the regulator

There is no consensus among regulators at to which methodology is the best

Benchmarking should not be applied mechanically

Sometimes different methods are applied simultaneously

Frontier methods preferred by regulators (in particular DEA and SFA)

- Parametric (econometric) models (Germany, UK)

- DEA analysis (Norway, the Netherlands, Germany, several countries in CEE)

- Reference network models (Spain, Sweden, Chile)

Choice of Benchmarking Method

8


4 November 2013


Overview of Benchmarking Methods

Benchmarking Methods

Partial

Methods Total Methods

Non-

Parametric Parametric

Index

Methods

Data

Envelopment

Analysis

(DEA)

Stochastic

Frontier

Analysis

(SFA)

Ordinary

Least

Squares

(OLS)

Corrected

Ordinary

Least

Squares

(COLS)

Total

Factor

Productivity

(TFP)

Uni-

Dimensional

Ratios

Performance

Indicators

Linear

Programming Econometrics

Engineering

Models

Reference

Networks

(Virtual

Networks)

9


4 November 2013


Partial methods use uni-dimensional ratios; comparison of single performance

indicators between firms:

Partial methods produce simple, easy to calculate and straightforward indicators of

performance

But: they fail to account for the relationships between different input and output

factors and do not recognise trade-offs between different improvement possibilities

Total methods can capture this trade-off

…at the expense of higher computational complexity

Partial vs. Total Methods

Productivity Indicators:

• GWh/Employee

• OPEX/GWh

• OPEX/Employee

• GWh/Line Length

Financial Indicators:

• Debt/Equity Ratio

• Return on Investment

(ROI)

• Return on Capital

Employed (ROCE)

10


4 November 2013


Index method – Total Factor Productivity (TFP):

- Measure of physical output of a regulated company produced by a given

quantity of inputs

- With multiple inputs (Y) and outputs (X), outputs are usually weighted by

their revenue shares (sR) and inputs are weighted by their cost shares (sC)

- Usually used for assessments of company performance over time

Frontier-based methods:

- based on the concept that all companies should be able to operate at an optimal efficiency

level/ “frontier” that is determined by other efficient “peer” companies in the same sample

- The companies that form the efficiency frontier use the minimum quantity of inputs to

produce the same quantity of outputs (input oriented model)

- The efficiency frontier is used as a reference against which the comparative performance of

all other companies (that do not lie on the frontier) is measured

- The distance to the efficiency frontier provides a measure for the inefficiency

Index- vs. Frontier-based Methods

n

j

jjC

m

i

iiR

Xs

Ys

TFP

1

1

11


4 November 2013


Non-parametric vs. Parametric Models (Frontier-based Methods)

x1 /y

Most Efficient

Observation

Input X

(Costs)

Output Y

Parametric Methods

Ordinary Least

Squares (OLS)

Stochastic Frontier

Approach (SFA) Corrected Ordinary

Least Squares

(COLS)

Econometric methods use cost or production

functions and regression analysis. SFA accounts

for stochastic noise in the data sample

Efficiency Frontier

A

B

C

D E

Most Efficient

Companies E

E’ Inefficiency

Non-Parametric Methods 𝑋2

𝑌

𝑋1

𝑌

DEA uses multi-input / output analysis

based on linear programming

12


4 November 2013

Agenda





13


4 November 2013

Application of Efficiency Results

Efficiency Assessment and Price Control

Efficiency

Assessment

Efficiency

Scores

Efficiency

Improvement

Targets

Integration in

Price

Control

Allowed Revenue (Tariffs) Efficiency

Inte

rface

Benchmarking

Approach

Sample

Model Orientation

Data Collection

Data Validation

Conversion

Convergence Time

Convergence Profile

Inefficiency Caps

Efficiency Bands

Integration

Chargeable Basis

Capex Treatment

Revenue Requirements

Regulatory Formula

14


4 November 2013


Once calculated efficiency scores should be converted into efficiency increase

requirements (X-factors).

Defining Efficiency Increase Targets

Companies

Efficiency

Score

Measures of relative

inefficiencies towards best

performance

A B C D E

Conversion

(definition of

efficiency

increase

targets)

X-Factor ensures ex-ante

sharing of anticipated efficiency

gains

X-Factor can be calculated:

- Indirectly as a difference between

the level of actual costs and target

(efficient) costs

- Directly without reference to target

costs using just past performance

In some regulatory regimes the

X-factor has a dual function:

- Efficiency improvement

- Revenue profiling

15


4 November 2013


The X-factor prescribes the rate of change in the company’s prices or revenues, reflecting the

expected transition from the existing price level towards the efficient price level

The efficiency convergence may be based on an initial one-off cut or gradual adjustment path

during the regulatory period

Efficiency Convergence Speed

Advantage of initial one-off cut, prices can be

brought to more realistic levels at once

Large one-off adjustments quickly eliminate

inefficiencies at the beginning, but decrease

incentives for further efficiency improvements

by the company

Incentives for efficiency increase can be

further supported by efficiency carry-over

schemes: companies are allowed to continue

keeping part of the efficiency gains of the

previous period

Allowed

Revenue

Initial

Level

Initial one-off cut

1 2 3 4 5 Regulatory Period

Proportional

decrease

16


4 November 2013

Agenda





17


4 November 2013

International Examples

Country Benchmarking Methods Benchmarking Sample

United Kingdom COLS until 2009; DEA and OLS (OPEX) 14 electricity distribution companies

8 gas network distribution companies

The Netherlands DEA (total controllable costs) 19 Dutch utilities (electricity)

Germany DEA, SFA (total controllable costs) 198 electricity distribution companies

188 gas distribution companies

Austria DEA, MOLS (total controllable costs) 20 electricity distribution companies

20 gas distribution companies

Finland DEA, SFA (OPEX) 88 electricity distribution companies

Norway DEA 150 national distribution utilities (electricity)

Sweden Reference network model until 2007;SFA, DEA 170 electricity distribution companies

Spain Network reference model 5 large and 320 smaller electricity distribution

companies

Portugal DEA 11 gas distribution companies

Poland OLS; COLS & DEA

18


4 November 2013

End of Session 2.

Dr. Konstantin Petrov

Service Line Leader Markets & Regulation / Business Line Director Gas Consulting Services

DNV KEMA Energy & Sustainability

KEMA Consulting GmbH

Kurt-Schumacher-Str. 8

53113 Bonn

Tel: +49 228 44690 56

Fax: +49 228 4469099

Mobile: +49 173 515 1946

E-mail: [email protected]

19


4 November 2013

www.dnvkema.com

20


4 November 2013

Appendix: Efficiency Assessment Models


4 November 2013


Calculates the relative Input-Output efficiency of a regulated company by benchmarking an

individual company in relation to the best-practice (most efficient) companies

Companies that are able to produce a given output at minimum cost or a maximum output with

a given input define the best-practice frontier that envelops all data points

Inefficiency is determined by the distance between the observed company and the best-

practice frontier

Calculation of inefficiency is conducted via a series of linear programming (mathematical

software needed)

The programs will output a series of efficiency scores, which may be normalised, ranked, and

split according to a number of components (scale, purely technical, allocative etc.)

Advantages: multi-dimensional method; functional relationships between input and output

factors not required; distinguishes between different types of inefficiency

Disadvantages: results may be influenced by random errors; no information about statistical

significance of the results; danger of over-specification of model and “made-up” results for

efficiency scores; “extreme” parameters regarded as efficient “by default”

Non-parametric Model: Data Envelopment Analysis (DEA)

22


4 November 2013


Non-parametric Model: Data Envelopment Analysis (DEA)

Output 1 Input 1

Input 2

Data Envelope

A

B

C

D E

most efficient

companies

F

F’

Inefficiency

Input Minimisation

Inefficiency

Output 2

Data Envelope

A

B

C

D

E

F

most efficient

companies

F’

Output Maximisation

G

G’

23


4 November 2013


Regression analysis: Mathematical relationship (functional form) that describes the relationship

between a dependent variable and one or more independent variables

Use of regression residuals to characterise relative distances between observations in the

sample

Treats best practice as a “stochastic” process (a mix of true efficiency and “random noise”

effects, SFA)

Advantages: ability to control for unobserved heterogeneity among companies; less sensitive

to inputs and/or outputs than other parametric models; allows to assess the significance of

each network cost driver; considers stochastic errors explicitly

Disadvantages: requires assumptions of functional form; requires large data sets in order to

create a robust regression relationship; complex and statistically demanding

Parametric Models

24


4 November 2013


Parametric Models

Corrected OLS (COLS)

Ordinary Least Square (OLS)

Stochastic Frontier Analysis (SFA)

Most efficient

observation

Input (Costs)

Output

25


4 November 2013


Construct an efficient (engineering-designed) reference network according to commonly

accepted planning principles and taking into account technical and geographical constraints

The regulated firm’s relative (in)efficiency is estimated by the firm’s performance in relation to

the virtual network

Advantages: Virtual network models are not dependent on obtaining and analysing data of

“real” companies; does not require a significant set of comparable companies as benchmarks

Disadvantages: It might be complicated and difficult to specify; model sensitive to changes in

inputs; reasons for the deviation from reference network might be beyond control of the

company

Virtual Network Models

26

Technology

Clean Energy Regulators Initiative - Role of Efficiency Analysis