1

Load Management and Forecasting in EU Energy markets.

Matrica

2

Simone Conference Simone Conference Dubrovnik Dubrovnik

October 2008October 2008

Load forecastingLoad forecastingBy

MatricaMatricawww.matrica.co.uk

Simone Conference Simone Conference Dubrovnik Dubrovnik

October 2008October 2008

Load forecastingLoad forecastingBy

MatricaMatricawww.matrica.co.uk

3

HistoryMatrica has been providing consultancy and software solutions for the Energy industry since 1994 and has successful installations in over 20 major energy companies which includes: EDF, EON, Essent, DONG, Centrica, Sonelgaz, etc.

Academic affiliations: Kings & Imperial Colleges, University of London, ENI University of Milan; student researchers: Post graduate programme

Business Activities

• Software Applications & Consultancy for the Energy industry• Products for Load Demand Management, Price Forecasting, Data Warehousing,

Data Transfer and Analysis• Solutions for: Distributors, Transporters, Generators and Energy Traders in both

Gas and Electricity markets• Proprietary in-house developed products; own all IPR plus selected third party

products/services: “the best of breed”

Presenter:Prudence Mauthoor, Founder DirectorMSc Applied Mathematicsprudence@matrica.co.uk

Company & the PresenterCompany & the Presenter

4

What constitutes good forecast and what does it What constitutes good forecast and what does it involve?involve?

Good Forecasts:

A tool that is right for the job and provides reliable, timely and most accurate forecasts. It also should provide some degree of flexibility and scalability

It involves:A range of forecasting techniques to support different forecasting needs

Fast & reliable data processing

Capable & easy to use software

Expertise (industry and forecasting)

5

Forecasting methodologies and supplementary Forecasting methodologies and supplementary techniquestechniques

Forecasting methods for the energy industry (especially the gas supply/demand) include:

• Multivariate nonlinear and self learning techniques: Artificial Neural Networks

• Multivariate linear techniques: Regression,

• Exponential Smoothing (e.g. ARIMA, trend correction, etc.)

• Interpolation/extrapolation, filtering, similar day

• Impact factors that can be applied directly or after preprocessing: correlation, weather correction

• Optimization functions automatically determining the best forecasting parameters and topology for the learning process

• Optimization functions to automatically select the impact factors: Combination, GRNN or Forward Regression

• Segmentation techniques: rule base, statistical, self learning, customer profiles, load shapes

• Monte Carlo Simulation

6

Data Processing, Software and ExpertiseData Processing, Software and Expertise

Wining combination: a fast and reliable data processing; capable software plus human expertise

Ability to quickly process changes in the customer portfolio is a must in short-term forecasting

– For major accounts: ability to capture new meter measurement or info and to adapt forecast in real time based

an automatic optimization routine for setting appropriate new forecast parameters and conducting new analysis almost instantly

Validation and data enhancements as the captured data series are not always complete or reliable:

Data Repair facilities to identify and rectify incorrect or incomplete data: comparing the bad data with historic equivalents, etc.

– Preventing the avoidable inaccuracies significantly improves the reliability of the forecasts

But there is more to a successful operation than merely software/hardware solutions. Success begins and ends with knowledge, and that means people with expertise

7

A holistic forecasting to support the energy value chainA holistic forecasting to support the energy value chain

Co

ns

um

ers

Ge

ne

rati

on

/ P

rod

uc

tio

n

$ $ $

Tra

din

g

$

Re

tail

$

Energy Markets

Markets Infrastructure(May be regulated)

8

A Holistic Energy Forecasting A Holistic Energy Forecasting ApproachApproach

• Need for a complete forecasting vision for the long, medium, short and the immediate short terms and a common tool kit for this purpose

• Energy Demand Forecasting is a permanent process that follows a fixed cycle: starts with the capture, storage, verification and management of consumption data

• Forecasts are made using specially designed models/scenarios based on consumption patterns and impact factors (drivers) so the data and parameters for the scenarios can easily be changed

• Ability to develop and compare different scenarios; scenarios that correspond with the actual situation most closely and are used as the ultimate forecast

• This is compared with the actual consumption data to enable an imbalance analysis

• The results of this analysis are in turn used as input for data management at the start of the cycle

9

Customer Segmentation: Different Consumer Customer Segmentation: Different Consumer GroupsGroups

10

Customer Segmentation: 1Customer Segmentation: 1stst step in step in forecastingforecasting

Customer portfolio is rarely homogenous and Energy Demand forecasting is a balance between: The top down approach : a single forecast of the combined energy consumption of all

customers

The bottom up approach: the individual forecasts for every customer then combined to produce an end result

In reality a middle course is adopted: bottom up for very large C&I with options, and intelligent data grouping or segmentation and “Customer Profiling” by: customer or contract type, geographic region, user sector SICC, etc.

Standard approach is splitting customers into clusters creating a number of virtual load curves or virtual meters; each segment representing hundreds/thousands of consumer data series

Forecasts are produced for each of these virtual meters or segments and aggregated

Process for individual forecasts and aggregated cluster forecasts can be applied concurrently

Finally obtaining an optimal demand forecast

11

Demand Variability: Demand Variability: Customer type, Geographical & Seasonal SegmentationCustomer type, Geographical & Seasonal Segmentation

Careful segmentation is vital

12

Medium-term Forecast

Short-term Forecast

Intraday Balancing

Flexi market Optimisation

Forecasting for gas dispatching and portfolio Forecasting for gas dispatching and portfolio managementmanagement

For Minimizing imbalance costs within an entry-exit model by optimizing flexibility and imbalance costs for gas dispatching need:

– A ‘capable’ forecasting tool and mechanisms to apply the intraday corrections to re-nominate supply at the latest permitted time

– Rolling reliable forecasts a few days ahead (mid term) to make sure that facilities (contingencies) are in place for the intraday balancing.

– Intraday readjustment schematics to re-nominate:

A good starting point for balancing provides the best chance for minimizing the flexibility costs

13

Day Ahead ForecastingDay Ahead Forecasting

Accurate balancing needs both the day ahead and intraday forecasting - very important steps in reducing penalties (for deregulated regimes).

For the day ahead quantitative forecast techniques need: Historic consumption data for the individual exit points to create a model

that explains the underlying patterns and forms the basis for future forecasts Good knowledge of the behavior patterns of different consumer groups to

create appropriate customer segments and different forecasting approaches/methods for different parts of the portfolio

Understanding of the drivers influencing the demand variations, e.g. weather.

History of the data for the relevant influencing drivers Using frequent measurement of data where possible, especially for large

volume C&I customers Using customer nominations/forecasts directly (depending on the contract

terms), or as a factors to create better forecasts thus improving the overall forecast results

14

Variable SelectionVariable Selection

After segmentation, consumption data for each segment should be analysed for correlations with known drivers that can impact the forecasting models

Seasonal and periodic patterns should be taken into account Intraday forecasting should use the latest available measurements and

automatically adjust/apply correction for imbalance in previous hours Forecasting environment should respond to sudden changes in demand

(e.g. industrial unplanned stoppages) Rules should be in place to handle faulty and missing data, especially for

the intraday results: “distinguishing between ‘real’ data errors and unplanned shutdowns”

Regulatory requirements must be considered: Granularity Gate closure Settlement process Error tolerance

Data availability should be considered: historic and ongoing Future path, Budget, Available Expertise

15

Variables for the Bottom up, Top-down or Profile demand models are typically:

• Number of customers in the group (aggregate models)

• Weather elements, temperature, wind, radiation, humidity, pressure, etc.

• Seasons, months, week days, hours, sub-hours

• Holidays, special days, regional factors, work patterns and practices, school holidays, academic start/end

• Electricity demand (affecting gas demand for power production): TV Programs, Football, election, new year, etc.

• Long/short days

• Gas-day Issues

• Price tariff & Marketing (demand response issues)

• Economic & Socio-economic factors, etc. (for long term)

• Technological, political, environmental, social, regulatory changes and trend (for long term models)

Typical Demand DriversTypical Demand Drivers

16

Domestic Aggregate DemandDomestic Aggregate Demand

17

Industrial Demand ForecastIndustrial Demand Forecast

18

UK Winter Demand with Weather Correction UK Winter Demand with Weather Correction ComponentComponent

19

Flexibility options in gas supply and demandFlexibility options in gas supply and demand

• Prep for the intraday balancing starts with the Mid-term portfolio management: anticipating the changes in demand and providing flex options for balancing purposes

• Reliable rolling forecasts for the next few days enables gas and power positions be corrected several days in advance - in time to limit intraday risks and opportunity costs considerably

• Flexi purchase contracts are expensive - so it is worthwhile to check value/cost of all sources of flexi options in supply and demand:

– large energy intensive industries with interrupt options – Flexi power generators selling to power exchange used for correcting intraday

imbalances - their opportunity cost is often underestimated - Spark Spread often shows attractive peaks during the high gas demand and vice versa - so other flex sources may be more attractive

• Supply side flexi contracts can be found in hub trading, storage, special flex contracts, production fields and full-service supply contracts

• Flexi sources have their own capacity limits, marginal cost and risk/uncertainty levels

Optimisation of gas dispatching is for cost/penalty minimization making best use of flexibility opportunities is for the short term earnings

20

Medium & Long Term Forecasting and ScenariosMedium & Long Term Forecasting and Scenarios

• Medium term models

– in principal have the same structure as the short term models but the process includes scenario forecasts to examine stresses/extremes and use of the options

– models are usually based on multivariate techniques in order to provide the elasticity for the scenario analysis and the confidence intervals

• Long term models

– based on a combination of mathematical models for underlying /changing patterns and simulation of conditions (Monte Carlo)

– rule based assumptions to incorporate planned commitments: e.g. housing developments, license for new hospitals, factories, power plants, etc.

– Sometimes a profile forecasting approach is used in which case profiles must be multivariate to provide the elasticity

• Forecasting Scenarios– Confidence Limits, Optimal Exit point volumes, Interconnector usage

volumes, National Aggregate Demand (U.K, Spain, etc)

• In all cases What-if Scenarios and Monte Carlo simulation techniques should be used to analyse and assess the risks

21

Customer ProfilesCustomer Profiles: : for Long term Forecasting and Scenario for Long term Forecasting and Scenario planningplanning

Load profiles or shapes represent the average daily patterns of energy usage for a particular load class. This is used to build a load model from the represented land use

Planners need to know how loads behave during the system peak Hourly data should be taken from the SCADA at the distribution-level feeder to

develop the final load curves

22

Scenario Analysis for mid term risk and long term forecastsScenario Analysis for mid term risk and long term forecasts

Assess the probability of forecasts diverging from the normal patternsAssess the probability of forecasts diverging from the normal patterns Assess the Risk, Stress test and for the Planning purposesAssess the Risk, Stress test and for the Planning purposes Scenario analyse based on: (i) different weather conditions or (ii) calendar variables (iii) any Scenario analyse based on: (i) different weather conditions or (ii) calendar variables (iii) any

other variables in the model: other variables in the model: Business growth; Hot/Mild winter; High cost of Oil/Gas; Business growth; Hot/Mild winter; High cost of Oil/Gas; Global warmingGlobal warming

Generate Scenario ForecastsGenerate Scenario Forecasts Run Monte Carlo simulations on scenarios for risk assessmentsRun Monte Carlo simulations on scenarios for risk assessments

23

Defining the Growth Rates in long term Defining the Growth Rates in long term ModelsModels

• Tow events cause load growth: – new customers and

– the consumption growth of existing customers • The number of new connects should be forecasted then converted

into growth rates using counts of existing customers by class

• The growth rates are adjusted to produce an overall % load growth from the base year to the first forecasted year

• The consumption growth rates are through load demand models

24

Long Term Planning: Spatial Load ForecastingLong Term Planning: Spatial Load Forecasting

Spatial forecasting is marrying GIS models (load profile geographical models) with the distribution system and development plans, e.g. transportation infrastructure, forecast of urban centers, location and timeline of community

development, etc.

Spatial load forecasting can: Predict the extent and timeline of community development, Identify where infrastructure

investments should be directed Explore the impacts of new initiatives or localized developments Demonstrate the effects of changes in fully developed areas, e.g. Conversion to central air

conditioning and conversion from winter to summer peaks , community redevelopment

Anticipates the location of future demand in a relative timeline, assisting capital investment planning, acquisitions, permitting lead times,

Allows a more productive interaction with the community being served Allows energy planners and engineers to predict large load additions to the system

years in advance of their current methods Helps determine where new infrastructure should be added and the authorities to plan

property for requisite permits, etc. Forecast results are used to predict future load centers, set priorities on projects, and

obtain budgeting approval while minimizing risk

25

Spatial Load Forecasting: the processSpatial Load Forecasting: the process

First need to determine the load classes, representing the regional load defined by consumer load class

It is critical to model both the known and the anticipated development in a region for both supply and demand

Land will generally develop to its highest financial potential - often developers have the money and influence to alter development plans

The model should construct:

Long term scenarios based on the changes in the current load patterns

Developments, taking into the account the practical constraints: the resources, timescales or social constraints, e.g. regions of employment or commercial activity that encourage residential and commercial growth in locations

26

Demand ResponseDemand Response

It is a form of demand management under the direct control of the consumer rather than being operated and controlled by the energy provider or utility.

Demand response differs in a number of ways from the DSM (demand side management):

– active load control, time-of-use metering, and passive energy efficiency programs.

– It is currently applicable to power except for the large C&I customers

– But it impacts the gas demand, therefore its trends should be considered in gas models

Demand response assumes and implies that large customers are financially motivated to actively control their demand and scale

back levels of their loads, or shed them on short notice, and/or shift their loads from on-peak to off-peak times

– Therefore the dispatcher must either rely on the customer forecasts and/or price curves and the spot markets

For C&I customers and aggregated loads (e.g. energy cooperatives), demand response programs can work like the interruptible and curtailable rate programs that have been available for sometimes

Whereas curtailment and interrupt tariffs offer low average prices as incentives, Demand Response programs tend to offer reasonable prices plus actual payments made to customers when they make a demand shift or cut back

27

Other important issuesOther important issues

There are some other important issues impacting the forecasting process and quality of forecasts:

Forecasting is a process of learning from the past and applying what’s learnt to predict the future. Therefore the quality of data is paramount; Rubbish in Rubbish out

Good Quality Data is means: Representative: relevant to the problem, not too out dated Complete, no missing values Relevant granularity and scales: e.g. ½ hourly, etc. Accurate, credible and validated sufficient in volume and consistent (e.g. kWh & kW)

Issues to do with missing data and grouping difficult data Error analysis and confidence limits for models diagnostics and

forecast validation Automation for speed and most importantly error avoidance

Data management and capable reporting

28

29

Summary: Role of Demand Summary: Role of Demand ManagementManagement

Since 90s advances in technology (communications, internet, manufacturing process, etc.) has had an impact on the energy marketplace and resulted in a need for a more advanced generation of demand management tools/services

Shift from residential load management to C&I demand management has increased the effectiveness of DSM initiatives and coupled with the spiraling prices has resulted in a need:

– sophisticated demand forecasting, demand response initiatives and active load control systems are needed to meet demand/ supply imbalances and maintain a reliable and stable delivery system

– Implementing a holistic demand management system that can incorporate DSM and demand response initiatives and combine load controls when spot prices exceed a certain set point and market pricing is volatile in an ongoing basis

– The deregulated energy business itself can implement demand management programs as a strategic measure, part of an enterprise-wide operating strategy, for the benefit of both energy supplier and user

– Finally, an important incentive for the players is the opportunity to sell reserves created by effective demand management programs and demand response initiatives

30

www.matrica.co.ukTel: +44 (0)2 7536 2951

Thank YouThank You

Questions?Questions?