Analytics at the Electricity Authority

Presentation at the EPOC Winter Workshop

5 September 2013

Outline

• Market Analytics within Market Performance• Priority projects for Market Analytics• Other significant initiatives

• EMI-tools in a browser• Retail tariff information• Profile expansion model (PEM)• GEM data update

Market performance – two teamsMarket Performance

Bruce Smith, GM

Market MonitoringDoug Watt, Manager

Market AnalyticsPhil Bishop, Manager

Data and reporting portal

Provision of tools and data to analyse, monitor and report on

market performance

Workstream supportData warehouse

EMI-tools – develop and maintain suite of

models

Ad-hoc data wrangling and

visualisation tools

Enquiries, reviews and investigations (ERI)

Monitor and report on market performance

Annual publications e.g. Year in review

CBAs for Code amendments

Post-implementation reviews

Market Analytics – near term priorities• Consolidate/rationalise/harmonise data collections in data

warehouse• Access arrangements and data dictionary• Data integrity and checking procedures• Remove duplication

• Upgrade data and reporting portal• Portal will render CDS redundant• More performance metrics – not just data

• Hydrology dataset – more on this in a moment• Use of Github – https://github.com/ElectricityAuthority

More on near term priorities…• New version of EMI-tools with Doasa and vSPD

• Sophisticated model code and data updating/management functionality

• Improved override (vSPD) functionality – sets and parameters

• Doasa more-or-less ready to go now• Doasa ‘lite’, fixed NZ network, independent inflows, etc

• vSPD v1.3 available now with original EMI• Audited in same way as SPD• 3 user interfaces – EMI, Excel, standalone

• vSPD v1.4 being worked on as we speak• Incorporates FTR-rent and scarity pricing rules• Will need to be audited• Codes already on Github

Hydrological datasets

• Last update in 2010• Update overdue and will miss the October 2013 CDS

• Two main hydrological datasets:• Comit Hydro (NIWA)• Opus

• About to start review aimed at:• Improving accuracy and confidence in data

• HRCs, customer compensation scheme• Improve alignment between datasets around plant factors used

to convert volumetric data to GWh and consented storage• Publish annually• Consolidate formats• Snow pack? Southern Oscillation Index?

EMI-tools in a browser• Looking at option to host EMI-tools on web server

• Alongside or integrated with upgraded portal

• Addresses long-standing concern that cost of a GAMS license is prohibitive for some participants and interested stakeholders

• Initially just exploration and visualisation of final pricing cases, i.e. vSPD inputs and outputs, and probably Doasa too• No need for GAMS licensing to be resolved but also no ability to solve

GAMS-based models• .NET framework identical beneath the user-interface layer

• EMI-tools would be able to access data warehouse directly via the portal• SQL ↔ XML – by-pass GDX if just exploration and visualisation• No need for users to manage/store large vSPD input data library• No need to solve vSPD in order to get outputs – they’re in the data

warehouse

Retail tariff information

• Item C8 in Authority work programme• A major undertaking – planning just getting underway now• Most likely begin with an Auckland pilot before rolling out

nationally• Will involve consultation and almost certainly some Code

changes• Aim to:

• Enhance analysis and performance monitoring of retail competition

• Facilitate innovation in product offerings, switching providers and new entry into retail sector

• Be future-proofed re: smart metering data provision

Profile expansion model (PEM)• A model to help understand the cost imposed on the system

by various types of load• Residential versus industrial versus commercial• Flat versus peaky – day, season, year• Winter versus summer• Actual or contrived

• A bit like GEM but without the computational burden• A mix of generation technologies each with capital and operating costs,

operational capabilities, etc• No integer variables• Load able to be represented chronologically with up to 17,520 periods or as

a load duration curve

• Start with a clean sheet of paper or expand the existing generation fleet

Load profile factors, 2012North Island

Co

mm

erci

al South Island

Ind

ust

rial

Res

iden

tial

GEM

• Generation Expansion Model• MBIE and Transpower use GEM frequently• Used much less so these days at the Authority• Several other occasional users – based on calls for help• Also, many people find the dataset useful, if not the model• About to start a small project to update the GEM database

and the data preparation tools• (Probably) a Python-based toolkit to source, document, and

compile GEM datasets• GDX ↔ Text/CSV ↔ XML ↔ SQL

• Move codes from Google Codes to Github• Integrate into EMI-tools (next year?)