Analytics at the Electricity Authority
Presentation at the EPOC Winter Workshop
5 September 2013
Outline
• Market Analytics within Market Performance• Priority projects for Market Analytics• Other significant initiatives
• EMI-tools in a browser• Retail tariff information• Profile expansion model (PEM)• GEM data update
Market performance – two teamsMarket Performance
Bruce Smith, GM
Market MonitoringDoug Watt, Manager
Market AnalyticsPhil Bishop, Manager
Data and reporting portal
Provision of tools and data to analyse, monitor and report on
market performance
Workstream supportData warehouse
EMI-tools – develop and maintain suite of
models
Ad-hoc data wrangling and
visualisation tools
Enquiries, reviews and investigations (ERI)
Monitor and report on market performance
Annual publications e.g. Year in review
CBAs for Code amendments
Post-implementation reviews
Market Analytics – near term priorities• Consolidate/rationalise/harmonise data collections in data
warehouse• Access arrangements and data dictionary• Data integrity and checking procedures• Remove duplication
• Upgrade data and reporting portal• Portal will render CDS redundant• More performance metrics – not just data
• Hydrology dataset – more on this in a moment• Use of Github – https://github.com/ElectricityAuthority
More on near term priorities…• New version of EMI-tools with Doasa and vSPD
• Sophisticated model code and data updating/management functionality
• Improved override (vSPD) functionality – sets and parameters
• Doasa more-or-less ready to go now• Doasa ‘lite’, fixed NZ network, independent inflows, etc
• vSPD v1.3 available now with original EMI• Audited in same way as SPD• 3 user interfaces – EMI, Excel, standalone
• vSPD v1.4 being worked on as we speak• Incorporates FTR-rent and scarity pricing rules• Will need to be audited• Codes already on Github
Hydrological datasets
• Last update in 2010• Update overdue and will miss the October 2013 CDS
• Two main hydrological datasets:• Comit Hydro (NIWA)• Opus
• About to start review aimed at:• Improving accuracy and confidence in data
• HRCs, customer compensation scheme• Improve alignment between datasets around plant factors used
to convert volumetric data to GWh and consented storage• Publish annually• Consolidate formats• Snow pack? Southern Oscillation Index?
EMI-tools in a browser• Looking at option to host EMI-tools on web server
• Alongside or integrated with upgraded portal
• Addresses long-standing concern that cost of a GAMS license is prohibitive for some participants and interested stakeholders
• Initially just exploration and visualisation of final pricing cases, i.e. vSPD inputs and outputs, and probably Doasa too• No need for GAMS licensing to be resolved but also no ability to solve
GAMS-based models• .NET framework identical beneath the user-interface layer
• EMI-tools would be able to access data warehouse directly via the portal• SQL ↔ XML – by-pass GDX if just exploration and visualisation• No need for users to manage/store large vSPD input data library• No need to solve vSPD in order to get outputs – they’re in the data
warehouse
Retail tariff information
• Item C8 in Authority work programme• A major undertaking – planning just getting underway now• Most likely begin with an Auckland pilot before rolling out
nationally• Will involve consultation and almost certainly some Code
changes• Aim to:
• Enhance analysis and performance monitoring of retail competition
• Facilitate innovation in product offerings, switching providers and new entry into retail sector
• Be future-proofed re: smart metering data provision
Profile expansion model (PEM)• A model to help understand the cost imposed on the system
by various types of load• Residential versus industrial versus commercial• Flat versus peaky – day, season, year• Winter versus summer• Actual or contrived
• A bit like GEM but without the computational burden• A mix of generation technologies each with capital and operating costs,
operational capabilities, etc• No integer variables• Load able to be represented chronologically with up to 17,520 periods or as
a load duration curve
• Start with a clean sheet of paper or expand the existing generation fleet
Load profile factors, 2012North Island
Co
mm
erci
al South Island
Ind
ust
rial
Res
iden
tial
GEM
• Generation Expansion Model• MBIE and Transpower use GEM frequently• Used much less so these days at the Authority• Several other occasional users – based on calls for help• Also, many people find the dataset useful, if not the model• About to start a small project to update the GEM database
and the data preparation tools• (Probably) a Python-based toolkit to source, document, and
compile GEM datasets• GDX ↔ Text/CSV ↔ XML ↔ SQL
• Move codes from Google Codes to Github• Integrate into EMI-tools (next year?)