SLIDE 1

HOW WEBMETHODS IS USED TO

SUPPORT THE ELECTRICITY AND GAS

MARKETS IN AUSTRALIA.

September 2017

WARREN VELLA

PRINCIPAL INTEGRATION SPECIALIST

SLIDE 2

TOPICS

1. Who is AEMO?

2. AEMO’s Integration Objectives

3. The AEMO Integration Platform

4. AEMO’s 4 Year Integration Journey

5. How AEMO achieves HA with the webMethods

platform

6. How AEMO deploys changes rapidly

7. What’s next for AEMO?

SLIDE 3

WHO IS AEMO?

SLIDE 4

WHO IS AEMO?

As Australia’s independent energy markets and power systems operator,

AEMO provides critical planning, forecasting and power systems information,

security advice, and services to the energy industry.

ELECTRICITY

AEMO operates Australia's National

Electricity Market (NEM), the

interconnected power system in

Australia’s eastern and south-eastern

seaboard, and the Wholesale

Electricity Market (WEM) and

power system in Western Australia.

GAS

AEMO operates gas retail markets

in NSW, ACT, QLD, SA, VIC and WA.

AEMO also operates wholesale gas

markets in south-eastern Australia.

AEMO also operates the Natural

Gas Services Bulletin Board

(GBB), and is system operator of the

Victorian Gas Declared

Transmission System.

SLIDE 5

NATIONAL ELECTRICITY MARKET (NEM)

SLIDE 6

TWO FUELS

ELECTRICITY

GAS

40,000 KM / 5 min dispatch

200 Terawatt Hours

9 Million customers

7.7 billion dollars

market value traded

4 hour schedules

200.8 Peta Joules

4.8 Million customers

900 million dollars

market value traded

SLIDE 7

AEMO’S INTEGRATION OBJECTIVES

SLIDE 8

To improve is to change;

to be perfect is to change often.

W. Churchill

SLIDE 9

AEMO’S BUSINESS OBJECTIVES

Market

participant and

consumer

satisfaction

Ability to

innovate

Adapt to

legislative and

industry

change

quickly.

Promote efficient investment in and operation of Australia’s

electricity and gas markets for the long-term interests of

consumers.

SLIDE 10

AEMO’S IT OBJECTIVES

The cornerstone

of any

automation

platform is

visibility

Achieving agility

through

governance

Enhance,

improve and add

value

Shift from data processing and managing paper trails to

management of automated processes and IT systems.

SLIDE 11

THE AEMO INTEGRATION PLATFORM

SLIDE 12

THE AEMO INTEGRATION PLATFORM

B2B Messaging

User Portal

CSV/XML Transformation

Legacy System Inter-Operability

Real-Time SLA Management

API Management Platform

A single, centrally managed webMethods integration platform

supporting both gas and electricity market systems providing:

SLIDE 13

FUNCTIONAL VIEW

Participant FTP Gateway*

Participant API Gateway*

Transaction Management System

Message Management

System

Participant A Endpoint

Participant B Endpoint

Transaction Management

System

Message Management System

Participant FTP Gateway*

Participant API Gateway*

Participant A AEMO Participant B

Hub

API

FTP

SLIDE 14

LOGICAL VIEW

Electricity Retail Participants

Retailers LNSPsMDPs

Others

Gas Retail Participants

RetailersDNSPsMDAs

Others

Other Markets

WholesaleSettlements

etc

Protocols

FTPebXML SOAP

Services

Enterprise Interaction / Gateway

Gas Retail Application Electricity Retail & Other Apps

ebXML MSHs(Gas FRC - webMethods)

Managed File Transfer

Other Markets FTP & File Servers (e.g.

MIBB)

MSATSGas FRC Retail

1

FTPWebservices

Enterprise Integration

Universal Messaging & Integration Layer (webMethods)

MSATS File Handlers & other File processors

Other Market AppsEMMS

DWGM STTM

3

4

5

2 6

7

8

1020

9

MSATS Participant

Server (Logical)*

New SMP Protocols

New protocols – Elec Gateways &

Elec MSHs (webMethods)

MIBB

OTHERS

11

12

13

14

15

16

17

19

20

21

22

Impacted Systems

OPDMSHSM

EMMS Participant

Server (Logical)*

MMS FTP 23

GSH

GBB

23

24

EMM

S SM

B

20

a

a a

a

18

Gas Retail via ebXML

Elec Retail via Hokey Pokey

Other Markets

Gas Retail via FTP

Elec Webservices

Impacted SMPData flow

SLIDE 15

VOLUME AND THROUGHPUT

Integration Platform

+650 TN Partners

184 production instances

50 production

servers

5TB

database

+2million API calls per

day

12,000 messages

p/minute or 200 p/second

SLIDE 16

KEY OPERATIONAL CHALLENGES

• Gas – 99.9%

• Electricity – 99.99%

Multiple SLA’s

• Outage cannot affect both fuels

Different Maintenance Windows

• Gas – ebXML (SOAP)

• Electricity – FTP and REST API’s

Different B2B Messaging Technologies

• GB’s per day

Large Data Growth

SLIDE 17

AEMO’S 4 YEAR

INTEGRATION JOURNEY

SLIDE 18

AEMO’S INTEGRATION AND API JOURNEY

2014 | Renovate

Re-build webMethods

from 6.5 to 9.6 for SA, VIC, QLD

and WA.

2015 | Integrate

Integrate NSW & ACT B2B and

B2M messaging

2016 | Strengthen

Build ICC capability and

upgrade platform from 9.6 to 9.12

2017 | Expand

Build and rollout API management platform capability

and expand to electricity.

Gas

Retail Market

Integration

Electricity

Retail Market

Integration

SLIDE 19

API MANAGEMENT PLATFORM

SLIDE 20

HOW AEMO ACHIEVES HA

WITH THE WEBMETHODS PLATFORM

SLIDE 21

THREE CORE OPERATIONAL VALUES

Zero Impact Work Standards

Strict Maintenance and Support Schedules

Mature High Availability Practices

SLIDE 22

ZERO IMPACT WORK STANDARDS

No-outage maintenance

work

An issue on one fuel should

not affect the other fuel

The system cannot tolerate

reduced capacity during

maintenance work

Upgrades to the database

infrastructure must not force

an outage to the applications

SLIDE 23

STRICT MAINTENANCE AND SUPPORT

SCHEDULES

Platform Stability

The platform is upgraded every 12

months

Security and Reliability

Product patches and fixes are rolled out every 3 months

Continuous Integration

Codes fixes are rolled out on-demand using

CI/CD practices

24/7 on-call support

24/7 on-call support with end-to-end SLA

monitoring

Integration Competency Centre

SLIDE 24

MATURE HIGH AVAILABILITY PRACTICES S

ite T

ran

sfe

r

Validate near-

instant recovery every 6 weeks

Au

tom

ati

on

Minimise manual

deployment and transfer

steps Ho

t S

tan

db

y

Applications in the

inactive site are always running

SLIDE 25

HOW DO WE EXECUTE SITE TRANSFERS?

SLIDE 26

THE AEMO SITE TRANSFER

B2B Zone ESB

API Zone DB

Market Systems Portals

Inbound

Load

B2B Zone ESB

API Zone DB

Market Systems Portals

NSW QLD

5 min replication

SLIDE 27

THE AEMO SITE TRANSFER

B2B Zone ESB

API Zone DB

Market Systems Portals

Inbound

Load

B2B Zone ESB

API Zone DB

Market Systems Portals

NSW QLD

5 min replication

SLIDE 28

THE AEMO SITE TRANSFER

B2B Zone ESB

API Zone DB

Market Systems Portals

Inbound

Load

B2B Zone ESB

API Zone DB

Market Systems Portals

NSW QLD

5 min replication

SLIDE 29

THE AEMO SITE TRANSFER

B2B Zone ESB

API Zone DB

Market Systems Portals

Inbound

Load

B2B Zone ESB

API Zone DB

Market Systems Portals

NSW QLD

5 min replication

SLIDE 30

THE AEMO SITE TRANSFER

B2B Zone ESB

API Zone DB

Market Systems Portals

Inbound

Load

B2B Zone ESB

API Zone DB

Market Systems Portals

NSW QLD

5 min replication

SLIDE 31

THE AEMO SITE TRANSFER

B2B Zone ESB

API Zone DB

Market Systems Portals

Inbound

Load

B2B Zone ESB

API Zone DB

Market Systems Portals

NSW QLD

5 min replication

SLIDE 32

SITE TRANSFER BY ZONES

Inte

gra

tio

n P

latfo

rm

API Management Zone (API) Zone

API Enterprise Gateway, API Gateway

Enterprise Service Bus Zone (ESB) Zone

Integration Server, Universal Messaging

Business to Business (B2B) Zone

Enterprise Gateway, Trading Networks, ebXML

Portal Zone My webMethods Server,

API Portal

Reporting Zone Optimise, MashZone, CentraSite, Command

Central

Continuous Integration (CI) Zone

Deployer, Asset Build Environment, Jenkins

SLIDE 33

SITE TRANSFER BY BAYS

Node 1

Node 2

Node 3

Node 1

Node 2

Node 3

Node 1

Node 2

Node 3

Node 1

Node 2

Node 3

Bay A

Bay B

Bay A

Bay B

NSW QLD

9.6 9.6

9.6 9.6

SLIDE 34

SITE TRANSFER BY BAYS

Node 1

Node 2

Node 3

Node 1

Node 2

Node 3

Node 1

Node 2

Node 3

Node 1

Node 2

Node 3

Bay A

Bay B

Bay A

Bay B

NSW QLD

9.6 9.6

10.0 10.0

SLIDE 35

SITE TRANSFER BY BAYS

Node 1

Node 2

Node 3

Node 1

Node 2

Node 3

Node 1

Node 2

Node 3

Node 1

Node 2

Node 3

Bay A

Bay B

Bay A

Bay B

NSW QLD

9.6 9.6

10.0 10.0

SLIDE 36

SITE TRANSFER BY BAYS

Node 1

Node 2

Node 3

Node 1

Node 2

Node 3

Node 1

Node 2

Node 3

Node 1

Node 2

Node 3

Bay A

Bay B

Bay A

Bay B

NSW QLD

10.0 10.0

10.0 10.0

SLIDE 37

SITE TRANSFER BY BAYS

Node 1

Node 2

Node 3

Node 1

Node 2

Node 3

Node 1

Node 2

Node 3

Node 1

Node 2

Node 3

Bay A

Bay B

Bay A

Bay B

NSW QLD

10.0 10.0

10.0 10.0

SLIDE 38

HOW DO WE DEPLOY CHANGES

RAPIDLY?

SLIDE 39

CONTINUOUS INTEGRATION

SLIDE 40

WHAT’S NEXT FOR AEMO?

SLIDE 41

WHATS NEXT FOR AEMO?

Advanced automation,

automatic transfers

Maturing continuous integration, for quicker

deployments

Quarterly patching cycles

webMethods 10 upgrade,

quick provision and tear downs

Extension of the API’s

to the Business to

Market messages

(B2M)

Next wave: Building

enhanced API

services

SLIDE 42

QUESTIONS?