ERCOT Control Center Situation Awareness - EMMOSemmos.org/prevconf/2013/4.ERCOT Control Center... · ERCOT Control Center Situation Awareness ... –One-lines that couldn’t pass

ERCOT Control Center Situation

Awareness

Michael E. Legatt, Ph.D.Principal Human Factors Engineer

EMMOS Users Conference

Monday, September 23, 2013


September 23, 2013 2

Copyright / Trademark

The materials included within the presentation are the intellectual

property of the Electric Reliability Council of Texas, Inc.

All content contained herein, literary, pictorial, graphic or otherwise are

copyright protected © 2013, ERCOT, Inc., Michael E. Legatt, Ph.D.

“Macomber Map®” is a registered trademark of ERCOT, Inc.

All rights reserved.



X



X



Situation Awareness

“…the perception of the elements in the environment within a volume of

time and space, the comprehension of their meaning, and the projection

of their status in the near future.” – Endsley, 1988

• ERCOT level:

• 23 million customers, 73% competitive

choice (6.7 million meters, 15-minute data)

• 40,530 miles of high-voltage transmission

• Over 74 GW capacity, including over 10GW

renewables

• Data ranging from 300 times per second to

once every 4 seconds

• Many systems to handle network model,

reliability, market, outages, etc.

• Control room operators need to see the “big

picture”/decision support



• Need for situation awareness and user experience enhancements in control rooms

• Blackout of 2003 and other low-SA root causes

• Main issues:– Need for wide-area and high-level views

– “Alarm storm”

– “Dust storm”

– Locations of equipment

– Navigation between one-lines

– Naming conventions for equipment

– Data integration across multiple systems

– Increased cognitive load as operators increasingly managing more tasks with more complex systems.

• This trend is only continuing (e.g., advanced metering)

History



• Fatigue / diet / exercise / tobacco / caffeine

• Reliability coordinators are not power systems engineers

• Power systems engineers are not reliability coordinators

• Collaborative process

• Shared stress

• Holistic analyses needed

• Agile software development

• Engagement of all players, leveraging different expertise

• Prototyping

Needs communications

http://www.soloseo.com/blog/2007/11/03/what-the-customer-actually-wanted/

http://www.soloseo.com/blog/2007/11/03/what-the-customer-actually-wanted/



• NMMS Post-Processor– Substation One-Line Validation process

• Previously manual process

• Every substation one-line validated for topology and layout (~4.5k)

– One-lines that couldn’t pass full validation are assigned to a network model coordinators

– Contingency process• Breaker-to-breaker tracing

– One-lines are automatically generated (~ 11.5k)

– Single circuit contingencies auto-generated

– Special flags for transient (mid-model change) state, and for contingencyable/normal operation state

– Extracts include contingency deltas

• Double-circuit contingencies, tracing around seed elements

– Macomber Map topological model & one-lines

– Inclusion of OE Future models – bulk changes covered in advance

Integrating technologies in ERCOT operations



Control Room Complexity

10

Black Start Simulations



• ERCOT brings more than

700 TOs, QSEs and

ERCOT operators together

for annual simulations

• QSEs use simplified

generation control systems

to simulate unit restorations

ERCOT Black Start Preparedness Training



• TOs and ERCOT operators

use Macomber Map to see

wide area views and control

only their equipment

• Participants each perform their

roles against ERCOT’s EMS

training simulator

• Simulation of severe weather

event, many transmission lines

unavailable

ERCOT Black Start Preparedness Training



• User Interface Enhancements to support:– One-Line view/navigation (substation & breaker-to-

breaker)

– Element control panel and switching capabilities for

transmission operators

– Improved regional/island views

– Navigational support for reactive devices, next start

units

– Generator status, synchroscope locations

– User action logging/auditing

– Integration of black start plan information

Macomber Map Enhancements for Preparedness Training



• Fatigue

• Political pressure / media pressure

• Sustained vigilance

• Separation from loved ones

• Unknown event duration

• Cognizance of life-or-death nature, individual and

societal

• Poor diet (e.g., high carbohydrate, sugar, caffeine)

• Nicotine withdrawal

Human components of black start



• Communications issues– Three-way communications

– Positive communications

– Shared telephone conference / need for “net control”

– Humor to diffuse stress

• “Hurry up and wait”– Different areas of responsibility mean many people

waiting with little to do while others working

Human components of black start



Overall Progress

• Of over 120,000 switching actions, operators made 4

errors.

– 1st trip @ 39 hours (shortly after shift turnover)

– 2nd trip @ 40 hours

– 3rd trip @ 41 hours

– 4th trip @ 48 hours

• 203 total simulation hours (starting at hour 35)

• Intentional load shedding in preparation for island pickup



Interesting facts

• TO behavior patterns different between 2012 and 2013.

– In 2012, with just one-line displays, operators moved

more quickly and had more island collapses

– In 2013, using Macomber Map, operators moved

more methodically, making significantly fewer

switching errors.

– However, existing human errors still occurred likely

due to fatigue/circadian rhythm.



Interesting facts

• Positive communications improved ERCOT/TO

communications and efficiency, reduced self-reported

stress



Interesting facts

• 2012 Simulation

– 27 GW restoration

– 17 simulator days for restoration

• 2013 Simulation

– 40 GW restoration

– 8.5 simulator days (203 hours) for restoration

– More than 120,000 switching actions, 4 errors



Into the future…



• Current, Macomber Map is supported by a team of

SMEs across ERCOT

• Ongoing enhancements and leveraging for various

groups within ERCOT

Future of Macomber Map



• Open source initiative this year

– First, client code with data interfaces

– Early next year, addition of server-side code

• Groups that have expressed interest in Macomber

Map:

– ISO/RTOs

– TDSPs & QSEs

– Academic Researchers

– Vendors

Open Sourcing Macomber Map

Documents

ERCOT Control Center Situation Awareness - EMMOSemmos.org/prevconf/2013/4.ERCOT Control Center... · ERCOT Control Center Situation Awareness ... –One-lines that couldn’t pass