461

Telecommunication Service Provisioning and Delivery in the Electrical Power Utility

Working Group

D2. 26

April 2011

Telecommunication Service Provisioning and Delivery in the Electrical Power Utility

Working Group D2.26

April 2011

Members

Mehrdad MESBAH, Convenor (France), Robert EVANS (Australia) Dugald BELL (Australia), Jan PIOTROWSKI (Poland) Jorge MENDES (Portugal) Pedro GAMA (Portugal),

Ion NEDELCU (Romania), Matjaz BLOKAR (Slovenia) Paul SCHWYTER (Switzerland), Paul RENSHALL (United Kingdom) Claudio TRIGO de LOUREIRO (Brazil), Elton BANDEIRA de MELO (Brazil) Suzana JAVORNIK VONCINA (Croatia),

Janine LEIFER (Israel) Masayuki YAMASAKI (Japan) Kazuto IWASAKI (Japan), Eva LASSNER (Hungary) Lhoussain LHASSANI (Netherland) Olav STOKKE (Norway),

Jorge FONSECA (Portugal) Pedro MARQUES (Portugal) Danilo LALOVIC (Serbia), Juan Antonio GARCIA LOPEZ (Spain)

Copyright 2011 Ownership of a CIGRE publication, whether in paper form or on electronic support only infers right of use for personal purposes. Are prohibited, except if explicitly agreed by CIGRE, total or partial reproduction of the publication for use other than personal and transfer to a third party; hence circu-lation on any intranet or other company network is forbidden. Disclaimer notice CIGRE gives no warranty or assurance about the contents of this publication, nor does it accept any responsibility, as to the accuracy or exhaustiveness of the information. All implied warranties and conditions are excluded to the maximum extent permitted by law.

ISBN: 978-2-85873-150-3

2

1 INTRODUCTION ................................................................................................................5 2 COMMUNICATION SERVICE IN THE POWER UTILITY ............................................8

2.1 Introduction...................................................................................................................8 2.2 EPU Communication Services......................................................................................9

3 OPERATIONAL APPLICATIONS ...................................................................................13 3.1 Protection Communication .........................................................................................13

3.1.1 State Comparison Protection Schemes (Command Schemes)............................15 3.1.2 Teleprotection Signalling Systems .....................................................................16 3.1.3 Analog Comparison Protection Schemes............................................................17 3.1.4 System-wide Protection Schemes .......................................................................17

3.2 Energy Management, SCADA and WAMS Communications ...................................18 3.2.1 SCADA Communications...................................................................................18 3.2.2 Inter-Control Centre Communications ...............................................................19 3.2.3 Remote Control Centre Operator Consoles ........................................................20 3.2.4 Generation Control Signaling .............................................................................20 3.2.5 Wide Area Monitoring System (PMU Communications) ..................................20

3.3 Remote Substation Control and Automation ..............................................................22 3.4 Operational Telephone System...................................................................................22 3.5 Settlement and Revenue Metering and Customer Communications ..........................24

3.5.1 Energy Metering in the Deregulated Environment.............................................24 3.5.2 Customer Metering, Advanced Metering Infrastructure.....................................24 3.5.3 Advanced distribution applications and Smart Grid...........................................25

4 OPERATION SUPPORT APPLICATIONS......................................................................26 4.1 Collaborative Multi-media Communications .............................................................26 4.2 On-line Documentation...............................................................................................26 4.3 Substation Automation Platform Management...........................................................27 4.4 Condition and Quality Monitoring Communications .................................................27 4.5 Substation data Retrieval ............................................................................................27 4.6 Mobile Workforce Communications ..........................................................................28

5 SECURITY, SAFETY AND ENVIRONMENTAL MONITORING................................29 5.1 Security of Sites and Assets........................................................................................29

5.1.1 Video-surveillance of sites..................................................................................29 5.1.2 Site Access Control.............................................................................................29 5.1.3 Environmental Hazards Monitoring (Sites and Assets)......................................30 5.1.4 Intruder Detection ...............................................................................................30

5.2 Human Safety & Operational Security .......................................................................30 5.2.1 Earth Connection Monitoring .............................................................................30 5.2.2 Isolated Worker Safety Communications ...........................................................30 5.2.3 Public Warning Communications .......................................................................31 5.2.4 Hydraulic Structure Operation and Maintenance Applications ..........................32

5.3 Cyber-Security applications communication..............................................................34 6 OPERATIONAL CONSTRAINTS AND SERVICE LEVEL AGREEMENTS ...............35

6.1 Operational Coverage and Topology..........................................................................35 6.2 Time Constraints.........................................................................................................36 6.3 Availability Constraints ..............................................................................................39 6.4 Service Survivability and Resilience ..........................................................................41

3

6.5 Service Security Constraints .......................................................................................43 6.6 Service Integrity..........................................................................................................44 6.7 Future Sustainability, Legacy Openness and Vendor Independence..........................46 6.8 Environmental Constraints..........................................................................................46 6.9 Defining Service Level Agreements ...........................................................................47

7 DISASTER RECOVERY AND SERVICE CONTINUITY..............................................54 7.1 Introduction.................................................................................................................54 7.2 Threats and Risk Management and Risk Assessment.................................................54 7.3 Business Continuity Plan and Disaster Recovery Plan...............................................56 7.4 Project Design Criteria................................................................................................57

7.4.1 Back-up Facilities ...............................................................................................57 7.4.2 Power Supply Independence...............................................................................58 7.4.3 Network Redundancy..........................................................................................58 7.4.4 Countermeasures against Natural Disasters........................................................59

7.5 Enhancing the emergency response capacity..............................................................60 7.6 Disaster Information Systems .....................................................................................62

8 TELECOM SERVICE DELIVERY MODELS..................................................................64 8.1 Introduction.................................................................................................................64 8.2 EPU Profiles - Telecom Service Users .......................................................................64

8.2.1 Coordinating or Operating Bodies without Network Assets ..............................68 8.2.2 Transmission System Operator (TSO) or Transmission Utility .........................69 8.2.3 Distribution Utility..............................................................................................70 8.2.4 Generation Utility ...............................................................................................71

8.3 Telecom Asset Ownership Profiles.............................................................................73 8.3.1 Introduction.........................................................................................................73 8.3.2 Physical layer assets............................................................................................74 8.3.3 Transport network assets.....................................................................................77 8.3.4 Application service networks and platforms.......................................................77

8.4 Telecom Service Provider Relationship to the User ................................................78 8.4.1 Integrated to the Operational User (Type A) ......................................................79 8.4.2 Sister Entity to the Operational User (Type B)...................................................80 8.4.3 Affiliated Service Company (Type C)................................................................80 8.4.4 Independent Service Contractor (Type D)..........................................................81 8.4.5 External Telecom Service Provider (Type E).....................................................81

9 FEDERATING OF SERVICES ON THE PRIVATE INFRASTRUCTURE....................84 9.1 Introduction.................................................................................................................84 9.2 Process and Organization Issues.................................................................................85 9.3 Technical Solutions.....................................................................................................87

9.3.1 Fibre Separation in Optical Cables .....................................................................87 9.3.2 Wavelength Separation through C- or D-WDM.................................................88 9.3.3 Bandwidth Separation through PDH/SDH .........................................................89 9.3.4 Virtual Network Separation (MPLS VPN or Ethernet VLAN) ..........................89

10 MANAGEMENT OF TELECOM SERVICE AND INFRASTRUCTURE..................91 10.1 Introduction - Need for a Management System..........................................................91 10.2 Present State Assessment and Target Definition ........................................................93

10.2.1 Telecom Business Maturity Modeling................................................................93

4

10.2.2 Management Process Maturity ...........................................................................94 10.3 Management Frameworks & Best Practices ...............................................................96

10.3.1 Introduction.........................................................................................................96 10.3.2 ITIL Framework..................................................................................................96 10.3.3 NGOSS Frameworx.........................................................................................97 10.3.4 Business Process Framework eTOM..................................................................98 10.3.5 Relating ITIL to eTOM Framework ...................................................................99

10.4 Towards a Utility Telecom Management Framework ..............................................101 10.4.1 Introduction.......................................................................................................101 10.4.2 Utility Telecom Management Operations Map (uTOM)..................................102

10.5 Upstream Management .............................................................................................105 10.5.1 Introduction.......................................................................................................105 10.5.2 Policy Definition & Business Planning ............................................................107 10.5.3 Strategic Deployment and Tactical Adjustments..............................................109 10.5.4 Business Development, Service Offer and Service Migrations........................112

10.6 Operational Management..........................................................................................113 10.6.1 Customer/User Relation Management ..............................................................114 10.6.2 Communication Service Management Process.................................................115 10.6.3 Network Resource & Infrastructure Management ............................................117 10.6.4 Provider/Contractor Relationship Management Process ..................................120 10.6.5 Enterprise Processes impacting Telecom Service Delivery..............................121

10.7 Management Tools and Information Systems ..........................................................123 10.7.1 Introduction.......................................................................................................123 10.7.2 Element & Network Management Systems......................................................124 10.7.3 Operation Support Systems (OSS)....................................................................125 10.7.4 Inventory & Configuration Data Base ..............................................................127

11 COST CONSIDERATIONS.........................................................................................131 12 FURTHER ACROSS THE HORIZON........................................................................135

12.1 Power System Evolution - Smart Grid .....................................................................135 12.2 EPU Organization and Environment ........................................................................137 12.3 Communication Service Provider Environment .......................................................138 12.4 Telecom Technology Evolutions ..............................................................................139 12.5 Information System Evolution - Cloud Computing..................................................141

APPENDICES ..........................................................................................................................144 A1. IP Voice in Utility Telecoms ........................................................................................144 A2. Sharing Mobile Emergency Service (TETRA).............................................................149 A3. Satellite Communications in Power Utilities................................................................151 A4. Disaster Counter-measures Learning from US 2005 Hurricanes ..............................154 A5. Survey of Electric Power Dimensioning Practice in EPU Data Centres ......................158 A6. Deploying a Management Framework Western Power .............................................161 A7. ITIL Management Framework......................................................................................165 A8. TM Forum NGOSS - Frameworx.................................................................................173 A9. List of Acronyms ..........................................................................................................182 REFERENCES .........................................................................................................................185

5

1 INTRODUCTION Some 15 years ago, a technical brochure prepared by CIGRE Study Committee 35 started with the following lines, summarizing the situation and the opportunities facing the power utilities in terms of telecommunications as seen in early 1990s: Over the last few years an increasing number of utilities have seen their telecommunication activities being influenced by technological and operational change. The choices that utilities are faced with today may have a significant impact on their future development. Traditionally, power utilities have used telecommunications networks primarily for control and operation of the power system. Standardized telecommunications equipment could not always be used in an operational environment. Leasing telecommunication services, on the other hand, was unacceptable in many cases because of low availability figures and service level from the monopolized public operator or simply because the public telecommunications network did not have sufficient geographic coverage to reach all the utilitys substations. The use of telecommunication network for other purposes was generally prohibited by legislation. As a consequence, the planning of future network developments was relatively straightforward. The networks were planned and designed to meet the utilities particular operational needs and this decided the extent of investment in new network infrastructure. The type of network infrastructure provided was also influenced by operational needs with specialized equipment such as Power Line Carrier (PLC) being used extensively. Although these criteria still play an important role in many networks today, the focus has now shifted to cover a broader range of issues: The requirement for increased capacity, speed and response time of the operational and

administrative services The need to improve the efficiency of network maintenance and quality of service The need to reduce dependence on vendor specific equipment The opportunities provided by the liberalization of the telecommunications market The opportunity to outsource all or parts of the telecommunications infrastructure or

service to third parties (Extracted from TB107 Power System Communications in the High Speed Environment, [1])

In the 15 years that separate us from these lines, if the essential issues have not changed, the situation is radically different. The creation of electricity market has created new participants and hence the need for new

communication services. A new group of services has been formed that can be called Business and Market communications. The requirement to communicate with the electricity customers and with individual producers is driving important metering infrastructure projects which may represent an opportunity to implement other utility or commercial services.

6

The great majority of electrical power utilities have implemented extensive optical fibre networks with SDH as the core technology providing the required capacity, speed and response time of their operational requirements.

Technological orientations for a packet data communication layer are no longer a subject of discussion. The omni-presence, the ubiquity and the strong industry support for the fully mature Ethernet and IP technologies make them the natural complementary layer for providing new services in the electrical utility. The multi-service capability of the IP technology as discussed extensively in CIGRE Technical Brochure TB249 [2], allows the creation of a single Integrated Service Network (ISN) to cover different operational, operation support and corporate IP requirements, or multiple networks, each dedicated to one of these families of Utility communications. At present, IP connection to the electrical substation is a strong requirement for many new applications and has been the subject of a Technical Brochure TB321 Operational Service using IP Virtual Private Networks [3] and of further ongoing work. In order to carry these IP connections and other more time-critical communications, Utilities implement new Ethernet transport over SDH, over fibre, or over an MPLS core. Wide Area Ethernet transport has been the subject of a separate publication in CIGRE [4].

Many new directions in the mode of service delivery, which were to be explored at the time, are today sufficiently evaluated, to allow a new analysis in the light of more than a decade of experience. Many pioneers of seeking commercial revenue from the operational network infrastructure have evolved into standard telecom Service Providers, moving away from their original goal. Those who have maintained their original scope have survived due to particular legal, legislative or practical contexts which are interesting to explore.

New problems and operational issues have appeared due to outsourcing or due to the provision of commercial services in the liberalized telecommunication market. A previous CIGRE publication TB108 Business Opportunities for Utilities in the Telecom Market [5] published in 1997 needs to be reviewed in the light of these new issues and problems. The effects of commercial service on the operational service provision are to be analyzed.

Finally, moving from the monopolized Public Telecom Operator to many concurrent Telecom Service Providers have changed radically the cost/performance and quality of service objectives for the procured services. There has been a clear change of orientation from a uniform quality objective towards a competitive, avoid non-contractual performance to reduce cost strategy. The relationship between the Service User and the Service Provider and the principle of Service Level Agreement between the two is of great interest and needs to be covered.

At present, Electrical Power Utilities (EPU) are increasingly dependent upon the existence of fast, secure and reliable communications services. These services interconnect the participants, platforms and devices constituting the technical, commercial, and corporate processes of the Utility across the different sites. The communication services are provisioned, managed and maintained in different ways depending upon different quality constraints, cost and regulatory imperatives and company policy considerations. The services can be integrated together into a

7

common network or provided through completely separate networks. The associated telecommunication organization of the Utility varies correspondingly among power utilities. Moreover, adopting a particular telecom service provisioning model is not an irrevocable decision. It is often re-examined and reviewed in the light of new situations, some of which are as follows:

New company policy and orientation, Regulatory issues and requirements Mergers and dislocation of activities, Availability or loss of adequate telecom services to be procured, Requirement for new services or change of scale in the existing services, incompatible

with the present provisioning model Lack of satisfaction from the services obtained through the existing provisioning model, Major capital investments and running costs required for refurbishment and extension of

existing facilities,

Technological changes in telecommunications and in power system technology Lack of qualified staff and the ageing of the concerned technical work-force This present document is not about telecommunication technology, which is extensively covered in other CIGRE published literature and on-going work It focuses on the analysis and provides a new look into the delivery of communication services associated with operational applications of the EPU. It covers quality requirements, architectural aspects, as well as the related organizational, and management issues across different types of EPU.

Management and maintenance of telecom network infrastructure and services are also covered from a Utility process and organization point of view rather than from a technological view which is again extensively covered in the literature and subject to very fast evolution. This Technical Brochure is mainly addressed to: EPU decision makers who assess telecommunication service provisioning strategies in

view of the changing nature and scale of requirements. EPU Telecom service providing entities who need to adapt to those same changing

requirements

Telecom Service Provider offspring of EPUs who have over time forgotten the service imperatives of their EPU operational customers

8

2 COMMUNICATION SERVICE IN THE POWER UTILITY

2.1 Introduction The term service is widely used often with a very loose definition and may lead to confusion and misunderstanding. We shall therefore start with some definitions that shall be used throughout the document. These definitions are illustrated in figure 2.1. Service is the perception of a User from a process implemented by a Provider. A Service Provider deploys a telecom infrastructure and corresponding management processes in order to offer Communication Services satisfying the requirements of its user community. The user perceives the service as a network cloud providing communication connectivity for its user applications and processes. ITU-T E800 defines the service as a set of functions offered to the user by an organization [6]. A communication service is delivered at a Service Access Point with a certain Quality of Service (QoS) as stipulated through a Service Level Agreement (SLA) between the Service Provider and the Service User. A service level agreement can be formally stipulated or implicit between the provider and the user. The process of assuring that the terms of the SLA are met is called Service Management. It relies upon a proper Infrastructure Management and Maintenance performed by the Service Provider.

DedicatedTelecom

Infrastructure

Procured Service

ApplicationPlatform

Communication Service

Infrastructure Management & Maintenance

Service Management

ApplicationPlatform

Service Access Point

User Application

UserUser

DedicatedTelecom

Infrastructure

Procured Service

ApplicationPlatform

Communication Service

Infrastructure Management & Maintenance

Service Management

ApplicationPlatform

Service Access Point

User Application

UserUser

Figure 2.1 Communication Service Model A Public Telecom Operators mission is to provide telecom services to its customers. Telecom service is in this case the end product and the final commodity for which the whole organization is working. The Service User to Service Provider relationship is therefore straightforward as shown in figure 2.2. The Telecom Operator designs and commercializes a catalogue of standard telecom services based on a market survey and from thereon has no particular concern regarding the customer applications employing these services other than continually adapting the catalogue to the markets evolution.

9

An Electrical Power Utility (EPU), on the other side, provisions telecom services essentially for its own requirements. The provision process may be multi-layer, employing dedicated infrastructures or procured services at different levels and presenting multiple User-Provider relationships as illustrated in figure 1. In this case, the catalogue of services must be based on detailed analysis and characterization of EPU applications in terms of communication requirements.

Public Telecom Operator

Service Provider

Customers Service User

Transmission Medium Fibre Connectivity

Telecom Connectivity

Service Platform & Service Network

Process User

Electrical Power UtilityUser / Provider

Relationship

Public TelecomUser / Provider

Relationship

Provider

User

Provider

User

Provider

User

Public Telecom Operator

Service Provider

Customers Service User

Transmission Medium Fibre Connectivity

Telecom Connectivity

Service Platform & Service Network

Process User

Electrical Power UtilityUser / Provider

Relationship

Public TelecomUser / Provider

Relationship

Provider

User

Provider

User

Provider

User

Figure 2.2 User-Provider Relationships

2.2 EPU Communication Services Communication services in the EPU can be identified according to the applications that they address. In particular, wherever the Service User entities and the telecom service providing entity are tightly related, there is a one-to-one correspondence between applications and communication service, resulting in an application-oriented definition of communication services (e.g. SCADA or Protection communication services mean communication services respecting the requirements of SCADA or Protection applications). The communication Service Provider is assumed to be sufficiently familiar with the applications to apply the necessary precautions in the delivery of the required service (i.e. implicit SLA). Whenever a new application is introduced or the requirements of an application change, the user and provider must seek a new common understanding of the service requirements. On the other hand, where communication service is provided by an external or formally separate entity, then the service provision contract (explicit SLA) defines the service attributes according to the providers service catalogue (e.g. Platinum, Gold, Silver, etc.). The Utility user must then decide upon the suitable SLA for his applications. In this report, we have identified communication services by the applications that they serve. Consequently, we define communicating applications related to the operation of the power system in chapters 3, 4 and 5, then the constraints and the required qualities in chapter 6,

10

before relating applications and constraints in section 6.9 (figures 6.6 and 6.7) allowing SLA specification whichever provisioning scheme is adopted (as characterized in chapter 8). The following application-oriented service categories can be identified:

1. Operational Services These communication services enable the coordination and exchange of information between the staff, devices and platforms directly involved in operational applications and processes used to operate, control and protect the power system and its constituents. The processes are necessary for the proper accomplishment of the Utilitys primary mission and therefore their communication services are referred to as mission-critical.

2. Operation Support Services Closely related to the Power system Operation, there exists increasingly a group of applications related to the maintenance and support of the Power System infrastructure. This includes voice and data applications for the field maintenance staff connecting them with central offices, servers and data sources allowing them to perform their tasks, as well as remote monitoring and surveillance applications. These are collectively referred to as Operation Support Applications and their telecom requirements as Operation Support Communication Services.

3. Security, Safety and Environmental Services A whole group of new applications related to the security of utility staff, public safety, Utility site security, and environmental security is emerging in many countries due to growing security and environmental concerns and consequent regulatory constraints. These applications which were previously considered as part of the operation support are increasingly considered as a distinct class of applications with extensive communication requirements and constraints.

4. Corporate Communication Services These communication services are related to the administrative applications of the Power Utility as a Corporate Enterprise, covering the administration and corporate needs of the Utility organization and its employees (including those located in Operational sites).

5. Business and Market Communication Services The Power Utility needs to exchange information with its external Market partners and its power customers. This includes communications between power generators, distribution companies, national and other country TSO, trading platforms and energy consumers. The required communication services are referred to as Business and Market Communication Services. Smart metering and Demand-side Management communications are part of this class of communication services.

6. Commercial or U-Telco Communication Services The Power Utility or an affiliated entity may provide commercial communication services as a source of revenue to other Utilities, to institutional customers (e.g. government or community offices), to telecom Service Providers, or to multi-site companies (and in certain cases to individual customers). The service may cover subscriber premises access (DSL), core communications, or both. Providing U-Telco communication services can be assimilated to the service provision model of a public telecom operator.

Criticality of communication services in the Power Utility can be assessed through the consequences of service loss and degradation. It is clear that a high degree of criticality can be

11

attributed to the operational services. However, it should be noted that the operational applications are not the only critical processes in the Power Utility. Security and human safety related applications present also a high level of criticality. The same can be said about communications related to Utility Business and Market activities where the financial consequences of a loss of communication can be tremendous. Corporate communications may be tolerant to longer periods of programmed unavailability, in particular for maintenance purposes. The loss of commercial communications has the same degree of criticality as that of other public telecom operators resulting immediately in a loss of revenues, the non-respect of contractual obligations if prolonged, and in the long run in a loss of customers. The performance objectives and the Quality of Service are also different among these different service types. Many operational services, such as Protection Relay applications, have extremely severe time delay and communication integrity constraints, whereas the other communication service types are mainly transactional with less severe time sensitivity. On the other hand, business and market communication services implicate access beyond the perimeter of the power company and may raise more severe security issues. The first three groups of services described above may collectively be called operation-related services (serving operation-related EPU applications). The present brochure focuses on these services even if other services are often mentioned, in particular when their provision interferes with (or influences) the way in which operation-related services are provisioned.

Operations-Related Communications

Security transactionsinternet-oriented

Service-Provider oriented

Enterprise NetworkIT-oriented

Electrical Power Utility Telecommunications

Operational Communication Services

Operation SupportCommunication Services

Administrative/CorporateCommunication Services

Business / MarketCommunication Services

Commercial / U-TelcoCommunication Services

Security & SafetyCommunication Services

Industrial Communications

Operations-Related Communications

Security transactionsinternet-oriented

Service-Provider oriented

Enterprise NetworkIT-oriented

Electrical Power Utility Telecommunications

Operational Communication Services

Operation SupportCommunication Services

Administrative/CorporateCommunication Services

Business / MarketCommunication Services

Commercial / U-TelcoCommunication Services

Security & SafetyCommunication Services

Industrial Communications

Figure 2.3 Power Utility Telecommunications

Considering the organizational diversity of EPUs and their different sizes, activities, and regulatory constraints, the exact perimeter of each category and the allocation of individual

12

EPU applications to these categories can vary to some extent and may evolve with organizational changes. Some of the factors that influence these allocations are as follows:

Security policy The definition of separate security domains in the EPU and the consequent allocation of applications to these different security domains can result in the communication service allocation. This means that the applications which are part of a same security domain shall exclusively use a same group of communication services.

Organization The organizational entity in charge of a group of applications may require exclusive usage of a service or a same group of communication services.

Company strategy Grouping of communication services may depend upon the companys strategy, for example to merge corporate and operation-related IT and telecoms, or to merge corporate and market related applications communications provision, etc.

Regulatory issues Regulation authorities may prevent operational applications to share communication services with non-operational, or may impose full separation of the U-Telco activities.

The grouping of different applications communication services strongly impacts the service integration strategy of the company as described in Chapter 9 on Federating of Services.

13

3 OPERATIONAL APPLICATIONS

3.1 Protection Communication Power system faults disrupt normal power flow by diverting current through a short-circuited connection and collapsing power system voltage. Power system fault clearing requirements are very important design and operational criteria for power systems. Faults can cause damage and breakdown to power apparatus such as circuit breakers,

transformers and cables. The repair work or full replacement in case of destruction is very costly and may take considerable time.

Faults can also cause severe operational disturbances resulting in collapse of power delivery and blackout for regions, and, in severe cases, even for several countries. Heavy reliance of modern society on electric power consuming devices for business activities, safety, lighting, heating, communication and many other conveniences make severe disturbances and blackouts unacceptable.

Transients due to faults in the power system can also adversely affect sources of generation and customer loads.

Faults can have also legal and financial consequences. o Manufacturer can be responsible for the consequences in case of a faulty device

(e.g. a breaker not acting correctly) Fault recorders can be used for this purpose o Customers may have to be paid for the Customer lost minutes and company

can get a penalty from the Regulation Authority Consequently, faults must be detected and isolated very quickly. Electric power system generators, transformers, Busbars, and power lines are monitored by Protective Relays designed to detect faults and operate isolating devices designed to interrupt damaging fault current.

Protection performance requirements specify the balance between the conflicting goals of dependability and security: Dependability goals require maximum sensitivity and fast response time to detect and

clear all faults quickly with very low probability of a failure to trip. Security goals require maximum selectivity and slow response time to minimize the

probability of spurious operation leading to an unwanted trip on a faultless circuit. Security is an issue during fault conditions as well as during normal, faultless conditions.

Therefore, the implementation of a Protection scheme should result in dependable operation of only those relays protecting the faulted unit, and secure non-operation of the relays during non-fault conditions and when faults occur on adjacent power system units. This balance is met only through proper protection scheme design, proper relay and equipment selection, and proper connection and setting of these relays and equipment to achieve appropriate sensitivity and coordination.

14

When protection schemes detect a fault on the equipment or line they protect, they signal (or trip) isolating devices, called circuit breakers, to open, in order to isolate the faulty segment of the system and restore normal voltage and current flow in the power system. When the protection scheme and circuit breakers operate properly, the fault is isolated within the required fault-clearing time. Protection applied on extremely high voltage systems, where fault-clearing times are most critical, typically detect faults and operate in about one to two cycles (or even less than one cycle in certain cases). Circuit breakers operate in one to three cycles. The combination of high-speed protection schemes and fast circuit breakers can interrupt a fault in about two cycles, although more common fault-clearing times range from three to six cycles. Many protection applications require the real time transfer of electrical measurements, signals and commands between electrical substations to enhance or to enable the trip/operate decision. Protection systems for substation units (generators, busbars, transformers, etc.) can

normally meet the fault clearing requirements without using telecommunication. Telecom services may be needed in this case, only to command a circuit breaker at a remote end if a local circuit breaker has been economized (Direct tripping) or exists but fails to interrupt fault-currents (Breaker Failure).

Protection schemes for HV lines generally need to exchange information with the protection device at the far end of the line to meet fault clearing requirements. Communication between the protection devices may be the basis for fault detection as in the case of a Current Differential Protection, or needed to ensure that time response and selectivity requirements are met, as in Permissive Distance Protections.

The Teleprotection function is part of the Protection system that adapts the signals and measurements from the Protection to the telecommunication channel. It may be integrated into the protective device, or the telecommunication access equipment, or it may constitute a stand-alone device. If telecommunication fails, backup protection schemes still ensure that power system faults will be cleared, but they may not be cleared within specified time requirements. Then the probability of uncontrollable power swings and partial or complete system blackout increases significantly. Protection communications between substations are at present carried through transparent dedicated telecom circuits ranging from analogue (e.g. PLC), to a sub-E1 or E1 circuits multiplexed into an SDH transmission system, a dedicated wavelength or a dedicated fibre. The communication requirements of different protection schemes have been described in detail in [7]. Their evolutions, in particular their interfacing and transport over an Ethernet connection, are currently being assessed in CIGRE JWG D2/B5-30 and shall be the subject of a separate Technical Brochure. Building additional generating stations or transmission lines is generally the other alternative to reduce the probability of fault-induced blackouts but is significantly more costly than reinforced protection schemes with adequate telecommunication services. This is the reason why Protection Relaying applications can, on their own, justify the implementation of

15

dedicated telecommunication infrastructures with particularly severe constraints in terms of the quality of communication service.

3.1.1 State Comparison Protection Schemes (Command Schemes) State comparison protection schemes use communication channels to share logical status information between protective relay schemes located at each end of a transmission line. This shared information permits high speed tripping for faults occurring on 100 percent of the protected line. The logical status information shared between the relay terminals typically relates to the direction of the fault, so the information content is very basic and generally translates into a command, requiring very little communication bandwidth. Additional information such as transfer tripping of a remote breaker (to isolate a failed breaker) and recloser blocking may also be sent to provide additional control. Even if the communication requirements for state comparison protection schemes are considerably less stringent than for Analog Comparison Protection schemes (described in the next section), the command transmission time is of great importance because the purpose for using communication is to improve the tripping speed of the scheme. Also, variations in transmission time are better tolerated in state comparison schemes than in the Analog Comparison protection schemes. Communication channel security is essential to avoid false signals that could cause incorrect tripping, and communication channel dependability is important to ensure that the proper signals are communicated during power system faults, the most critical time during which the protection schemes must perform their tasks flawlessly. Comparing the direction to the fault at one terminal with the direction to the fault at the other terminal permits each relay scheme to determine if the fault is within the protected line section, requiring the scheme to trip, or external to the protected line section, requiring the scheme to block tripping. If it were possible to set relays to see all faults on their protected line section, and to ignore faults outside of their protected line section, then there would be no need for communication schemes to assist the relays. However, protection relays cannot be set to see faults only within a precise electrical distance from their line terminal. They are imprecise because of many factors, including voltage and current transformer errors, relay operating tolerance, line impedance measurement errors and calculation tolerance, and source impedance variations. The primary relay elements used to detect line faults are therefore set to see or reach either short of the remote line terminal (this is called under reaching), or to see or reach past the remote line terminal (this is called over reaching). Communication for state comparison protection schemes must therefore be designed to provide safe, reliable, secure, and fast information transfer from one relay scheme to another. The communication scheme must also be able to transmit information in both directions at the same time. The amount of information required to transfer between relay schemes depends on the relay scheme logic.

16

The terminology used to describe these state comparison protection schemes is basically defined according to the impedance zone monitored by the protection relay as presented below. CIGRE Terminology Alternate Name

Intertripping Underreach Distance Protection Direct Underreach Transfer Tripping (DUTT) Permissive Underreach Distance Protection Permissive Underreach Transfer Tripping (PUTT) Permissive Overreach Distance Protection Permissive Overreach Transfer Tripping (POTT) Accelerated Underreach Distance Protection Zone Acceleration

Deblocking (or Blocking) Overreach Distance Protection Directional Comparison Unblocking (or Blocking)

Figure 3.1 - State Comparison Protection Schemes [7]

3.1.2 Teleprotection Signalling Systems Teleprotection signaling is the function of transforming the state information transmitted by the Protection Relay (e.g. a Binary Command) into a signal suitable for transmission over a telecommunication channel and to restitute the information to the remote Protection Relay or remote Circuit Breaker in a secure and prompt manner. Teleprotection signaling is associated to the communication of State Comparison Protection schemes and all direct tripping applications. The telecommunication channel is typically an analog circuit over PLC, or a digital sub-E1 or E1 over a multiplexed digital communication system or a dedicated fibre (or wavelength). The operational performance of a Teleprotection signaling system can be defined through the following parameters: Security is the ability to prevent communication service anomalies from restituting a

Command at the remote end when no command has been issued. Security is expressed as the Probability Pucof unwanted commands (command condition set at the receiving end for a time duration longer than a specified limit). Security is related to the communication service integrity (error performance) and the Teleprotection Signaling systems error detection capability.

Transmission time is the maximum time (Tac ) for the delivery of the command at the remote end, after which it is considered as having failed to be delivered. This is a constraint to the time performance of the communication service, not only in terms of nominal value but as a guaranteed limit.

Dependability is the ability to deliver all issued commands at all times without any statistical considerations. It is expressed as the Probability Pmcof missing commands (issued commands not arriving to the remote device, arriving too late or with a time duration shorter than a specified limit). This sets a very severe constraint on the availability and error performance of the communication service, challenging such telecom service concepts as errored seconds and degraded minutes being counted in the available time of a communication service.

17

3.1.3 Analog Comparison Protection Schemes Analogue Comparison Protection is based on the transmission and comparison of electrical parameters between the ends of a protected line. The analogue values that are compared across the line are, in particular, current samples although other schemes (e.g. Phase Comparison) also exist. Current Differential Protection (longitudinal current differential) is applicable to any overhead line or underground cable at all voltage levels and is used in particular for: Very short lines and cables where the low impedance makes the adjustment of settings

difficult for the use of Distance Relay Multi-terminal lines where the intermediate in-feeds modify the impedance seen by the

Distance Relays, implicating that the observed impedance is not only dependent on the distance to the fault, but also on the in-feed from the remote terminals, making impossible an accurate measure of the impedance.

Situations where only current transformers are installed at each end of the line (no voltage transformers)

EHV transmission lines where series capacitors may create protection issues. The transfer of analog samples between the ends of the protected line can be performed in several ways, the most common, at present, being the use of digital communications. The instantaneous current values at each end of the power line are sampled, converted to digital data and transmitted towards the other terminals with a sample rate ranging from 12 to 60 samples per cycle. Although the communication interface is generally a standard ITU-T (or EIA) interface, it should be noted that the time, integrity and availability constraints for these services are far from the standard telecommunication practice. Direct optical fibre connection between protection terminals or wavelength multiplexing of the optical protection signal can also be used with an enhanced reliability where dedicated fibre or wavelength is available. Current differential Protections are particularly time-sensitive as their operation is based upon the comparison of current samples collected from a remote point with those measured locally at the same instant of time in order to detect a fault. An error in sample timing and the delay compensation mechanism, results in a differential current that increases the risk of unwanted tripping. Modern systems provide a global time stamping of samples through GPS-synchronization. However, the older generation relays, still largely deployed, use the total round-trip transfer time to calibrate the time difference between local and remote current samples, assuming that the go and return times are strictly equal. This creates a great sensitivity of the system to any time difference and therefore implicates the same routing for the two senses of communication. This is a strong constraint on the operation of the communication network and the mode of resilience employed for the communication channel.

3.1.4 System-wide Protection Schemes System-wide Protection operates in a wider area than that for power line protections. It consists of measuring units at different locations across the power system, which sample in a

18

synchronized manner different vector measurement of voltage values (Synchronized Phasors or Synchrophasors) transmitting the information to a central equipment which takes protection decisions (Wide Area Protection & Control System, WAP&C). It should be noted, however, that not all system-wide protection systems are based on synchrophasor measurements. System-wide Protection can be used to implement an Adaptive Protection Scheme: a protection philosophy which permits and seeks to make adjustments automatically in various protection functions in order to make them more attuned to prevailing system conditions. System-wide Protection can also be used to prevent power system disturbance such as overload, power swing and abnormal frequency or voltage. These schemes are called System Integrity Protection Schemes (SIPS), also known as Remedial Action Schemes (RAS) or Special Protection Systems (SPS). They consist of automated systems that protect the grid against system emergencies, minimizing the potential and extent of wide outages through automatic measures such as load shedding, generator shedding or system separation. The telecommunication requirements for these system-wide protection schemes are similar to those for Current Differential Protections described above, but moving large volumes of information across a whole sub-network rather than between the adjacent nodes at the ends of a transmission line. This implicates time-constrained and fully predictable wide area network services. The required overall operating time is less than a few hundreds of milliseconds where the protection system transmission time should be less than several tens of milliseconds, and the propagation delay across the telecommunication system at most several milliseconds.

3.2 Energy Management, SCADA and WAMS Communications Energy Management covers all functions necessary to monitor and control the operation of the power network. Control Centres need to exchange information with generating stations, substations, other Control Centres, other Utilities, power pools and non-Utility generators. The information to be exchanged comprises real time and historical power system monitoring data including control, scheduling and accounting data. The reliability of the electricity supply depends ultimately on the security and reliability of the Energy Management System and its ability to exchange information. The Control Centre and its communications need therefore to be highly secure and reliable. The power system architecture and its operational organization often include different hierarchical levels of Control Centres as well as geographically distinct Back-up facilities and Remote Operator positions implicating inter-Control Centre and remote workstation communications.

3.2.1 SCADA Communications SCADA communication consists in the periodic exchange of short data messages between a central platform in the Control Centre and the Remote Terminal Units (RTU) in the electrical substations. The messages comprise status indications, measurements, commands, set-points and synchronizing signals that must be transmitted in real-time and requiring high data integrity, accuracy, and short transfer time.

19

Power transmission and distribution networks SCADA generally differ in their requirements, cost objectives and hence suitable communication solutions. The number of outstations and their corresponding size, cost, volume of traffic, and geographical dispersion are very different in the national transmission grid and in regional distribution networks. The time constraints and the required level of availability, fault tolerance and data integrity are also different. As a consequence, transmission grid SCADA communication is often implemented through a broadband private network with point-to-point or multi-point with a small number of RTUs (typically 2 or 3) per circuit, while in distribution networks (in particular for the MV level) lower capacity solutions such as UHF Multiple Address Radio systems (MARS), spread-spectrum licence-free radio systems or procured services (GPRS, VSAT, etc.) prevail. Still today, the widest employed communication mode for the substation RTU remains the Asynchronous Serial link through an RS232 interface, polled by the central control platform. The communication protocol associated to this mode has been standardized as IEC 60870-5-101 (IEC101), although many other protocols are still in use in legacy systems. The great advantage of Serial link SCADA is its conceptual simplicity. The major drawback to serial communication for SCADA is indeed its lack of flexibility (e.g. for back-up control centre connection) and cumbersome installation in particular at the Control Centre. Packet switching has been applied to SCADA services since the late 80s, essentially to save leased aggregated bandwidth on Control Centre links and to enhance system flexibility and resilience. SCADA networks have been implemented over X25 packet switching, Frame Relay systems and ATM in certain countries. However, worldwide popularity of SCADA communications over packet networks has been due to IP communications. SCADA RTU communications are migrating to TCP/IP based protocol IEC 60870-5-104, generally called IEC104. The RTU communicates through an Ethernet LAN access interface at 10 or 100Mbps, although the bandwidth allocated to each RTU communication remains often around 10kbps. Legacy RTU may be connected through a Terminal Server encapsulating Serial data. The use of TCP/IP enhances considerably the flexibility of the SCADA communication system, facilitating the relocation of an RTU or the switch-over of RTU communications to a back-up facility. The migration process for large legacy SCADA networks from existing serial communications to TCP/IP is a major concern in many Utilities. This process may be extended over many years, and does not necessarily cover at the same time the replacement of the RTU, its communication interface, the telecommunication network and the Control Centre Front-end facilities. Moreover, new RTUs dispersed across the network may be TCP/IP while the existing may remain serial linked, up to their programmed end-of-life.

3.2.2 Inter-Control Centre Communications Communications between Control Centres is necessary for connection to back-up facilities (e.g. for database synchronization), to other Control Centres (e.g. for dispatch coordination), or to other platforms (e.g. for market management applications). The primary purpose of the Inter-Control Centre Communications is to transfer data between control systems and to initiate control actions.

20

These communications are assured through the Inter-Control Centre Protocol (ICCP) standardized as IEC 60870-6 and Telecontrol Application Service Element (TASE-2) protocol, although earlier protocols may still be in use in certain older systems. ICCP uses an underlying transport-service, normally TCP/IP over Ethernet. The required bandwidth for an ICCP link is generally around 2Mbps (E1) which can be provisioned over an SDH network, although lower capacity links (64-128 kbps or even lower) have been used where no fibre and SDH capacity is available. The time constraint for an ICCP connection is of the order of hundreds of milliseconds which rarely constitute a constraint in an IP/Ethernet network over a digital communication infrastructure. Security is the fundamental issue in implementing ICCP connections. An inadequately protected ICCP connection may form an open door to the control of the nation-wide energy network.

3.2.3 Remote Control Centre Operator Consoles Remote connection of Operator positions to the Control Centre platform also requires high speed communications. An Ethernet connection with a throughput of 2-10Mbps generally allows an adequate quality communication link for connecting these remote workstations.

3.2.4 Generation Control Signaling Automatic Generation Control (AGC) are signals that the Control Centre sends to the different generation plants in order to maintain the frequency and tie line flow and to increase or reduce their power production accordingly. Generation Control signals are either through fast regulation loop (1- 5 seconds) or slower step-up and step-down signals transmitted from the Control Centre to Generation Plants/Units through dedicated communication channels (or TCP/IP). Their availability and security is therefore essential to the proper operation of the power system.

3.2.5 Wide Area Monitoring System (PMU Communications) Wide Area Measurement and Monitoring provides a GPS-synchronized snap-shot of the power system through the acquisition of complex parameters (amplitude and phase) across the power network. It enables a better visibility of power flow across the system incorporating dispersed generation and multiple Utilities. The collected complex parameters are Bus voltages, line currents, etc. The Phasor Measurement Unit (PMU) is the acquisition device in the HV substation, collecting time-tagged phasors. Measurements are transmitted to a central platform generally through a Phasor Data Concentrator (PDC) for different applications. These different levels of wide area applications have very different requirements in terms of information exchange and consequently telecommunication service [8]. Post-incident analysis and static modelling applications are offline systems where

collected data is used to analyze the cause of an event or to adjust the behaviour model for a system. Data can be collected continuously, daily or only on request. The communication service can be a TCP/IP file transfer service with no real time constraint.

21

Visualization and Situational Awareness applications collect data from sites and display them for human operator observation. These applications which constitute the great majority of present day systems have time requirements which are those of a human operator and must additionally present a level of sample loss unperceivable by the human operator. In terms of communication service a non-acknowledge UDP/IP is an adequate solution in this case whether through a dedicated network or a public provider.

Monitoring & Decision Support systems use collected data to produce analytical information helping operators respond to grid events and to position the grid for improved security and resilience. Stability diagrams and corresponding voltage collapse margins, as well as different monitoring applications (Voltage & Frequency stability, Power Oscillations, Line Temperature, etc.) are among these applications. Monitoring and decision support applications have time constraints which are similar to power system SCADA. This is achievable through UDP over a private IP network or a Service Provider VPN through a carefully specified SLA.

Closed Loop Applications are those which incorporate collecting of data from the grid, processing, automatic recognition of a pattern, and remedial action upon the grid. The systems are used for emergency situation control and special protection applications as described earlier in section 3.1.4. Closed loop synchrophasor applications are not yet widely implemented and their critical real-time nature necessitates particular attention on time control. Furthermore the decision to act automatically upon the network in real-time means that the data set (from different locations and sample stack from each point) must be complete, that is to say almost lossless. Providing lossless data across a telecom network generally implies error recovery which is constrained by time limitations.

PMU operation is specified by IEEE C37.118 which defines phasor construction using the GPS-satellite timing signal, as well as the phasors data format. The exact data volume associated with the transmission of a data packet from a PMU varies depending on the incorporated parameters and the way each of them is coded (i.e. floating point or not, etc.) but can be assumed to be around 80 100 octets. This data volume is to be transferred across the network at a rate which is governed by the sampling frequency of the PMU. The sampling frequency is expressed as a number of (or a fraction of) AC cycles. It is often 25 (or 30) samples per second corresponding to one sample every two cycles to 100-120 samples per second corresponding to two samples every cycle (Nyquist Rate). This latter rate allows the processing of the signal corresponding to the AC fundamental wave. The required communication throughput is then somewhere in the range of 16 100 kbps for a 50Hz power system although PDC links may require few hundred kbps upto 1Mbps or more.

22

Very Critical

Non-critical

Sample loss

tolerated

Sample loss not

perceivable

Data Availability

Telecom Service

LatencyOperation time

Different types of application

Ethernet VLAN with

fast recovery

10-100msseconds or below

Closed Loop Applications Emergency Situation Control and Protection Functions Remedial Action (RAS) and Special Protection Systems

TCP/IP File Transfer

N/AOff-lineminutes

System-level and grid asset models (static & dynamic) e.g. Power plant models

UDP over private IP network or VPN with

well specified SLA

secondsminutes or below

Decision support systems and Security assessment Analytical data helping operators respond to grid events Repositioning the grid for improved security Stability diagrams and voltage collapse margins Monitoring of Voltage & Frequency stability, Power Oscillations, Line Temperature

UDP/IP on dedicated network or

public provider

secondsHuman operatorMinutes

Wide-area Visibility and Situational Awareness Display of voltage, phase power swing and line loading Help operator understand what is happening in RT

in a region or for a grid asset

Very Critical

Non-critical

Sample loss

tolerated

Sample loss not

perceivable

Data Availability

Telecom Service

LatencyOperation time

Different types of application

Ethernet VLAN with

fast recovery

10-100msseconds or below

Closed Loop Applications Emergency Situation Control and Protection Functions Remedial Action (RAS) and Special Protection Systems

TCP/IP File Transfer

N/AOff-lineminutes

System-level and grid asset models (static & dynamic) e.g. Power plant models

UDP over private IP network or VPN with

well specified SLA

secondsminutes or below

Decision support systems and Security assessment Analytical data helping operators respond to grid events Repositioning the grid for improved security Stability diagrams and voltage collapse margins Monitoring of Voltage & Frequency stability, Power Oscillations, Line Temperature

UDP/IP on dedicated network or

public provider

secondsHuman operatorMinutes

Wide-area Visibility and Situational Awareness Display of voltage, phase power swing and line loading Help operator understand what is happening in RT

in a region or for a grid asset

Figure 3.2 Wide Area Applications communication service requirements [mes]

3.3 Remote Substation Control and Automation The HV substation is evolving into a networked environment around Ethernet (IEC61850) which will rapidly become the main interfacing technology for all data exchange applications in the electrical power substation. Even if the interactions in the automation and control of the substation are at present local, the connection of the substation automation platform to other substations and/or to remote monitoring and control platforms is rapidly becoming a requirement. Many Digital Substation Control (DCS) platforms employ Scada RTU-type communication protocols such as IEC104 for data exchange with the EMS/SCADA environment.

3.4 Operational Telephone System Highly reliable and secure voice communications are required for load dispatching and for network switching operations. At control centres, generating stations and switching substations, voice facilities are needed to allow operational staff to communicate quickly and efficiently. At times of disturbance on the system, the need for operational staff to communicate can be urgent. Normally, a private, highly secure operational telephone system is needed to provide the required facilities. The voice facilities for operational use include: Direct (hotline) telephone lines from the Control Centre to all major operational sites

control rooms Switched telephone service through PBX and a closed numbering scheme Additional redundancy and operation in situations of site isolation. Interconnection with the public telephone network. Voice and data traffic. Mobile radio voice facilities for access to operational staff who visit facilities. (Mobile

workforce communications is treated in a separate section).

23

The Operational telephone service is today evolving into IP telephony and becomes increasingly an Ethernet transported data service with particular time and bandwidth requirements. Some of the specific features of operational voice service are as follows:

Access restriction Use of the operational voice service is confined to operational staff and not accessible to unauthorized users.

High availability Voice service access for the operational staff and in particular the access of the Control Centre to the network substations and generating plants is essential and must present a very high availability through adequate route resilience and equipment duplication.

Resilience/fault tolerance The voice service must remain available even in the event of network faults, node failure and route unavailability. In particular, a star-structured network in which the failure of a single node may jeopardize the system is not acceptable. Multiple homing (at least dual homing) of secondary sites and a mesh interconnection of the main nodes is generally required to achieve the required level of fault tolerance.

Transfer to Backup Control Centre In emergency situations leading to the migration of power system control to a Back-up Control Centre, the telephone network must rapidly adapt in order to transfer the telephone calls for the Control staff to the Back-up facility. This transfer must be possible even if the communication equipment in the main control centre is no longer operational (e.g. fire, flood or power breakdown).

Very rapid call connection The call establishment time must be in line with the operational emergency situations in which the voice communication may become necessary. In particular, the structure of the telephone network (number of cascaded transits) and the employed signaling scheme may greatly influence the call connection speed.

Priority functions These functions allow critical communications to be established even when all voice network resources are occupied. This can be performed through Forced Releasing of facilities which are used by less critical communications, or by reserving the usage of certain facilities (e.g. communication channels) for priority calls only. Similarly, critical calls can Beak-in into an established communication of a busy called party. Priority status can be attributed permanently to a given user line (i.e. Control Operator), or obtained dynamically through a code for a given communication.

Caller identification and Call Queuing Control centre operators need to identify automatically the source of incoming calls and to establish queues of in progress communications in order to interact with many sites, in particular at times of power system emergencies. In progress and queuing calls must be accessible and transferable between different Control centre operator positions.

Mobile voice Control centre facilities and large power plants telephone systems must have the capability to connect mobile voice terminals to fixed telephone extensions. Depending on the implemented mobile radio network, these connections may require the existence of PTT (Push to Talk) facilities and associated conversion of Half-Duplex to Full-Duplex voice communications.

Ability to pre-select conference calls The voice system must present the capability to establish pre-configured conference calls, in particular between operational staff in the control centre, in multiple substations and maintenance staff.

24

Call Recording Control centre voice facilities include voice recorders which constantly record all communications of the operators which will be archived periodically. These call recordings are essential in order to establish the sequence of events and instructions given by the Control Operators in emergency situations.

Appendix 1 presents some examples of IP telephony usage in Utility Operational Voice systems.

3.5 Settlement and Revenue Metering and Customer Communications

Energy metering is the exchange of time integrated Energy Data at a commercial interface or boundary used for energy charging and billing.

3.5.1 Energy Metering in the Deregulated Environment The opening of the electricity market and exchanges between countries together with the consequent introduction of new players and roles in the power delivery system modifies the requirements regarding the metering information. The transmission grid operator performs energy metering at the HV grid access point in order to feed appropriate information with an adequate level of confidentiality to the different market participants and to enable settlement and reconciliation processes as well as invoicing of its transmission services towards the distributors. Metering data may be used for the following purposes at the Transmission Operator side: Invoice the grid access service, Calculate and invoice (or pay) imbalances, Calculate the compensation for losses on the network, Pay for system services (frequency, voltage), Check and pay offers on the Balancing Mechanism

At the customer side (Distributors, Industrial Customers, Generators): Sell or buy energy on the market Check and control the load curves (comparison with the supplier invoice), Optimize the access contract, Control the process in real time by direct access or through remote reading Make offers on the Balancing Market.

Customers with several plants connected to the network require completed metering data for each of their plants in order to get the global load profile. The data from distribution substations are used for the calculation of Distribution System Operator (DSO) losses, to set up the national load curve, and to calculate imbalances acting on the DSO network (spatial alignment and temporal reconciliation). Metering is also used by the Balance Responsible Entity in order to maintain balance on its perimeter and to check the imbalances [9].

3.5.2 Customer Metering, Advanced Metering Infrastructure Customer-related communications and revenue metering are not operational applications but represent enormous potential for telecommunication services in the power utility. They can

25

enable, in certain cases, other operational and monitoring applications in particular in the distribution network where the number of communication nodes is extremely high and the cost that can be attributed to the access for each node is very small. Distribution SCADA for the secondary network (e.g. 33/11 kV) and monitoring of MV transformers are typical examples of operational applications profiting from the deployment of advanced metering infrastructures. Advanced Metering Infrastructure (AMI) which covers the overall system composed of consumer data acquisition and collection as well as bidirectional communication with the electricity provider, is a further step from simple remote reading of customer meters. Several AMI projects in different countries are assessing new technologies for dedicated network coverage beyond the EPU sites perimeter (MV/LV PLC, meshed networking of packet radio systems, etc.) and some telecom/internet operators are working towards new service offers to occupy this promising market segment.

3.5.3 Advanced distribution applications and Smart Grid Many new applications allowing better coordination of the end energy consumer and the dispersed generation of electrical power with the overall power delivery system, under the banner of Smart Grid, require bidirectional communications between the centrally located control platform and dispersed consumer/producer premises. These applications which include demand response, selective load curtailment and dynamic and negotiated control of consumer power limitations must be served in terms of communications with a variety of telecom services whose requirements vary considerably according to the envisaged scenarios, with different impacts on the telecom service delivery mode of the EPU. These are further discussed under Chapter 12 Further across the Horizon.

26

4 OPERATION SUPPORT APPLICATIONS

4.1 Collaborative Multi-media Communications Site working process in the Power Utility is changing with the new information system and IT practices. For the execution of their site duties, site personnel and the intervening staff require expert support, remote diagnostics and reporting facilities. The following constitute some of the required services: Networked office applications (e.g. mail and calendar systems, file transfer, remote

database access, intranet), Work-order and ERP solutions (e.g. project control and time registration), Collaborative voice service, often called switched or PBX voice (increasingly

evolving into an IP-data service as a consequence of switch technology change, network change and also in the objective of cost reduction and new features),

Video conferencing facilities and electronic white-board in branch offices and control centres and PC-based video-streaming in the dispersed sites.

These applications require the secure extension of the corporate enterprise applications from the branch office to the operational sites, while remaining fully isolated from the operational applications. IT-support may be effectively administered from a corporate central site.

4.2 On-line Documentation Documentation is an essential base for efficient management of utility infrastructure. Previously, the site support staff found all necessary information to carry out their tasks at the substation either in the documentation residing at site (equipment maintenance manuals and schedules, drawings, etc.) or in their briefcase. Increasingly, an extensive amount of support information for the field intervention staff is available in centralized servers that can be accessed on-line when required. Pictures and video add particularly useful information in the dispersed environment of the power delivery system. These applications require a broadband network in order to meet an acceptable time performance. The introduction of inexpensive GPS equipment and commercial mapping applications makes Geographical Information Systems (GIS) an important tool for field based maintenance personnel. Connecting to maintenance applications in the substations and downloading accurate maps, pictures and work orders may effectively economize time. However, the use of GIS and increasingly automation of data acquisition of power line infrastructure (e.g. laser scanning) lead to heavily growing data volumes and need for scalable ICT infrastructure. On-line documentation is a well identified requirement and an existing facility in certain Utilities. The use of networking is to be coordinated through the Security policy of the Power Utility.

27

4.3 Substation Automation Platform Management The monitoring and configuration management of the different components of the substation automation system (protection relays, feeder bay controllers, etc.) is still often performed locally in the substation and through the substation controller. It can be assumed that network-wide communications shall be required to perform platform management tasks remotely either through native TCP/IP or through encapsulation and terminal servers. In this case, a TCP/IP traffic flow resulting from the supervision data relating to the operational performance, health and condition of the SAS, as well as File transfers due to configuration data and parameter settings can be assumed.

4.4 Condition and Quality Monitoring Communications Primary assets of the power system (circuit breaker, power transformer, etc.) generate condition monitoring data through their associated electronic intelligence which needs to be collected for maintenance requirements, and for determining duty cycle, capability and loading ability. Asset condition monitoring enables the safe and efficient use of network components to their real end of life, at full efficiency, without disruption of service due to asset failure, environmental risks, or unnecessary preventive replacement. Secondary assets of the electrical substation related to the measurement, protection and control as well as their respective power supply (including that of the telecom equipment) also need to be monitored through a remote platform. Condition monitoring in the substation generates a large volume of non-real time data to be transferred continuously to one or multiple remote platforms, hence creating the necessity for a monitoring network across the telecommunication infrastructure. The associated architecture is indeed utility-dependent but can be implemented conveniently through web-service, with servers residing in the substation or at some other location. Environment monitoring applications in the power system are performed for two different purposes:

Protect substation assets and premises (temperature monitoring, substation fire detection, etc.)

Protect the environment from industrial risks and hazards related to the substation assets, e.g. chemical pollution detection, etc. These are part of the security and safety applications as described in section 5.1.3.

Energy Quality Monitoring applications transfer data related to electrical power parameters at commercial interfaces or boundaries where energy is transferred between different power actors and may be sanctioned through financial penalties according to contractual quality of service criteria.

4.5 Substation data Retrieval Another group of data exchange applications in the power utility operational environment concerns the non-real time transfer of power process data captured in the electrical substation to data analysis platforms and engineering staff for the evaluation of events and confirmation of device configurations. Substation process data retrieval is typically a web-service

28

application, making formatted data available when required. Data to be transferred is as follows: Event Reports typically log files and reports generated by an event recorder or historical

system which provides information on the change of state of operational equipment Oscillography Files typically event triggered fault records generated by a protection

device or fault recorder. These may contain digital events and analogue waveforms. Device Parameters and Settings data files uploaded to provide information on the

actual configuration of a device.

4.6 Mobile Workforce Communications EPUs make extensive use of mobile communications in the management and support of their infrastructure. In addition to traditional voice services for field-based operational workforce, the evolution of working practices is leading increasingly to mobile data networking applications connecting the maintenance staff to their support base, on-line documentation, and workshop applications such as spare parts database. In particular, systems can largely benefit from dedicated mobile data applications [10]. The operational mobile terminal units are evolving towards Smartphones and Tablet Computers for which ruggedized field-proof versions appear on the market. A number of projects for advanced mobile workforce communications have been reported. A particularly interesting one in Japan [11] is dedicated to disaster recovery. Another experimental project has developed a wearable terminal that keeps field workers in continuous contact with support personnel and enables the real-time transmission of camera images and audio from the work site to the support base which can then provide precise support through voice interaction using a headset [12]. These permanent contact systems can also provide checklists and templates for the elaboration of on-line real-time reporting and hence improve work safety, accident prevention and work efficiency. The implementation of data-rich mobile communication systems requires indeed the existence of high availability, disaster-resistant, high throughput wireless connectivity. More than other communication services in the Utility, the provision model for mobile workforce communications is often under assessment. There is no doubt that the most economical solution is to use public mobile services which provide a high level of geographical coverage through extensive deployment of Base stations and related infrastructure, as well as continuous roll-out of new data services and applications. The EPU pays for its communications, not for the infrastructure providing coverage. However, an essential application of the mobile system, its usage by maintenance teams during power outages and in disaster recovery situations, is severely constrained due to public base stations insufficient power autonomy and severely degraded service accessibility/performance when the network is widely solicited (e.g. during a disaster situation). Deploying a security grade private mobile radio system (e.g. TETRA) with a high coverage is costly and the roll-out of new data services and applications cannot be performed in pace with their public counterpart. This may lead utility staff to use public mobiles even if the company is equipped with its own private mobile facilities [10]. Many Electrical Utilities, in particular those in the distribution sector, own or share with other critical users some type of private security grade mobile network, although the issue remains largely a subject of internal discussion. Appendix A2 describes one such experience in Portugal where a shared TETRA service is used.

29

5 SECURITY, SAFETY AND ENVIRONMENTAL MONITORING Security and safety applications in the Power Utility are not strictly speaking operational because they do not consist