NERC | Report Title | Report Date I

Risk Assessment Considerations During NERC Review of Exception Requests

February 2016

NERC | Risk Assessment Considerations During NERC Review of Exception Requests | February 2016 ii

Table of Contents

Preface ....................................................................................................................................................................... iii

Introduction ............................................................................................................................................................... iv

Objective of NRP .........................................................................................................................................................1

NRP Technical Review and Analysis ...........................................................................................................................2

Considerations for Risk Assessment ...........................................................................................................................3

General Considerations for Evaluation of Risk ...........................................................................................................6

NERC | Risk Assessment Considerations During NERC Review of Exception Requests | February 2016 iii

Preface The North American Electric Reliability Corporation (NERC) is a not-for-profit international regulatory authority whose mission is to assure the reliability of the bulk power system (BPS) in North America. NERC develops and enforces Reliability Standards; annually assesses seasonal and long-term reliability; monitors the BPS through system awareness; and educates, trains, and certifies industry personnel. NERC’s area of responsibility spans the continental United States, Canada, and the northern portion of Baja California, Mexico. NERC is the electric reliability organization (ERO) for North America, subject to oversight by the Federal Energy Regulatory Commission (FERC) and governmental authorities in Canada. NERC’s jurisdiction includes users, owners, and operators of the BPS, which serves more than 334 million people. The North American BPS is divided into several assessment areas within the eight Regional Entity (RE) boundaries, as shown in the map and corresponding table below.

The Regional boundaries in this map are approximate. The highlighted area between SPP and SERC denotes overlap as some load-serving entities participate in one Region while associated transmission owners/operators participate in another.

FRCC Florida Reliability Coordinating Council

MRO Midwest Reliability Organization

NPCC Northeast Power Coordinating Council RF ReliabilityFirst

SERC SERC Reliability Corporation

SPP RE Southwest Power Pool Regional Entity Texas RE Texas Reliability Entity

WECC Western Electricity Coordinating Council

NERC | Risk Assessment Considerations During NERC Review of Exception Requests | February 2016 iv

Introduction This document sets forth the considerations used by the NERC Review Panel (NRP)1 during review and risk assessment of a BES Exception Request record. This document is intended to identify current approaches and provide guidance to NERC and Regional Entities. The BES Exception Request Evaluation Guideline2 provides additional guidance to the Regional Entities and NERC on considerations and factors to be used in evaluating Exception Requests for the inclusion and exclusion of Elements. The Detailed Information Form contains a common set of data and information that entities must submit with every Exception Request. The Federal Energy Regulatory Commission (“FERC” or “Commission”) upheld this approach, in Order No. 773, as an appropriate alternative to the development of detailed criteria. Information that an applicant may submit is not limited to the Detailed Information Form. Rather, an applicant is expected to submit all relevant data, studies and other information that support the Exception Request. The Regional Entity and NERC may ask an applicant to provide additional data and studies beyond the Detailed Information Form.3 The NRP’s review of risk is completed in accordance with the NERC Rules of Procedure Appendix 5C.4 NERC evaluates each Exception Request on a case-by-case basis. To complete its review, the NRP examines the Exception Request and supporting documentation, as well as the Regional Entity Recommendation, Regional Technical Review Panel (if any) opinion and any other submissions, responses or information associated with each Exception Request to determine whether the Element or set of Elements are “necessary for the Reliable Operation of the interconnected bulk-power transmission system” as provided in the NERC Rules of Procedure at Appendix 5C.5 The outcome of a BES Exception Process review is a determination that a given Element is, or is not, a BES Element. As stated by FERC, in Order No. 773, “the Commission agrees that the Registry Criteria allows the Regional Entities and NERC to consider other factors regarding entity registration which may result in cases where the bulk electric system status and registry status differs for certain equipment owners and operators.”6 Some matters may best be addressed through the registration program rather than through the BES Exception Process. A prospective Submitting Entity should discuss the approach with its respective Regional Entity prior to submitting a request through the registration program or the BES Exception Process.

1 As noted below, the NRP conducts an independent assessment of the facts and circumstances presented and may perform its own studies, tests and analyses in reaching a proposed decision. 2 http://www.nerc.com/pa/RAPA/BES%20DL/BES%20Exception%20Evaluation%20Guideline%202-4-14%20REMG%20App.pdf 3 Revisions to Electric Reliability Organization Definition of Bulk Electric System and Rules of Procedures, Order No. 773, 141 FERC ¶ 61,236

(2012); order on reh’g, Order No. 773-A, 143 FERC ¶ 61,053 (2013), order on reh’g and clarification, 144 FERC ¶ 61,174 (2013); order denying reh’g, 146 FERC ¶ 61,070 (2014). On June 13, 2013, the Commission granted NERC’s request for extension of time and extended the effective date for the revised definition of BES and the Rules of Procedure exception process to July 1, 2014. Revisions to Electric Reliability Organization Definition of Bulk Electric System and Rules of Procedure, 143 FERC ¶ 61,231, at P 13 (2013).

Order No. 773 at PP 23 and 301 (“We decline to adopt Almeda’s suggestion that the Commission direct NERC to develop appropriate technical exception criteria. We accept NERC’s conclusion that it was more feasible to develop a common set of data and information that could be used by the Regional Entities and NERC to evaluate Exception Requests than to develop the detailed criteria. NERC’s proposal provides the needed flexibility to allow Regional Entities to make a recommendation of whether or not an element is necessary for the Reliable Operation of the interconnected transmission network. Thus, the detailed criteria that NERC requires, plus other information that an entity is free to include in its submission will provide applicants a reasonable basis for determining whether an element is necessary for the Reliable Operation of the interconnected transmission network. We also decline to direct NERC to determine how to treat exceptions for proposed facilities.”). See also Order No. 773 at PP 251-262.

4 http://www.nerc.com/FilingsOrders/us/RuleOfProcedureDL/Appendix_5C_ProcForReqAndRecExFromAppOfNERCDefBES_20140701.pdf 5 While this document sets forth NRP considerations, this document does not replace review of the NERC Rules of Procedure, and to the

extent that any statement herein is interpreted to conflict with the NERC Rules of Procedure, the NERC Rules of Procedure control. In addition, the NRP review is separate from review regarding entity registration.

6 Order No. 773 at P 94.

NERC | Risk Assessment Considerations During NERC Review of Exception Requests | February 2016 1

Objective of NRP The NRP risk assessment considers whether an Element or set of Elements is necessary for the Reliable Operation of the interconnected bulk-power transmission system, defined as:

Reliable Operation: Operating the Elements of the bulk-power system (BPS) within equipment and electric system thermal, voltage, and stability limits so that instability, uncontrolled separation, or cascading failures of such system will not occur as a result of a sudden disturbance, including a cybersecurity incident, or unanticipated failure of system Elements.

According to Section 8.0 of Appendix 5C of the NERC Rules of Procedure, the NRP’s decision is based on its “independent consideration of the full record.” Exception Requests must comply with the requirements under Appendix 5C, including, for example, that “[a] separate Exception Request shall be submitted for each Element or set of connected Elements for which the Submitting Entity seeks an Exception. The scope of an Exception Request shall cover the terminal connections of the Element or set of Elements as identified in the Exception Request.” (See Appendix 5C, at Section 4.2). Regional Entity and NRP risk review depends on the Submitting Entity clear identification of (i) specific Elements under the Exception Request and (ii) “terminal connections,” also sometimes referred to as “points of demarcation,” with the Bulk Electric System (“BES”). Therefore, the NRP performs a risk assessment of: 1) the Submitting Entity’s technical studies7 and supporting material, 2) the Regional Entity’s Recommendation, including technical studies, assessments, and related supporting material, and 3) an independent technical analysis of the Elements at issue.8 The NRP may ask for additional information, face-to-face meetings, or conference calls with the Regional Entity, Submitting Entity, respective Reliability Coordinator(s), and other Owners of the Element(s). A well-developed and complete Exception Request and record during Regional Entity substantive review of the Exception Request facilitates the NRP’s review process and may reduce the likelihood of requests for additional information. The NRP issues a proposed decision. The final decision may adopt the proposed decision or modify the proposed decision, and may reach a different conclusion than the proposed decision as to whether the Exception Request is approved or disapproved. The final NERC decision issued by the NERC President or designee9 shall be the decision of NERC with respect to Approval or Disapproval of the Exception Request.10

7 According to the BES Exception Request Evaluation Guideline (at 7), as “a general principle, studies should model actual control settings

and configurations, model the Year One horizon, address a range of system conditions in terms of load level and generation dispatch, and include an explanation of why the system conditions and contingencies studied are sufficient to address the most limiting system conditions relevant to the requested Exception.”

8 These include, but are not limited to: load flow analyses and stability simulations, distribution factor evaluations, Regional Entity criteria and planning standard reviews and record reviews of information submitted. The NRP also considers lessons learned from event analysis. The NRP conducts any additional studies that may be needed.

9 As of January 2016, this is delegated to the Chief Reliability Officer. 10 A decision by the NERC Board of Trustees Compliance Committee shall become the final NERC decision if there is a challenge to the NERC

decision on an Exception Request, in accordance with the NERC Rules of Procedure.

NERC | Risk Assessment Considerations During NERC Review of Exception Requests | February 2016 2

NRP Technical Review and Analysis As reflected above, after Regional Entity detailed review and analysis of Elements substantively reviewed under Appendix 5C, the NRP conducts detailed review and analysis for each Element for which an entity is seeking exception from the BES Definition. Considerations and factors used to determine whether an Element is necessary for the Reliable Operation of the interconnected bulk-power transmission system are specified in the BES Exception Request Evaluation Guideline. The NRP, for example:

1. Performs a detailed evaluation and review of technical studies and supporting material developed in support of the Exception Request.

2. Conducts power flow analysis and stability simulations, as appropriate, to determine if the Elements in question are necessary for Reliability Operation.

3. Requests additional data and information from the Submitting Entity, Regional Entity, or other related entities that may be impacted by the decision, as determined necessary.

4. Conducts additional technical analysis, such as sensitivity analysis, voltage stability analysis, Special Protection Systems/Remedial Action Schemes impact assessments, etc., as needed.

NERC | Risk Assessment Considerations During NERC Review of Exception Requests | February 2016 3

Considerations for Risk Assessment As informed by the risk assessment described below, the NRP evaluates and determines if the Element is necessary for the Reliable Operation of the interconnected bulk-power transmission system. The specific facts and circumstances, including the size, nature, scope, location, and electrical proximity to other BES Elements, necessarily affect the risk determination. As a result, each Exception Request is reviewed on a case-by-case basis, giving due weight to the facts at issue. For example, in light of the complexities of the interconnected BPS, solely removing the specified Element(s) from a technical study may not reveal whether the Element is necessary for the Reliable Operation of the interconnected bulk-power transmission system. Rather, other factors may need review to help complete the record and inform NERC’s decision (e.g., Misoperation history, delayed clearing fault occurrences, common mode failures, parallel paths, distribution shift factor analysis, common right of way issues, generator output, etc.). In its decision, NERC will identify any additional factors considered in reaching such decision. For Exception Requests, there is no standard threshold or measure for review under Appendix 5C that can be defined by “bright-line” criteria. Elements that seek Exclusion Exception Requests are, by the terms of the BES definition, already part of the BES through the “bright-line” criteria. The determination regarding necessity of the Element for the Reliable Operation of the BES, therefore, is a risk assessment that takes into account the totality of information and data gathered as part of the Exception Request, independent technical analysis and the independent judgment and expertise of the Regional Entity, the Technical Review Panel (if any) and the NRP. Each Exception Request is evaluated on its own merits, taking into account the individual facts and circumstances at issue. Aggregate impact(s) are one of the factors which may be evaluated in rendering decisions on Exception Requests. To the extent that NERC needs to take into account the aggregate impact on the BES when approving or denying an Exception Request, NERC will seek input from the respective Reliability Coordinator and the Regional Entity. Aggregate impact analysis will be tailored based on the particular facts and circumstances to avoid potentially overly broad analysis. In the event that aggregate impacts should be considered, for any type of BES Exception Request analysis, whether transmission or generation, NERC typically identifies:

1. aggregate class and characteristics (e.g., identification of Elements with a similar size and purpose)

2. limits or parameters that ensure the class is not overly inclusive (e.g., as relevant and appropriate in a particular case, electrical and geographic area, parallel flows, etc.), and includes only Elements that could reasonably have an impact that combines with the impact of the Element at issue

a. e.g., determining which generation would qualify as part of aggregate generation in a particular study may be based on electrical distance and/or generation residing within the load pocket or zone of the study area of the requesting entity’s Exclusion Exception Request11

b. Extreme events will generally not define the limits or parameters, but they may help inform establishment of such limits and parameters of the class

3. Elements that comprise the class

4. planning and modeling issues, as applicable

11 This aggregate analysis is different than evaluation of application of BES Definition dispersed generation resource threshold criteria. For

example, in applying the BES Definition Inclusion I4, NERC evaluates dispersed power producing resources that aggregate to a total capacity greater than 75 MVA (gross nameplate rating), and that are connected through a system designed primarily for delivering such capacity to a common point of connection at a voltage of 100 kV or more. All generation, without regard to ownership, that has such a common point of connection configuration is considered in calculating whether the threshold criteria is met. Aggregate does not mean, however, that if a particular generator is seeking an Exception Request that the analysis would consider every generator at or above the gross nameplate rating having a simultaneous outage or all such entities being granted an exception from application of the BES Definition, unless otherwise warranted by the particular facts and circumstances at issue.

Considerations for Risk Assessment

NERC | Risk Assessment Considerations During NERC Review of Exception Requests | February 2016 4

5. other rationale pertinent to the evaluation

The NRP impact analysis evaluates the potential risk to the BES caused by granting an Exception Request, and may include analysis of aggregate effects. Prior to undertaking an aggregate impact analysis as part of its risk analysis, the NRP would first consider whether the relevant area is sensitive to such aggregate impacts (e.g., such as aggregate impacts of generation in an area where such generation is integral to system reliability). For example, with respect to a generation-related Exception Request, the NRP may then perform the applicable studies and analysis for both individual and aggregate circumstances where applicable, including but not limited to:

• Import capability study

Evaluate if the particular generator is needed and relied upon by system operators and/or the Reliability Coordinator.

Evaluate reactive support or BES voltage control under heavy summer or winter peak conditions, as well as light load conditions when reactors are used to reduce voltage. Is there sufficient import capability in the load pocket or zone due to unavailability of the generation if the Exception Request is approved? The Reliability Coordinator will need to provide its insights to respond to this question.

Conduct a transmission adequacy assessment for thermal loading.

• Historical data from the Transmission Operator and/or Reliability Coordinator

Consider event analysis information.

• Voltage magnitude and voltage drop analyses

• Frequency response test

• Stability analyses

• Generator shift factor analysis

Evaluate a scenario, with and without the generation and the associated flow change on the BES, and determine the total cumulative generation in the area needed for the Reliable Operation of the BES.

• Load shedding requirements due to system contingencies if the Exception Request is granted The outcome of a BES Exception Request decision is a determination as to whether a given Element is a BES Element or not. As appropriate and applicable, NERC may also evaluate whether an alternative approach to a particular situation is warranted, such as options in the registration program. Some entities, as applicable, may also decide to pursue a local distribution determination by FERC. In the event that a Submitting Entity also seeks a local distribution determination by FERC for a given Element or set of Elements that also are included an Exception Request, NERC and the Regional Entities will hold the Exception Request in abeyance until the FERC order is issued. In addition, the NRP applies existing risk assessment processes already in place and used by the ERO Enterprise. The following resources are non-exclusive guidelines and reference materials that generally help inform the NRP’s review of an Exception Request, in identifying and conducting and evaluating the record information and results of studies, analyses and consideration of issues presented to the extent they pertain to risk assessments of Elements in the BES:

Considerations for Risk Assessment

NERC | Risk Assessment Considerations During NERC Review of Exception Requests | February 2016 5

• ERO Self-Report User Guide: Provides guidelines to help Registered Entities assess the risk to the reliability of the BPS posed by noncompliance with a Reliability Standard. The purpose is not to establish a rigid set of criteria; rather the goal is to define certain principles to use when assessing risk associated with a particular instance of noncompliance. Depending on a Registered Entity’s size and organizational structure, the nature and complexity of the risk due to similar instances of noncompliance could be different. These guidelines will assist Registered Entities across the different Regional Entity footprints to perform a thorough and complete, consistent assessment of risk resulting from an instance of noncompliance.

• ERO-Enterprise Inherent Risk Assessment Guide: Describes the process Compliance Enforcement Authorities (CEAs) use to assess inherent risk of registered entities and serves as a common approach for the NERC and the Regional Entities for implementing and performing an Inherent Risk Assessment.

• Risk-Based Registration Phase 1 – Enhanced Design Framework and Implementation Plan: Provides guidance for the NERC-led Review Panel on Materiality considerations in the registration process.

NERC | Risk Assessment Considerations During NERC Review of Exception Requests | February 2016 6

General Considerations for Evaluation of Risk Depending on applicability to the Element in question, the NRP may consider the following non-exhaustive considerations towards reaching its determination. No single factor or combination of factors will necessarily be determinative in any given case. Provided below are non-exclusive factors considered by NERC in evaluating an Exception Request: Generation and Dispersed Power-Producing Resources

• Does it provide reactive support or BES voltage control?

Is a voltage magnitude and drop test necessary?

• Does it provide frequency response?

• Is it included in a restoration plan as a Blackstart resource?

• Is it included in a Balancing Authority’s Emergency Operations Procedures?

• Does it supply off-site power to a nuclear plant?

• What is the impact of the generator shift factor, with and without the generation and the associated flow change on the BES, and does the total cumulative generation needed for the Reliable Operation of the BES?

• What is the capacity factor?

What are the historical data for the past two years and future plans?

• Is it a Capacity resource?

Without the generation, would load shedding be required following system contingencies based on transmission adequacy assessment?

• Are there aggregate impacts, such as those noted immediately below, indicating that an Exception Request for Inclusion or Exclusion is appropriate?12 Insights from the Reliability Coordinator are needed to respond to this question and its sub-parts (See the discussion above on the evaluation of aggregate impacts when performing risk assessments).

The aggregate stability impacts of generation below 100 kV could affect the review of an Inclusion Exception Request.

The aggregate effect of excluding the Elements within a portion of the BES (e.g., where all or many of a group of similarly-situated facilities would materially affect the Reliable Operation of the system during a wide-area disturbance). For example, wind generators are generally comprised of multiple

12 See Order No. 773 at P 95 (stating, “Regarding ISO New England’s assertion that generators that connect to the bulk electric system via

transmission facilities with voltages below 100 kV are needed for reliability, the Commission believes these generators can be added to the bulk electric system through the exception process, and if registration is warranted for the owners and operators of these generators, the Registry Criteria provides NERC and the Regional Entities the option of registering ‘[a]ny generator, regardless of size, that is material to the reliability of the Bulk Power System.’ Aggregate stability impacts of generation below 100 kV could fall into this category of ‘material to the reliability of the Bulk Power System.’”) (emphasis added); and id., at P 253 (adding, “We also find that NERC’s explanation, that it was not feasible to develop a single set of technical criteria that would be applicable to all Exception Requests so it developed the Detailed Information Form (discussed in detail below) to ensure that a consistent baseline of technical information is provided for NERC to make a decision on all Exception Requests, is reasonable. We find that this information, coupled with the proposed exception process, allows NERC to provide consistent determinations on Exception Requests submitted from different regions involving the same or similar facts and circumstances, and allows NERC to take into account the aggregate impact on the bulk electric system of approving or denying all the Exception Requests. Thus, we find that NERC’s proposal is clear, transparent, and uniformly applicable and is as equally efficient and effective as the Order No. 743 directive to establish an exception process for excluding facilities that are not necessary for the Reliable Operation of the interconnected transmission network.”) (emphasis added).

General Considerations for Evaluation of Risk

NERC | Risk Assessment Considerations During NERC Review of Exception Requests | February 2016 7

wind turbines and ownership interests can be shared by multiple entities. This scenario was discussed in significant detail during proceedings before the FERC for revision and modification of the BES Definition. When examining similar requests, aggregate capacity of likewise situated generation in the system must be considered, regardless of whether the total MVA for any single entity falls below the threshold amount. In other contexts, it may be necessary to evaluate the associated risk imposed to the BES by removal of the Element and all similar generation Elements (i.e., no longer in service) in the same zone or load pocket. In other contexts removal may refer to similarly situated entities that no longer are obligated to comply with Reliability Standards. NERC will make clear in its decisions the applicable context.

Substation Shunt Devices

• Does it provide reactive support or BES voltage control?

Is a voltage magnitude and drop test (for dynamic devices) necessary?

• Does it provide voltage stability?

Is an import capability test necessary?

• Is the Element installed because of power quality issues, including the following:

power factor correction;

flicker control; and/or

harmonic filtering? Circuits - Transformers

• Is the Element used to control the flow of Real Power (e.g. Phase-shifting or variable frequency transformers)?

• Is the Element part of an associated System Operating Limit (SOL) or Interconnected Reliability Operating Limit (IROL) and what is the basis for the limit (e.g., thermal, voltage stability)?

• Is the Element part of a Blackstart cranking path?

• What are the distribution factor and impacts on the BES by simulating outage of the transformer? Circuits - Lines

• Is the line part of an associated System Operating Limit (SOL) or Interconnected Reliability Operating Limit (IROL) and what is the basis for the limit (e.g., thermal, voltage stability)?

• Is the line part of a Blackstart cranking path?

• What are the distribution factor and impacts on BES by simulating the outage of the line?

• Does the line support the BES by providing parallel path flow?

• Are there additional Elements that comprise the Facility, where applicable (e.g., series capacitors, series reactors, shunt reactors)?

Networks

• What is the maximum power flow that may flow through the network with:

all lines in service; and/or

first contingency conditions.

General Considerations for Evaluation of Risk

NERC | Risk Assessment Considerations During NERC Review of Exception Requests | February 2016 8

• What are the contingency and system conditions under which the flow may occur?

• Does the network support the BES by providing parallel path flow to the BES in the event an adjacent BES facility is taken out of service for maintenance?

• What are the results of the Import capability test to serve load in the network?

What are the BES impacts of importing power from the BES to serve load in the network?

• What are the topology change effects within the network and the impacts on the BES due to the unavailability of generation or reactive devices or transmission Elements in the network?

• Do Elements in the network provide reactive support to control BES voltage?

What are the results of the voltage magnitude and drop test (for dynamic devices)?

• What are the locations of the interrupting devices on the network?

• Is there coordination of Protection Systems on the network and on the BES?

• What is the proximity to nuclear generation and associated reactive and offsite power requirements where applicable?

Radial System

• What is the gross nameplate MVA rating of generators on the radial system?

• What is the Interconnection voltage of each of the BES resources connected to the radial system?

• What is the effect of disconnecting the radial system from the BES in whole, or in part?

o For example E3 exclusion exception may have two or more point on interconnection with the BES, NERC evaluates what happens to the reliable operation of the BES if one interconnection point from the local network to BES is removed for maintenance from the local network requesting E3 exclusion.

• Do Elements in the radial system provide reactive support or BES voltage control?

What are the results of voltage magnitude and drop test (for dynamic devices)?

• What are the results of the Import capability test to serve load in the radial system and the impacts on the BES?

• What are the locations of interrupting devices on the radial system?

• Is there coordination of Protection Systems on the radial system and on the BES?

• What is the proximity to nuclear generation and associated reactive and offsite power requirements where applicable?

Protection System Assessment

• What are the results of the uncleared fault evaluation, including:

Generator loss of synchronism;

Overloads on BES facilities; and

Low voltage on critical BES facilities?

• What are the compliance obligations for NERC Reliability Standards?

General Considerations for Evaluation of Risk

NERC | Risk Assessment Considerations During NERC Review of Exception Requests | February 2016 9

Entity remains subject to NERC Reliability Standards applicable to Protection Systems needed for the Reliable Operation due to uncleared faults.

Additional Non-exclusive Risk Considerations

• Lessons Learned

Prior event analysis and lessons learned produced by NERC and the electric industry will be considered, as applicable. Events, such as the 2011 Pacific Southwest Outage and the 2015 Washington, DC disturbance, resulted in recommendations for future enhancements and reliability considerations to prevent reoccurrence and to address issues that may not be covered under NERC Reliability Standards.