Reliability Assurance Initiative

Earl Shockley, Senior Director of Compliance Operations

Sonia Mendonca, Assistant General Counsel and Director of Enforcement

Compliance Committee Meeting

November 6, 2013

RELIABILITY | ACCOUNTABILITY 2

End-State Vision – Compliance

• Compliance regime shifts to using standard, risk-based audit practices similar to other industries

• Scope of audit based on standard approach to assessing entity’s risk to reliability

• Compliance focus shifts to assess strength of management controls relative to meeting standards

• Process allows for lower risk violations to stay in compliance space

• Visibility of all violations maintained, including self-reported, to allow for trend analysis

RELIABILITY | ACCOUNTABILITY 3

End-State Vision – Enforcement

• Focus on noncompliance that poses a serious and substantial risk to the reliability of the bulk power system (BPS) NERC and the Regional Entities exercise discretion whether to initiate an

enforcement action or to address issues outside of enforcement that do not pose a serious or substantial risk

• Recognize existing processes in place and encourage registered entities to continue to self-identify, mitigate, and record noncompliance under the oversight of NERC and the Regional Entities Well-controlled entities will log self-identified issues that do not pose a

serious or substantial risk

NERC and the Regional Entities continue to maintain visibility regarding all noncompliance

RELIABILITY | ACCOUNTABILITY 4

Four Key Areas of Focus for 2013

• The auditor handbook is currently under development and is targeted for completion by the end of 2013. The training and rollout efforts will occur in 2014.

Auditor Handbook

• The prototypes and pilot programs are currently underway and will continue throughout 2013. By year-end, the results and lessons learned will be evaluated and will serve as inputs into an ERO-wide, risk-based auditing approach.

Prototypes and Pilot Programs

• Improvements to self-reporting process, including process and communication improvements, are being implemented in 2013. Other improvements will be tested in a series of pilot programs extending into 2014.

Improvements to Self-Reporting

• Enhancements to the FFT process, including a triage process will be implemented in 2013. Other improvements associated with greater exercise of enforcement discretion will be tested in a series of pilot programs extending into 2014.

FFT Enhancements

RELIABILITY | ACCOUNTABILITY 5

Timeline

QTR 2 QTR 3 QTR 4 QTR 1 QTR 2

2013

Planning Phase Develop

the end game pilot strategy

2014

2015

Early Adoption/Design

Completion

Implement “quick wins”,

complete design and file where

necessary

Implementation

Execute roll-out

across ERO.

Pilot Phase

Pilot the approach – audits, self

certs, self reports, and

“exceptions” disposition.

QTR 1 QTR 2 QTR 3 QTR 4

RELIABILITY | ACCOUNTABILITY 6

2013 2014 2015

Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2

Industry Focus

Group Activity

Pilot Evaluation and

RAI Rollout Webinar Continuous

Communication, Training

and Feedback During

Deployment

Q3 Q4

Industry Engagement Timeline

Pilot evaluation criteria finalized

Industry Readiness

Assessment Webinar

(Broad Use of

Compliance

Approach

Auditor Handbook

Informational

Webinar

Annual

Implementation Plan

Webinar

Includes risk assessment, identifying and testing of controls

Co

mp

lia

nce

E

nfo

rcem

ent

Industry Focus

Group Activity

Presentation of

Pilot Results via

workshop or webinar

RELIABILITY | ACCOUNTABILITY 7

RAI from Industry Perspective

CMEP Posted

2013 2014

• I m p r o v e d s e l f r e p o r t i n g / M R R E p r o c e s s e s

• F F T e n h a n c e m e n t s i m p l e m e n t e d

• S i m p l i f i e d A M L

• A u d i t s c o p e i n c o r p o r a t e s r i s k c r i t e r i a

2015 2016

• L e s s e r r i s k v i o l a t i o n s d o n o t r e s u l t i n e n f o r c e m e n t a c t i o n s

• R i s k b a s e d a u d i t s f o r P h a s e 1 E n t i t i e s

• A u d i t s c o p e b a s e d o n c o m m o n E R O r i s k b a s e d m e t h o d o l o g y

• E n d s t a t e e n f o r c e m e n t i m p l e m e n t a t i o n

• T r a n s i t i o n t o r i s k b a s e d a u d i t s f o r r e m a i n i n g e n t i t i e s

Q3 Q3 Q3

RELIABILITY | ACCOUNTABILITY 8

Reliability Assurance Initiative –

Compliance Track

RELIABILITY | ACCOUNTABILITY 9

Key Deliverables

• Auditor Manual and Handbook

Defined audit activities

Structured audit approach

• RAI Regional Pilots

Test approaches to risk assessment, audit scoping and controls assessment

Evaluation of pilots

• Common ERO-wide methodologies for:

Annual Planning for Compliance Activities (Annual Implementation Plan)

Entity risk assessment

Audit scoping approach

Management controls assessment

RELIABILITY | ACCOUNTABILITY 10

Auditor Handbook

Organization

• Manual and Handbook framework completed

• Format and selected sections presented at Auditor Workshop – September 18

• Target completion – December 2013

• Auditor training and rollout plan under development

RELIABILITY | ACCOUNTABILITY 11

RAI Regional Pilots

• By year-end, results and lessons learned will serve as inputs into an ERO-wide, risk-based auditing approach.

• Expectations from pilots:

Process methods for risk assessments

o Standardize risk-evaluation criteria

o Link risk to a common scoping approach

Process methods for testing of management controls

o Establish common control assessment criteria

o Define testing methodology and documentation requirements

Evaluation criteria to support best of class methods

• All methods/prototypes will be: o Evaluated to determine best of class and most reasonable to adopt

o Reviewed and set as policy at the ERO EMG level

• Policy will be anchored in Compliance Auditor Manual and Handbook, as well as the annual implementation plan

RELIABILITY | ACCOUNTABILITY 12

Pilots are designed and will be evaluated to a selected set of criteria

Pilot Development and Evaluation

RFC/AEP

MRO/ATC

END STATE

DELIVERABLES

• Entity risk

assessment

• Audit scoping

approach

• Management

controls assessment

and testing

Common

Evaluation

Criteria

SERC/PowerSouth

NPCC/NYPA

WECC/Confidential

RELIABILITY | ACCOUNTABILITY 13

RAI Regional Pilots

*Second round of pilots

Region Partner(s) Risk Assessment Audit

Scoping

Controls

Assessment

MRO ATC Established risk criteria

and entity survey input

Predetermined

standards

Detailed audit

testing

RFC AEP Established risk criteria

(via maturity model)

and entity survey input

Determined by

risk assessment

results

Detailed audit

testing

SERC PowerSouth* Established risk criteria

and entity survey input

Determined by

risk assessment

results

Detailed audit

testing

WECC Confidential Established risk criteria

and entity survey input

Determined by

risk assessment

results

Detailed audit

testing

NPCC NYPA Established risk criteria

and entity survey input

Determined by

risk assessment

results

Detailed audit

testing

RELIABILITY | ACCOUNTABILITY 14

ERO Pilot Evaluating Criteria

Evaluation Criteria Criteria Explanation

Transparency for oversight purposes

Program design elements effectiveness

Alignment to the Reliability Standards

Implementation requirements

Impact on Registered Entities

• Ability to document scope for oversight review (FERC

and NERC)

• Change management

• Defined approach demonstrates repeatability and

scalability

• Program identifies an entity’s key functional activities

• Consideration of the 11 CMEP-IP risk factor elements

• Are risks factors ranked only as they relate to specific

auditable standards and requirements?

• Are other reliability risks considered?

• Does program require additional regional resources?

• Does complexity require complex IT platforms?

• Impact on different entity levels (large, medium, small)

• Timely, reasonable implementation

RELIABILITY | ACCOUNTABILITY 15

The Differences

• Improved Bulk Electric System reliability due to enhanced focus on high reliability risks and controls that mitigate those risks.

• Increased ERO Enterprise compliance and enforcement consistency.

• Tools and scoping appropriate to the entities risk and management practices.

• Resources focused on reliability and effective controls.

• Improved Reliability Standards development and retirement as a result of informational feedback loop.

• Higher level of compliance program maturity.

RELIABILITY | ACCOUNTABILITY 16

The Differences

• Recognizing and rewarding registered entities that design and implement strong management control programs

• Process harmonization

• More comprehensive use of compliance tools

• Transition away from a comprehensive checklist-style audit with burdensome administrative redundancy to a targeted, risk approach

• Scalability of approach

• Distinctions in the application of compliance monitoring given the risk to reliability

RELIABILITY | ACCOUNTABILITY 17

Next Steps

• Continue quarterly industry forums

Compliance pilot results

Principles for entity risk assessment and assessing internal controls

• Continue to review program milestones and schedule at BOTCC

• Complete compliance design elements by Q2 2014

• Filing on internal controls approach for CIP Version 5 targeted for Q2 2014

RELIABILITY | ACCOUNTABILITY 18

Reliability Assurance Initiative –

Enforcement Track

RELIABILITY | ACCOUNTABILITY 19

Issues Identified by Industry Focus Group

• Combination of long overall processing time for minimal risk issues with lack of regular communication throughout enforcement process.

• Lack of information by registered entities on sufficiency of content and process for self-reports

• Lack of centralized information collection (particularly for multi-region registered entities)

• Remaining process inconsistencies being phased out

RELIABILITY | ACCOUNTABILITY 20

Overall Approach to Solutions

• Early identification and streamlined processing of lesser risk issues with appropriate visibility

• Increased exercise of discretion by NERC and Regional Entities

• Incentives for registered entities to develop and maintain the necessary processes to self-identify and mitigate all noncompliance

• Pilots allow testing of concepts associated with end-state vision in a controlled manner (limited scope of issues and entities during the pilot phase)

RELIABILITY | ACCOUNTABILITY 21

Key Activities in 2013

• User guide and improved communication

• Triage and off-ramp for minimal risk issues

• Multi-Region Registered Entity Process

• Improved intake and process flow

• Pilots Aggregation of minimal risk issues

Alternative path to enforcement

RELIABILITY | ACCOUNTABILITY 22

Short-term Solutions – User Guide and Improved Communication

• Additional guidance on process and content of self-report: ERO enterprise user guide for self-reports and mitigation

Point of contact at Regional Entity for additional guidance

RELIABILITY | ACCOUNTABILITY 23

Short-Term Solutions – Triage

• Early triage with off-ramp for minimal risk issues Triage executed within 60 days on average

Possible outcomes from triage:

o Enough information to support a finding of minimal risk

- Issue does not trigger an enforcement action and notification is issued to NERC and FERC (pilot), or

- Issue is processed as an FFT

o More information is required prior to determining the disposition, or

o Noncompliance needs to be enforced and processed as a SNOP or full NOP

Triage-related metrics being developed

RELIABILITY | ACCOUNTABILITY 24

Short-Term Solutions – MRRE

• Multi-Region Registered Entity Process Evaluation of current practices (e.g., assignment of lead CEA and

improved coordination among the Regional Entities involved)

Development and publication of a process that reflects the current best practices and any other necessary changes

RELIABILITY | ACCOUNTABILITY 25

Medium- to Long-Term Solutions – Process Flow

• Improved intake and process flow: Ability to log noncompliance information prior to self-report

Ability to augment information

Ability to cross reference information already provided

Ability to store streamlined record for matters that do not trigger an enforcement action

RELIABILITY | ACCOUNTABILITY 26

Medium- to Long-Term Solutions – Process Flow

RELIABILITY | ACCOUNTABILITY 27

Medium- to Long-Term Solutions – Pilots

• Pilots Aggregation of minimal risk issues

Alternative path to enforcement

• Timing First cycle – October 2013 to April 2014

RELIABILITY | ACCOUNTABILITY 28

Aggregation Pilot – Parameters

Minimal risk issues only

Selected Regional Entities and selected registered entities (see Scope, below)

Record maintained by registered entity during aggregation cycle, until system changes allow direct input of information into Regional Entity system

Format and content of record is similar to FFT spreadsheet

Periodic review of aggregated issues by Regional Entity.

o First cycle began in October 2013; First evaluation of results will be in April 2014.

Notifications will be sent to NERC and FERC at the time of review and at disposition.

RELIABILITY | ACCOUNTABILITY 29

Aggregation Pilot – Scope

Regional Entity Registered Entity Scope (Minimal Risk

issues)

FRCC Duke Energy Florida PRC-005, R2

MRO Nebraska Public Power

District

Alliant East and West

All standards

NPCC NYPA All standards

SERC Associated Electric

Corporation Inc.

All standards

Texas RE Lower Colorado River

Authority

All standards –

transmission registration

only

ReliabilityFirst AEP Standards mapped to

key process areas

assessed by RF

RELIABILITY | ACCOUNTABILITY 30

Alternative Path to Enforcement – Parameters

Minimal risk issues only (during pilot phase)

Pilot to consider initial findings from Compliance Audit pilots at MRO and SERC; as pilots progress, NERC and Regional Entities will identify a specific set of self-reported issues for inclusion in the pilot

Notifications to NERC and FERC at the time of intake and disposition

Format and content of record is similar to FFT spreadsheet (recorded in a spreadsheet during pilot phase and subsequently entered into Regional Entity portal)

Records retained by Regional Entity for review by NERC and FERC

NERC to review sample issues disposed without an enforcement action

Begin in November 2013 with findings from audit pilots

RELIABILITY | ACCOUNTABILITY 31

Next Steps

• Continue to work with industry focus group to review deliverables

• Beginning in April 2014, NERC and the Regional Entities will review the results of the pilots and develop a strategy for expanding based on results Consider whether to continue to allow aggregation and whether to

expand concept beyond initial scope or issues and entities

• NERC expects to make a filing with FERC to review the results of the pilots and seek any changes to existing rules on or before the fourth quarter of 2014 Consider the appropriate scope for implementing the alternative path

to enforcement throughout the ERO enterprise

RELIABILITY | ACCOUNTABILITY 32

Please send any questions or comments to: RAIcomments@nerc.net

CIP Version 5 Transition Program Roadmap

Matt Blizard, Director of Critical Infrastructure Protection

Compliance Committee Meeting

November 6, 2013

RELIABILITY | ACCOUNTABILITY 2

Purpose of the Transition Program

Address V3 to V5 transition issues

Provide a clear roadmap for V5

steady-state

Identify resources needed to

implement V5

Foster communication and knowledge sharing

Support all entities in a timely, effective, and

efficient transition to the Critical Infrastructure

Protection (CIP) Version 5

RELIABILITY | ACCOUNTABILITY 3

CIP V5 Transition Timeline

7/13 10/13 1/14 4/14 7/14 10/14 1/15 4/15 7/15 10/15 1/16 4/16 7/16 10/16 1/174/11/2013

6/14

Self-Correcting Language

FERC Compliance Filing

6/14

Final CIP Version 5 Transition

Guidance Published

4/13

CIP Version 4 Transition

Guidance Published

10/13 - 3/14

Transition

Implementation Study

11/13

FERC Order Approving

CIP Version 5 (TBD)

9/13

CIP Version 5 Transition

Guidance Published

4/16

CIP Version 5

Enforcement Date (TBD)

4/16 - 4/17

Steady State

4/14

Transition Implementation

Study Report Published

4/1/2017

10/13

Implementation

Study FERC Filing

10/13 - 4/16

Transition Period

RELIABILITY | ACCOUNTABILITY 4

CIP V5 Transition Program Elements

Periodic Guidance

Implementation Study

Compliance and Enforcement

Outreach and Communications

Training

RELIABILITY | ACCOUNTABILITY 5

Selection Criteria for Study Participants

• Compliance History and Performance

• Asset and System Diversity

• Entity engagement and resources availability

RELIABILITY | ACCOUNTABILITY 6

Study Participants

NERC - Regional Entities – Responsible Entities

RELIABILITY | ACCOUNTABILITY 7

Study Approach

Determine Scope and Scale

Conduct Studies

Communicate Results

Study Deliverables • Periodic webinars highlighting lessons learned • Final Study report • Input to revised Transition Guidance

RELIABILITY | ACCOUNTABILITY 8

Study – Tackling Challenging Issues

• Significant Version 5 changes from Version 3

• “Identify, assess, correct” process; self-corrected and reportable deficiencies

• And other areas identified by Study Participants

RELIABILITY | ACCOUNTABILITY 9

Some CIP V5 Implementation Challenges

Input from Study Participants - September 20, 2013, Kickoff

• Understanding the technical details of V5 requirements

• Including many new assets due to V5 bright-line criteria

• Training staff regarding the new V5 requirements

• Supporting all entities in the transition, not just Study Participants

• Committing sufficient resources to the Study

• Ensuring sufficient resources and time to implement V5

• Maintaining v3 compliance and transitioning to V5 in parallel

• Developing effective Identify, Assess, Correct language and consistent with RAI

• Ensuring consistency across all regions

RELIABILITY | ACCOUNTABILITY 10

1 RELIABILITY | ACCOUNTABILITY

Key Compliance and Enforcement Metrics and Trends

Compliance Committee Meeting November 6, 2013

2 RELIABILITY | ACCOUNTABILITY

• Measuring Enforcement Efficiencies

• Key Enforcement Activities

• Key Compliance Activities

• Third Quarter Trends for Compliance and Enforcement

Main Topics

3 RELIABILITY | ACCOUNTABILITY

Measuring Enforcement Efficiencies

Measuring Enforcement Efficiencies

ERO Caseload

Index

Violation Aging - ERO

Violation Aging by

Disposition Method

Processing of Pre-2012 Caseload

Self-identification

by Registered

Entities

4 RELIABILITY | ACCOUNTABILITY

ERO Caseload Index

ERO Caseload Index (as of September 30, 2013)

Violations in Inventory

Violations filed with FERC*

Total Number of Dismissals*

Caseload Index (Months)

1,814 1,893 639 8.6

Caseload index excludes violations that are held by appeal, a regulator, or a court. Violations in inventory are comprised of active violations that have not been filed with FERC. *From October 2012 to September 2013.

Corporate goal for 2013 caseload index is 10 months. As of January 2013, the ERO caseload index was 13 months.

5 RELIABILITY | ACCOUNTABILITY RELIABILITY | ACCOUNTABILITY

Violation Aging - Disposition Method

ERO Violation Aging by Disposition Method

For violations discovered after 1/1/2012 and filed by 9/30/2013.

FFT = 7.4 months

SNOP= 10.7 months NOP = 13.1 months

6 RELIABILITY | ACCOUNTABILITY RELIABILITY | ACCOUNTABILITY

Progress in FFT Processing

0%

25%

50%

75%

100%

2011 2012 2013

25%

44%

89%

Percent of FFTs filed within 6 months of discovery by discovery year

The percent of FFTs filed within 6 months of discovery continues to increase.

7 RELIABILITY | ACCOUNTABILITY RELIABILITY | ACCOUNTABILITY

Caseload Index and Violation Aging

ERO

Current inventory: 1,814

Months in the inventory: 12.4

Nine months caseload index: 9.4

Regional Entity

Current inventory: 1,713

Months in the inventory: 12.1

Nine months caseload index: 10

NERC

Current inventory: 101

Months in the inventory: 1.1

Nine months caseload index: 0.7

These new metrics are developed based on violations processed in the first 9 months of 2013. Future metrics will be based on 12 months data. Violations that are held by appeal, a regulator, or a court are excluded from computation of these metrics. The numbers represented above are computations based on averages and formulas.

8 RELIABILITY | ACCOUNTABILITY

Reduction of Pre-2012 Caseload

Corporate goals for 2013 include eliminating pre-2012 caseload.

Active violations pre-January 2012 excludes violations that are held by appeal, a regulator, or a court.

0

100

200

300

400

500

600

700

800

900

1000

Dec-12 Jan-13 Feb-13 Mar-13 Apr-13 May-13 Jun-13 Jul-13 Aug-13 Sep-13

997

958

906

786

664

535

498

403 373

322

Active Violations Pre-Jan 2012 Excludes on hold violations

9 RELIABILITY | ACCOUNTABILITY

Promoting Self Assessment and Identification of Noncompliance

A significant number of violations continues to be identified by registered entities internally.

29%

71%

Internal vs. External Identification From 2007 to September 30, 2013

External Internal

0%

20%

40%

60%

80%

100%

120%

2007 2008 2009 2010 2011 2012 2013

14% 27%

40% 33% 30% 28% 26%

86% 73%

60% 67% 70% 72% 74%

Percent Internal vs. External Identification For all violations

By Discovery Year

External Internal

10 RELIABILITY | ACCOUNTABILITY

Third Quarter Key Enforcement Activities

• Implemented enhancements to Find, Fix, Track and Report (FFT) approved by FERC in June 2013

• Worked with other NERC departments on enforcement issues associated with the CIP Version 5 Transition Program

• Provided training to ERO auditors and industry on FFTs and RAI

• Provided training on finding and using enforcement information on the NERC website

11 RELIABILITY | ACCOUNTABILITY

Key Compliance Activities

Complete ERO Implementation Plan

including streamlined, risk-

based Actively Monitored List

Strategic partnerships

to provide compliance

guidance

Promote a culture of reliability excellence

12 RELIABILITY | ACCOUNTABILITY

Comprehensive ERO Implementation Plan Including Streamlined, Risk-based Actively Monitored List

• Completed and posted 2014 CMEP Implementation Plan (IP) on the NERC website

• Established a risk-based approach for development of Active Monitored List (AML)

• Revised NERC and Regional Entity processes and procedures for CMEP annual planning and IP development NERC and Regional Entities developed a standard Regional IP

template to promote consistency in Regional IPs

Emphasis on Regional reliability assessments and audit scoping

AML serves as Regional planning baseline for compliance monitoring activities

Regional Entities have flexibility to tailor compliance monitoring

13 RELIABILITY | ACCOUNTABILITY

Coordination

• Developed Reliability Standard Audit Worksheet for draft standard COM-003-1

• CIP Version 5 Transition Guidance released in September

• CIP Version 5 Implementation Studies initiated

• Annual Technical Feasibility Exception (TFE) report completed

• Compliance Operations and Event Analysis conducting standards/compliance gap analysis

14 RELIABILITY | ACCOUNTABILITY

Promote a Culture of Reliability Excellence

• Organization Registration and Certification Group Staff Training

o Staff training focus resulted in increased RC System Operator certification for NERC personnel

Expanded Registration Form o Ensures all entities that should be registered are registered, thus removing

reliability gap

o Facilitates complete mapping of entity relationships

15 RELIABILITY | ACCOUNTABILITY

Promote a Culture of Reliability Excellence

• Six Certification Evaluations were conducted between July 1– September 30, 2013 New Brunswick System Operator (RC/TOP) - Replaced SCADA system

and amalgamated into New Brunswick Power

MISO - Carmel (BA) - Expanded footprint

MISO - Sheridan (RC/BA/TOP) - New control center

South Texas Elec Coop (TOP) Expanded footprint

Constellation Energy Control Center and Dispatch (BA) New BA

ITC Great Plains (TOP) New TOP

• Six Certification Evaluations are planned for the fourth quarter of 2013

16 RELIABILITY | ACCOUNTABILITY

Third Quarter Trends for Compliance and Enforcement

• New Possible Violations (NPV) trends

• Filing trends

• Registration

• Certification

• Compliance evaluation

• Audits

17 RELIABILITY | ACCOUNTABILITY

New Possible Violations Trends Third Quarter 2013 Compared with Past Four Quarters

Internal discovery method and CIP/non-CIP trends remain steady. 332 new possible violations were received in Q3 2013.

0%

20%

40%

60%

80%

100%

Past 4 Qs Q3-2013

69% 64%

31% 36%

CIP and non-CIP for NPVs Q3 2013 Compared with Past Four Quarters

CIP Non-CIP

0%

20%

40%

60%

80%

100%

Past 4 Qs Q3-2013

25% 23%

75% 77%

Discovery Method for NPVs Q3 2013 Compared with Past Four Quarters

External Internal

18 RELIABILITY | ACCOUNTABILITY

New Possible Violations Trends by Standard

The top 10 violated standards for the third quarter of 2013 closely follow similar trend for the past four quarters.

0

50

100

150

200

250

300

350

CIP-007 CIP-006 CIP-005 PRC-005 CIP-004 CIP-002 CIP-003 VAR-002 CIP-009 FAC-009

Top 10 Standards Violated Q3 2013 Compared with Past Four Quarters

Q3 2013 Past 4 Quarters

19 RELIABILITY | ACCOUNTABILITY

FFT Utilization

FFT utilization levels remain stable.

44%

34%

22%

Filing By Disposition From September 2011 to September 2013

FFT SNOP NOP

48%

30%

22%

Filing By Disposition From January 2013 to September 2013

FFT SNOP NOP

52%

24%

24%

Q3 2013

FFT SNOP NOP

20 RELIABILITY | ACCOUNTABILITY

Processing Tracks – Past 12 Months

Regional Entities continue to effectively utilize enforcement efficiencies.

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

FRCC MRO NPCC RFC SERC SPP TRE WECC ERO Enterprise

Percent Filings by Regions during the Past 12 months October 2012 to September 30, 2013

NOP SNOP FFT

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Percent Filings by Regions during Q3 2013

FFT SNOP NOP

21 RELIABILITY | ACCOUNTABILITY

Top 10 Violated Standards by Filing Disposition

CIP Reliability Standards are most frequently violated; however most have been processed as FFTs or SNOPs.

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Percent Filing disposition for the top 10 Violated Standard during Q3 2013

FFT SNOP NOP

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Percent Filing disposition for the top 10 Violated Standard during the past 12 months

FFT SNOP NOP

22 RELIABILITY | ACCOUNTABILITY

Registration Activity for Third Quarter 2013

Functions not shown had no activity during this period

2

14

15

2

7

0

1

3

1

0

3

14

15

8

15

3

0 0

1 1

0

2

4

6

8

10

12

14

16

DP GO GOP LSE PSE RP RSG TO TOP TP

Activations Deactivations

Beginning Qtr 543 898 862 516 684 182 17 345 186 187

Ending Qtr 542 898 862 510 676 179 18 348 186 186

23 RELIABILITY | ACCOUNTABILITY

Certification Engagements

Demonstrates increased engagements, primarily reviews, for 2013 due to increased entity awareness of need to reflect changes in basis of certification.

24 RELIABILITY | ACCOUNTABILITY

Investigations

25 RELIABILITY | ACCOUNTABILITY

Complaints

26 RELIABILITY | ACCOUNTABILITY

Compliance Evaluations

27 RELIABILITY | ACCOUNTABILITY

Operations and Critical Infrastructure Audits 2013

28 RELIABILITY | ACCOUNTABILITY

Total Audit Trend by Region

29 RELIABILITY | ACCOUNTABILITY

Questions and Answers