1 RELIABILITY | ACCOUNTABILITY

Key Compliance and Enforcement

Metrics and Trends

Compliance Committee Meeting August 14, 2013

Agenda Item 4 Compliance Committee Meeting August 14, 2013

2 RELIABILITY | ACCOUNTABILITY RELIABILITY | ACCOUNTABILITY

Key Enforcement and Compliance Contributions to ERO Strategic Goals

Activities related to ERO Strategic Goals

2013- 2016

Compliance and Enforcement Reform

• Reliability Assurance Initiative (RAI)

Monitor Caseload Aging

Monitor Mitigation Aging

Monitoring strategies commensurate with

entity’s role and risk to BPS

Promote a culture of reliability excellence

3 RELIABILITY | ACCOUNTABILITY

Developing self-report process improvement and FFT enhancement

as part of the RAI

Measuring process efficiencies

Promoting self-assessment and identification of noncompliance

Key Enforcement Initiatives

4 RELIABILITY | ACCOUNTABILITY

Developing Self-Report Process Improvements and FFT Enhancement as Part of the RAI

• Incremental enhancements to FFT program from March 2013 filing approved in June 2013 and implemented in July 2013.

Broader scope of issues eligible for FFT treatment.

Additional processing enhancements.

• Positive impact of FFT program on processing time, a lead concern identified by registered entities.

Average processing time for all violations is approximately 12 months; average processing time for FFTs is six months.

5 RELIABILITY | ACCOUNTABILITY

Measuring Process Efficiencies

• Enforcement Metrics

Caseload Index

Violation Aging

Violation Aging by Disposition Method

Mitigation Aging (in development)

6 RELIABILITY | ACCOUNTABILITY

ERO Caseload Index

ERO Caseload Index (as of June 28, 2013)

Violations in Inventory

Violations filed with FERC*

Total Number of Dismissals*

Caseload Index (Months)

1,995 1,814 633 9.78

Caseload index excludes violations that are held by appeal, a regulator, or a court.

Violations in inventory are comprised of active violations that have not been filed with FERC.

*From July 2012 to June 2013.

Corporate goal for 2013 caseload index is 10 months.

7 RELIABILITY | ACCOUNTABILITY RELIABILITY | ACCOUNTABILITY

0

100

200

300

400

500

600

700

800

900

2009 2010 2011 2012 2013Non-CIP 12 62 233 186CIP 16 91 317 592 486

Violation Aging - ERO

Average age of violation, in months, in the ERO Enterprise inventory is about 12 months.

* This excludes violations that are held by

appeal, a regulator, or a court.

Shown by Discovery year

8 RELIABILITY | ACCOUNTABILITY RELIABILITY | ACCOUNTABILITY

Violation Aging - Disposition Method

ERO Violation Aging by Disposition Method

For violations discovered after 1/1/2012 and filed up to date

FFT = 6.9 months

SNOP= 9.7 months NOP = 12.9 months

9 RELIABILITY | ACCOUNTABILITY RELIABILITY | ACCOUNTABILITY

Caseload Index and Violation Aging

ERO

Current inventory: 1,995

Months in the inventory: 12.4

Six months Caseload index: 10.34

Regional Entity

Current inventory: 1,787

Months in the inventory: 11.8

Six months Caseload index: 9.98

NERC

Current inventory: 208

Months in the inventory: 1.1

Six months Caseload index: 1.49

These new metrics are developed based on violations processed in the first six months of 2013. Violations that are held by appeal, a regulator, or a court are excluded from computation of these metrics.

10 RELIABILITY | ACCOUNTABILITY

Promoting Self Assessment and Identification of Noncompliance

Significant number of violations continues to be identified by registered entities internally.

0%

20%

40%

60%

80%

100%

120%

2007 2008 2009 2010 2011 2012 2013

14% 27%

40% 33% 30% 28% 23%

86% 73%

60% 67% 70% 72% 77%

Percent Internal vs. External Identification For all violations

By Discovery Year

External Internal

29%

71%

Internal vs. External Identification From 2007 to June 28 , 2013

External Internal

11 RELIABILITY | ACCOUNTABILITY

Promoting Self Assessment and Identification of Noncompliance

Audit 23%

Complaint 0%

Exception Report 0%

Investigation 2%

Periodic Data Submittal

0%

Self-Certification 19%

Self-Report 51%

Spot-Check

5% Audit

Complaint

Exception Report

Investigation

Periodic Data Submittal

Self-Certification

Self-Report

Spot-Check

Internal Discovery Methods: Self-Report, Self Certification, Periodic Data Submittal and Exception Report External Discovery Methods: Complaint, Audit, Spot Check and Investigation.

The graph is based on all violations discovered from 2007 through June 28, 2013

12 RELIABILITY | ACCOUNTABILITY

Comprehensive ERO Implementation Plan including streamlined, risk-based Actively Monitored List

Partnering with Standards to develop pro forma compliance guidance early in the standards

process

Promote a culture of reliability excellence

Conducting prototyping and piloting to test RAI concepts

Completion of Audit Handbook

Key Compliance Initiatives

13 RELIABILITY | ACCOUNTABILITY

Comprehensive ERO IP Including Streamlined, Risk-based AML

• Compliance Monitoring Enforcement Program Implementation Plan (CMEP)

• Revised NERC and Regional Entity processes and procedures for CMEP annual planning and Implementation Plan (IP)

development. Finalized Regional IP template

o IP scheduled to be posted on September 1, 2013

Established a risk-based approach for development of AML Revision

14 RELIABILITY | ACCOUNTABILITY

Partnering with Standards

• Developed pro forma compliance guidance early in the Standards process for six sets of standards. Pro forma packages included:

Draft Standard Authorization Request Form (SAR),

Draft Standard, and

Draft compliance guidance.

15 RELIABILITY | ACCOUNTABILITY

Promote a Culture of Reliability Excellence

• Certification Activities

Seven Certification Evaluations were conducted between April 1, 2013 – June 30, 2013

Six Certification Evaluations are planned for the third quarter of 2013

Incorporated questions based on recommendations from the Southwest Blackout Report

16 RELIABILITY | ACCOUNTABILITY

Promote a Culture of Reliability Excellence

Number of certification site visits.

17 RELIABILITY | ACCOUNTABILITY

• Compliance Investigations

Compliance Investigations group provides quality assurance to Regional Entities

NERC’s technical investigation staff reviews and facilitates a compliance evaluation of bulk power system events

Process is published on the NERC website

o Reduction in older events

o Risk determined faster

Promote a Culture of Reliability Excellence

18 RELIABILITY | ACCOUNTABILITY

June July August May

May 20

REs notified of CIP-001-2a

KRSSC

June 17-21

CIP-001-2a Detailed Evidence Review

July 29 –August 2

EOP-004-1 Detailed Evidence Review

May 30

REs notified of Selected Audits

July 11

REs notified of Scope

Expansion to include

EOP-004-1

August 19-23

Compile and Analyze

KRSSC results

September 3-9

Develop Guidance and Draft Report

September 10-30

NERC Management Review and Approval of

KRSSC Report

October Nov. September

October 2-25

Wrap up, and Communicate Issues to other

NERC Departments

as appropriate

November 6-7

Present KRSSC Results to BOT

Key Reliability Standard Spot Check (KRSSC)

Promote a Culture of Reliability Excellence

19 RELIABILITY | ACCOUNTABILITY RELIABILITY | ACCOUNTABILITY

Focus on Dismissals

20 RELIABILITY | ACCOUNTABILITY RELIABILITY | ACCOUNTABILITY

Possible Violations Tracks

• There are four disposition methods for a violation namely: dismissal, Full Notice and Penalty (NOP), Spreadsheet Notice of Penalty (SNOP) and Find, Fix, Track and Report (FFT)

21 RELIABILITY | ACCOUNTABILITY

Violation Disposition by Discovery Year

2007 2008 2009 2010 2011 2012 2013

48%

63% 72%

60% 52%

30%

8%

51%

31% 24%

28%

31%

31%

10%

1% 5% 4%

11% 17%

39%

82%

Violation Disposition by Discovery Year As of June 28, 2013

Filed Dismissed Inventory

On average 30% of the violations discovered annually have been dismissed.

22 RELIABILITY | ACCOUNTABILITY

Dismissal By Internal and External Discovery Method

Dismissal- External

23%

Dismissal- Internal

77%

Dismissals By Percent Internal and External Discovery

As of June 28, 2013

10%

19%

48%

36%

21%

14%

19%

90%

81%

52%

64%

79%

86%

81%

2007

2008

2009

2010

2011

2012

2013

Dismissals By Percent Internal and External Discovery

By Discovery Year As of June 28, 2013

Dismissal- External Dismissal- Internal

Only 23% of the violations dismissed are discovered through external methods (spot-check, audit, complaints and investigation).

23 RELIABILITY | ACCOUNTABILITY

Basis for Dismissed Violations Discovered Through External Methods

Additional evidence/ further

review and discussion with

enforcement 59%

Another requirement was

applicable 2%

Other 13%

Transfer to another Region

6%

Transfer to another Violation -

Expansion of scope/Duplicate

20%

Most instances of dismissals of externally discovered violations relate to availability of additional information.

Violations discovered from January 1, 2012 to December 31, 2012 and dismissed as of March 15, 2013.

24 RELIABILITY | ACCOUNTABILITY

Basis for Dismissed Violations Discovered Through Internal Methods

Another requirement was

applicable 5%

Other 5%

Sufficient evidence provided later

22%

Transfer to another Region

1% Transfer to another Violation -

Expansion of scope/Duplicate

67%

Violations discovered from January 1, 2012 to December

31, 2012 and dismissed as of March 15, 2013.

Most instances of dismissals of internally discovered violations relate to duplication.

25 RELIABILITY | ACCOUNTABILITY

Appendix

• Supplemental information on Enforcement

New Possible Violations (NPV) trends

Filing by disposition method

o FFT utilization

o Top 10 violated standards by filing disposition

o ERO processing in the past four quarters

26 RELIABILITY | ACCOUNTABILITY

New Possible Violations Trends Q2 2013 Compared with Past 4 Quarters

Internal discovery method and CIP/ non-CIP remain steady. 302 New Possible Violations were received in Q2 2013.

0%

10%

20%

30%

40%

50%

60%

70%

CIP Non-CIP

69%

31%

67%

33%

CIP and non-CIP for NPVs Q2 2013 Compared with Past Four Quarters

Past 4 Quarters Q2- 2013

0%

10%

20%

30%

40%

50%

60%

70%

80%

External Internal

27%

73%

25%

75%

Discovery Method for NPVs Q2 2013 Compared with Past Four Quarters

Past 4 Quarters Q2 2013

27 RELIABILITY | ACCOUNTABILITY

New Possible Violations Trends by Standard

The top 10 violated standards for Q2 2013 closely follow similar trend for the past four quarters.

0

50

100

150

200

250

300

350

CIP-007 CIP-006 CIP-005 PRC-005 CIP-002 CIP-004 CIP-003 VAR-002 CIP-009 FAC-009

Top 10 Standards Violated Q2 2013 Compared with Past Four Quarters

Q2 2013 Past 4 Quarters

28 RELIABILITY | ACCOUNTABILITY

FFT Utilization

FFT utilization levels remain stable.

FFT 42%

NOP 22%

SNOP 36%

21 Months Filing From September 30, 2011 through June 28,

2013

FFT 53%

NOP 15%

SNOP 32%

Q2 2013 Filings

29 RELIABILITY | ACCOUNTABILITY

Processing Tracks – Past 12 Months

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

WECC

TRE

SPP

SERC

RFC

NPCC

MRO

FRCC

ERO Enterprise

WECC TRE SPP SERC RFC NPCC MRO FRCC ERO Enterprise

FFT 25% 54% 44% 43% 41% 60% 57% 45% 41%

SNOP 37% 23% 28% 38% 28% 39% 39% 40% 33%

NOP 38% 23% 28% 19% 32% 1% 4% 15% 26%

Violations Filed from July 1, 2012 to June 28, 2013

Regional Entities continue to effectively utilize enforcement efficiencies.

30 RELIABILITY | ACCOUNTABILITY

Top 10 Violated Standards by Filing Disposition

CIP Reliability Standards are most frequently violated; however most have been processed as FFTs or SNOPs.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

FAC-009

EOP-005

CIP-003

VAR-002

CIP-002

CIP-004

CIP-006

CIP-005

PRC-005

CIP-007

Percent Filing disposition for the top 10 Violated Standard

FFT SNOP NOP

31 RELIABILITY | ACCOUNTABILITY

Processing for the Past Four Quarters

-450

-400

-350

-300

-250

-200

-150

-100

-50

0

50

Q3 2012 Q4 2012 Q1 2013 Q2 2013

ERO Processing Throughput Past 4 Quarters

Throughput=NPV-(FERC filing +Dismissal)

32 RELIABILITY | ACCOUNTABILITY

Processing of Pre-2012 Caseload

Corporate goals for 2013 include eliminating pre-2012 caseload.

997

958 906

786

664

535

498

0

200

400

600

800

1000

1200

Dec-12 Jan-13 Feb-13 Mar-13 Apr-13 May-13 Jun-13

Active Violations Pre Jan 2012 Excludes on hold Violations

Active pre 2012- excludes on hold

Active violations pre-January 2012

excludes violations that are held by appeal, a regulator, or a court.

33 RELIABILITY | ACCOUNTABILITY

Questions and Answers