11

SUCCESSFUL NERC SUCCESSFUL NERC CIP COMPLIANCECIP COMPLIANCE

Robert E. HoopesRobert E. Hoopes

PPL CorporationPPL Corporation

November 2, 2011November 2, 2011

22

OverviewOverview

Simple Compliance ModelSimple Compliance Model CIP CredentialsCIP Credentials Success RequirementsSuccess Requirements CIP ExperienceCIP Experience How Much?How Much? Audit Prep TimelineAudit Prep Timeline The PayoffThe Payoff

33

Reality CheckReality Check

““The issue is not whether your network is protected. The issue is not whether your network is protected. We know that your network is protected. The issue is We know that your network is protected. The issue is about strict compliance to the plain reading of the about strict compliance to the plain reading of the language in the CIP standards.” language in the CIP standards.”

Corporate Risk Solutions, Inc. (CRSI), December 2009Corporate Risk Solutions, Inc. (CRSI), December 2009

Translation…while the real objective is protecting Translation…while the real objective is protecting critical assets, the report card is based on compliance critical assets, the report card is based on compliance to the minutia in the CIP standards, as judged by the to the minutia in the CIP standards, as judged by the auditors.auditors.

44

Simple Compliance ModelSimple Compliance Model

Three ElementsThree Elements Clear requirementsClear requirements Clear accountabilityClear accountability Documented programDocumented program

All three elements are necessary for successAll three elements are necessary for success By far, clear accountability is the most importantBy far, clear accountability is the most important Accountable individuals make things happenAccountable individuals make things happen

This is the same for executives down to the Subject This is the same for executives down to the Subject Matter ExpertsMatter Experts

55

CIP CredentialsCIP Credentials Responsible for corporate NERC compliance since late 2006Responsible for corporate NERC compliance since late 2006

Assigned by the CEO as CIP “senior manager” Assigned by the CEO as CIP “senior manager”

Built CIP program for multiple Registered Entities from the Built CIP program for multiple Registered Entities from the ground up, as part of overall NERC compliance programground up, as part of overall NERC compliance program

External consultant supportExternal consultant support One failed gap analysis – Un-named consultantOne failed gap analysis – Un-named consultant One gap analysis and two mock audits - CRSIOne gap analysis and two mock audits - CRSI

One CIP audit (covered five GO/GOP Register Entities – 36 One CIP audit (covered five GO/GOP Register Entities – 36 requirements); verbal feedback:requirements); verbal feedback: Advanced documentation provided was “far superior than anything we Advanced documentation provided was “far superior than anything we

have seen”have seen” ““Best CIP compliance program we have seen”Best CIP compliance program we have seen” Audit completed in 3.5 daysAudit completed in 3.5 days Two minor issues identifiedTwo minor issues identified

66

Success Requirements (8)Success Requirements (8)

Successful NERC CIP Compliance requires:Successful NERC CIP Compliance requires: Leadership engagementLeadership engagement An organizational culture of complianceAn organizational culture of compliance An “effective” CIP Senior ManagerAn “effective” CIP Senior Manager A strong foundational programA strong foundational program Technically competent Subject Matter ExpertsTechnically competent Subject Matter Experts Sufficient resourcesSufficient resources Strong consulting supportStrong consulting support Extensive audit preparationExtensive audit preparation

77

Success Requirement #1:Success Requirement #1:Leadership EngagementLeadership Engagement

All compliance is localAll compliance is local

Executive management must communicate to senior Executive management must communicate to senior management that CIP compliance is importantmanagement that CIP compliance is important Senior management in turn must communicate this Senior management in turn must communicate this

message to line managementmessage to line management Line management makes it happenLine management makes it happen

Communicates importance of CIP complianceCommunicates importance of CIP compliance Provides sufficient resourcesProvides sufficient resources Sets the prioritiesSets the priorities Periodically checks on performancePeriodically checks on performance

Clear accountability is essentialClear accountability is essential

88

Success Requirement #2:Success Requirement #2:Org. Culture of ComplianceOrg. Culture of Compliance

Strict compliance is often counter-intuitive to Strict compliance is often counter-intuitive to individuals who have not been previously exposed to individuals who have not been previously exposed to itit Mountains of records are required…Why?? “Because…Mountains of records are required…Why?? “Because…

it’s the law.”it’s the law.”

Establishing a culture of compliance takes time. Establishing a culture of compliance takes time. People watch their leaders for cues…”Do as I do…” People watch their leaders for cues…”Do as I do…” will help, if leaders are engaged.will help, if leaders are engaged.

Always do the right thing. This sets the tone for the Always do the right thing. This sets the tone for the organization.organization.

99

Success Requirement #3:Success Requirement #3:Effective CIP Senior ManagerEffective CIP Senior Manager

CIP compliance does not just happen. While it is a function of CIP compliance does not just happen. While it is a function of smart people wanting to do the right things, absent sound smart people wanting to do the right things, absent sound leadership there will be gaps in compliance. Different parts of leadership there will be gaps in compliance. Different parts of the organization will do what they think is required but they the organization will do what they think is required but they may leave gaps in the “white space” between internal work may leave gaps in the “white space” between internal work groups.groups.

CIP-003 Requirement 2 calls for the assignment of a single CIP-003 Requirement 2 calls for the assignment of a single manager with overall responsibility and authority for leading manager with overall responsibility and authority for leading and managing adherence to the CIP standards.and managing adherence to the CIP standards.

The CIP founders got this one right.The CIP founders got this one right. However, the CIP senior manager does not relieve local line leadership However, the CIP senior manager does not relieve local line leadership

of CIP accountabilityof CIP accountability

1010

Effective CIP Senior Manager Effective CIP Senior Manager cont.cont.

Perfunctory assignment of a high level senior Perfunctory assignment of a high level senior leader as the required “senior manager” to leader as the required “senior manager” to meet the CIP-003 R2 requirement is meet the CIP-003 R2 requirement is problematic.problematic. If the assigned senior manager is too high in the If the assigned senior manager is too high in the

organization to be engaged in the ongoing issues organization to be engaged in the ongoing issues related to CIP compliance, problems will arise and related to CIP compliance, problems will arise and find you at a later time.find you at a later time.

1111

Success Requirement #4:Success Requirement #4:Strong Foundational ProgramStrong Foundational Program

A solid compliance program has three elementsA solid compliance program has three elements Clear requirements (CIP standards…)Clear requirements (CIP standards…) Clear accountability (engaged leadership)Clear accountability (engaged leadership) Documented programmatic controls (policies and Documented programmatic controls (policies and

procedures)procedures)

Programmatic controls must documentProgrammatic controls must document Who is responsible for what?Who is responsible for what? CIP-003 Requirement 1 calls for a cyber security policy CIP-003 Requirement 1 calls for a cyber security policy

that addresses the requirements in Standards CIP-002 thru -that addresses the requirements in Standards CIP-002 thru -009.009.

The cyber security policy should document the what and who is The cyber security policy should document the what and who is responsible (across the organization) for doing itresponsible (across the organization) for doing it

1212

Success Requirement #5:Success Requirement #5:Technically Competent SMEsTechnically Competent SMEs

Study and understand the CIP requirementsStudy and understand the CIP requirements

Identify how to comply and make it happenIdentify how to comply and make it happen

Accountable to their line leadershipAccountable to their line leadership

Should be responsible for producing and storing Should be responsible for producing and storing required evidence of compliancerequired evidence of compliance

Explain to the auditors how/why the entity is Explain to the auditors how/why the entity is compliant to the applicable CIP requirementcompliant to the applicable CIP requirement

1313

Success Requirement #6: Success Requirement #6: Sufficient ResourcesSufficient Resources

How much is enough?How much is enough? Line leadership must decide, based on competing Line leadership must decide, based on competing

objectives for available resourcesobjectives for available resources

Not enough can lead to painful shortfallsNot enough can lead to painful shortfalls Can result in expensive violationsCan result in expensive violations

CIP compliance must be part of individuals’ job CIP compliance must be part of individuals’ job functionsfunctions Full time CIP resources are the exceptionFull time CIP resources are the exception CIP audit preparation is labor intensive, beginning months CIP audit preparation is labor intensive, beginning months

before the audit and involving the various CIP SMEs in the before the audit and involving the various CIP SMEs in the business line, IT, security and other support groupsbusiness line, IT, security and other support groups

1414

Success Requirement #7:Success Requirement #7:Strong Consulting SupportStrong Consulting Support

Outside eyes on your CIP program and evidence is Outside eyes on your CIP program and evidence is absolutely essentialabsolutely essential They will see and interpret things differently than your They will see and interpret things differently than your

SMEsSMEs Based on their industry experience, they will be right most Based on their industry experience, they will be right most

of the timeof the time

Can help identify and help fix problem areasCan help identify and help fix problem areas

Choose good CIP consultantsChoose good CIP consultants Excellent audit support record (based on input from your Excellent audit support record (based on input from your

peers)peers) Those that perform CIP audits for Regions have a unique Those that perform CIP audits for Regions have a unique

perspective that is invaluableperspective that is invaluable

1515

Success Requirement #8:Success Requirement #8:Extensive Audit PreparationExtensive Audit Preparation

Begin immediately and do it annuallyBegin immediately and do it annually If you have not yet started, you are lateIf you have not yet started, you are late

Complete the CIP RSAWS and organize supporting evidence Complete the CIP RSAWS and organize supporting evidence of complianceof compliance

The audit package for some CIP standards can exceed 1,000 The audit package for some CIP standards can exceed 1,000 pagespages

Audit packages should be signed by a Preparer, Reviewer and Audit packages should be signed by a Preparer, Reviewer and Approver.Approver. Approver is the local VP or GM - responsible for CIP compliance in Approver is the local VP or GM - responsible for CIP compliance in

their organizationtheir organization Sometimes more than one preparer and reviewer sign the packages, Sometimes more than one preparer and reviewer sign the packages,

based on distribution of laborbased on distribution of labor

1616

Extensive Audit Preparation Extensive Audit Preparation cont.cont.

Recent GO/GOP CIP audit preparation and conduct Recent GO/GOP CIP audit preparation and conduct involved 33 CIP compliance personnel and SMEsinvolved 33 CIP compliance personnel and SMEs Evidence collectionEvidence collection Evidence reviewingEvidence reviewing Evidence packagingEvidence packaging SME reviewSME review Legal reviewLegal review SME audit presentation trainingSME audit presentation training SMEs standing by during audit to present and/or answer SMEs standing by during audit to present and/or answer

auditors’ questionsauditors’ questions

1717

CIP Experience:CIP Experience:Program Start-up to CIP AuditProgram Start-up to CIP Audit

JAN 2007 – Began development of the required CIP Cyber JAN 2007 – Began development of the required CIP Cyber Security Policy and other program documentsSecurity Policy and other program documents Laid out internal responsibilities for each CIP requirementLaid out internal responsibilities for each CIP requirement Identified key CIP compliance individual in each affected organizationIdentified key CIP compliance individual in each affected organization

MAY 2008 – Reviewed MAY 2008 – Reviewed CIP-002 and CIP-006 implementation CIP-002 and CIP-006 implementation with external consultants with external consultants

JUN 2008 – Turned on PSP securityJUN 2008 – Turned on PSP security

APR 2009 – Aborted CIP Gap AnalysisAPR 2009 – Aborted CIP Gap Analysis Consultants were the wrong fitConsultants were the wrong fit

JUL 2009 – Conducted JUL 2009 – Conducted CIP Gap AnalysisCIP Gap Analysis (CRSI) (CRSI) Numerous issues needed refinementNumerous issues needed refinement

1818

CIP Experience:CIP Experience:Start-up to Audit Start-up to Audit cont.cont.

SEP 2009 – Turned on security for the remaining ESPsSEP 2009 – Turned on security for the remaining ESPs

DEC 2009 – Conducted DEC 2009 – Conducted CIP Mock AuditCIP Mock Audit (CRSI) (CRSI) One major deficiency, self-reported prior to 1/1/2010 (mandatory One major deficiency, self-reported prior to 1/1/2010 (mandatory

enforcement date)enforcement date)

JAN 1, 2010 – CIP Standards mandatory and enforceable for JAN 1, 2010 – CIP Standards mandatory and enforceable for PPL Registered EntitiesPPL Registered Entities

MAY-OCT 2010 – Self-reported minor CIP violationsMAY-OCT 2010 – Self-reported minor CIP violations Several were residual issues from prior to January 1, 2010Several were residual issues from prior to January 1, 2010

MAR 2011 – Conducted MAR 2011 – Conducted CIP Mock AuditCIP Mock Audit (CRSI) (CRSI) Minor issues needed refinementMinor issues needed refinement

MAY 2011 – Conducted MAY 2011 – Conducted CIP AuditCIP Audit Two minor issues identifiedTwo minor issues identified

1919

How Much?How Much? CIP Gap Analysis and Mock AuditCIP Gap Analysis and Mock Audit

each around consulting 80 man-hours plus travel and each around consulting 80 man-hours plus travel and expensesexpenses

CIP Audit Prep – High volume of internal resources CIP Audit Prep – High volume of internal resources expendedexpended Audit included five Registered Entities (two with Critical Audit included five Registered Entities (two with Critical

Assets)Assets) Four compliance personnel and the various Subject Matter Four compliance personnel and the various Subject Matter

Experts put in many hoursExperts put in many hours Months of preparationMonths of preparation Post Audit – 33 individuals recognized for their Post Audit – 33 individuals recognized for their

contributions to the preparation and conduct of the auditcontributions to the preparation and conduct of the audit

2020

Audit Prep TimelineAudit Prep Timeline December 2010 December 2010

Began CIP Audit Package DevelopmentBegan CIP Audit Package Development

February 8, 2011 February 8, 2011 PPL received 90-day notification letterPPL received 90-day notification letter

February 28 – March 4February 28 – March 4 Conducted Third Party Mock AuditConducted Third Party Mock Audit

March 10March 10 Submitted Pre-Audit Survey and QuestionnaireSubmitted Pre-Audit Survey and Questionnaire

March 30 March 30 Submitted RSAW and EvidenceSubmitted RSAW and Evidence

May 9 -13May 9 -13 Conducted RFC Onsite AuditConducted RFC Onsite Audit

2121

Audit Package PreparationAudit Package Preparation One package for each of the eight CIP standardsOne package for each of the eight CIP standards

Most packages > 1,000 pagesMost packages > 1,000 pages

Work began in DecemberWork began in December Compliance Specialists led this effort with support from the SMEsCompliance Specialists led this effort with support from the SMEs

Compliance staff met weekly with SMEs to review RSAW Compliance staff met weekly with SMEs to review RSAW language and supporting evidence language and supporting evidence

Audit packages were reviewed by SMEs and OGCAudit packages were reviewed by SMEs and OGC

Two Day Offsite Meeting with all SMEs and Compliance staff Two Day Offsite Meeting with all SMEs and Compliance staff to review completed packagesto review completed packages

2222

The PayoffThe Payoff

Regional Entity Feedback:Regional Entity Feedback: CEO: Advanced documentation provided was “far CEO: Advanced documentation provided was “far

superior than anything we have seen”superior than anything we have seen” Audit Team: “Best CIP compliance program we Audit Team: “Best CIP compliance program we

have seen”have seen”

Audit completed in 3.5 daysAudit completed in 3.5 days Included the review of more than100 TFEsIncluded the review of more than100 TFEs

Two minor issues identifiedTwo minor issues identified

2323

Was It Worth It??Was It Worth It?? Enforcement space is very expensiveEnforcement space is very expensive

Even a minor violation receiving a minor penalty Even a minor violation receiving a minor penalty has many thousands of dollars in hidden has many thousands of dollars in hidden processing costsprocessing costs

While the real objective is protecting critical While the real objective is protecting critical assets, the report card is based on compliance assets, the report card is based on compliance to the details in the CIP standards, as judged to the details in the CIP standards, as judged by the auditorsby the auditors

You be the judge…You be the judge…

2424

Questions?Questions?