Meeting Minutes Personnel Certification Governance Committee Highlights... · f. Exam Development and Delivery Task Force: The EDD TF discussed the current exam development timeline

Meeting Minutes Personnel Certification Governance Committee May 5-6, 2014 Hyatt Regency St. Louis at the Arch 315 Chestnut Street St. Louis, MO 63102

Administrative

A meeting of the Personnel Certification Governance Committee (PCGC) was held on May 5-6, 2014 in St. Louis, MO. The agenda and action item list are affixed as Attachments A and B, respectively.

1. Welcome and Quorum

PCGC Chair, Brett Hallborg welcomed the committee members and NERC staff. The Chair welcomed new committee member Mike Hoke. A quorum was determined at 8:02 a. m. The list of meeting attendees can be found at the end of the meeting minutes.

NERC Antitrust Compliance Guidelines

The NERC Antitrust Compliance Guidelines were reviewed and all attendees agreed to abide by the NERC Antitrust Guidelines.

Public Meeting Notice

Trion King reminded all participants the meeting is public and was posted on the NERC website.

Task List Review

The PCGC reviewed and updated the Task List, Attachment B.

2. Consent Agenda/Approve Meeting Minutes

The PCGC reviewed, amended and approved the agenda for this meeting. A motion was made and seconded to approve the February 2014 meeting minutes. The motion carried unanimously.

3. System Operator Certification and Continuing Education (SOCCED) Program Updates

System Operator Certification:

NERC transitioned to a new test center vendor. No exams were available during the month of January. Only 107 exams were taken versus the projected 225 for the quarter. Therefore, the first quarter results are higher than expected pass rates for all credentials except Balancing and Interchange (BI). The Reliability Coordinator (RC) credential remains the primary exam taken by most candidates (72.9%). Credential renewals for first quarter were higher than projected. There were 527 credential renewals versus the projected 412 renewals.

Meeting Minutes – Personnel Certification Governance Committee Meeting – May 5-6, 2014 2

Active Credentials:

Currently, there are 7,000 active credentials in System Operator Certification Continuing Education Database (SOCCED). Some system operators hold more than one credential.

Continuing Education:

During the first quarter (2014), 804 Continuing Education (CE) courses were approved and 3,406 continuing education hours (CEHs) were entered onto system operator transcripts. A total of 62 audits were conducted during the first quarter. There are 324 active providers registered in the CE Program. As part of an on-going review of Appendix A, several courses which were submitted for CEHs are out of scope for certified system operators and their roles to maintain the reliability of the bulk power system (BPS).

4. James Merlo, Director of Reliability Risk Management

James Merlo attended day one of the two day session. This is a summary of Mr. Merlo’s presentation: The four primary focus areas for NERC are Reliability, Accountability, Learning and Risk Based Models. Situational Awareness (SA) is the first line of event reporting. Event Analysis (EA) uses Causal Coding to separate real events and human error. Two deliverables from SA and EA are Alerts and Lessons Learned which are published to the industry. Data is available to assist with future system reliability topic selection for Appendix A. Several examples of available data was provided during the meeting.

5. SOCCED Update

Melinda Nicius, SOCCED Database Administrator, provided the latest update on SOCCED Phase 2A upgrade. Phase 2A will enhance the Provider platform of the database. The project is on a very tight schedule. A Request for Quote will be sent to the database vendor, MCG, by the end of the week. Meetings will be set up to discuss the development of Phase 2A. Development is expected to begin before the end of second quarter of 2014. A revised timeline has the desired production date during first quarter 2015. The SOCCED Joint Task Force which consisted of members from the PCGC and Personnel Subcommittee (PS) assisted with the design of Phase 2A.

6. Closed Session 11:00 – 11:45 AM

PCGC entered into closed session to review a hardship case. Also, the Hardship Clause was reviewed and potential changes were discussed among the Committee. The Hardship Task Force (TF) will develop an internal scope document to provide more guidance for the hardship process.

7. Task Force Working Sessions

Exam Development and Delivery (EDD) Task Force: EDD evaluated the current exam development timeline. The timeline is expected to change significantly with the new exam development vendor, PSI. PSI offers dynamic item change out which provides item flexibility and increases exam integrity.

Policy Oversight Task Force: The Policy Oversight TF reviewed all PCGC documents and System Operator Certification Program documents currently on NERC’s website for accuracy. An action item to update the NERC website has been added to the Task List.


8. Task Force Reports

a. Nomination Task Force: The Nomination TF will create a scope to document the process for members who would like to be re-appointed by the NERC Board of Trustees (Board). The TF will create a slate of PCGC members who are up for re-appointment in 2015 and present them during the August meeting. All members with terms ending in 2015 must self-nominate before the August meeting so that the TF can create the slate of nominees. The slate will be submitted to the Board during the annual November meeting so that there is not a gap from November to February. The slate will be created on an annual basis for the appropriate members. Five of the 10 person committee renew in the even years. The remaining five members renew during the odd years.

Moving forward, all new appointees will fill the remainder of the term of the resigning member. In order to remain on the designated expiration year, a second term will be added to the end of the term that is being filled by the new member.

b. Budget Task Force: The Budget TF reviewed the first quarter budget for 2014. The TF will look at more granularity for the revenue forecasts.

c. Record Retention, Storage and Transmittal Task Force: Dave Carlson, TF chair, will schedule an onsite training meeting with NERC staff in Atlanta.

d. Implementation Task Force: There have been a few minor issues with the new test center vendor, PSI. PSI called during the Implementation TF update to discuss the facilities and proctor issues that occurred during February through April. PSI provided survey results from candidate exam surveys. Overall, the candidates rated the experience as good. The scratch paper and pencil issue was the most frequent complaint from the candidates. PSI provided additional training to all proctors and verified the new instructions via testing of the proctors. The second highest complaint was the lack of test center locations for certain areas, particularly, Canada. PSI continues to add additional test center locations to address the identified location gaps. The noted concerns will be re-evaluated during the second quarter to confirm resolution. PSI is pleased with the results and resolutions during the initial transition period. PSI conference call participants are listed on the last page of the meeting minutes.

e. Policy Oversight Task Force: The TF presented a revised version of the System Operator Certification (SOC) Program manual which updated the test center/exam development vendor information. A motion was made and second to accept the vendor information corrections to the SOC Program Manual. The motion carried unanimously.

A legal name change request has been submitted to NERC. The current SOC Program Manual does not address legal name changes. A legal name change paragraph was created and agreed upon by the Committee at Large. The Policy Oversight TF will update the SOC Program manual to address legal name change requests.

The Exam Working Group (EWG) Scope was transferred to the Exam Development and Delivery TF.


The NERC website needs to be cleaned up. Documents that are no longer relevant will be removed from the website. Action items are noted on the Task list.

f. Exam Development and Delivery Task Force: The EDD TF discussed the current exam development timeline. One of the main goals for the TF is to document the exam development process in detail and fully understand where the 2015 exams are in the process. Several items which are noted in the Action Item Task list will need to occur during the second quarter of 2014. Among the action items are: updating Reliability Standards document; update “How to Apply for an Exam” document; update study guides for each credential; review exam item bank; and selection of a “freeze” date for standards and glossary terms. EDD will work closely with the EWG members to review items and create the new exams which will be published in January 2015.

The EWG scope was changed to include free credential renewal for active EWG members. A motion was made and seconded to accept the change to the EWG Scope document. The motion was approved unanimously.

g. SOCCED Joint Task Force: The SOCCED Joint TF assisted NERC staff with the development of Phase 2A. Phase 2A is the upgrade to the Provider platform of the SOCCED.

9. Future PCGC Meetings

2015 Meeting dates have been added to the minutes.

Dates: Location: Comments:

August 12-13, 2014 Seattle, WA

November 12-13, 2014 Atlanta, GA (Westin) Annual Board Update

February 10-11, 2015 Tampa, FL Joint Meeting with PS

May 5-6, 2015 San Antonio, TX

August 11-12, 2015 TBD

November 3-4, 2015 Atlanta, GA Annual Board Update

10. Roundtable

11. Closing Comments and Adjourn – Brett Hallborg, Chair

Meeting was adjourned at 3:00 p. m. Tuesday, May 6, 2014.

12. May 2014 Attendees:

Brett Hallborg, Chair Mike Anderson, Vice Chair Jake Burger Dave Carlson (Tuesday, May 6th Only) Eric Dietzman Bill Ellard Mike Hoke


John Kerr Don Oatman Trion King, NERC Staff, Secretary James Merlo, NERC Staff (Monday, May 5th Only) Tuesday, May 6, 2014 Conference Call Participants: Jo Anne Wingate, VP Client Services Ed Romanowski, Director Product Development Alicia Allegrini, Director Client Services Dan Chase, Director Client Services

Attachment A

Agenda Personnel Certification Governance Committee

May 5-6, 2014 | 8:00 a.m. – 5:00 p.m. Central Hyatt Regency St. Louis at the Arch 315 Chestnut Street St. Louis, MO 63102 Introductions and Chair’s Remarks NERC Antitrust Compliance Guidelines and Public Announcement Agenda

1. Administrative

PCGC Member(s): Trion King

Objective: Review agenda, meeting minutes, action items and task list

Presentation: No Duration: 5 minutes Background Item:

2. Approve Meeting Minutes


Objective: Approve previous meeting minutes: February 4-5, 2014 Phoenix, AZ

Presentation: Duration: 5 minutes Background Item:

3. James Merlo, Director of Reliability Risk Management

PCGC Member(s):

Objective: Future outlook and collaboration

Background:


4. Status Reports

a. System Operator Certification Program Reports


Objective: Present statistics of System Operator Certification Program

Background: PowerPoint Presentation will be provided during meeting.

Presentation: Yes Duration: 45 minutes Background Item:

b. Continuing Education Program Reports


Objective: Present statistics of Continuing Education Program

Background: PowerPoint presentation will be provided during the meeting.


Agenda – Personnel Certification Governance Committee Meeting May 5-6, 2014 2

c. CE Program Sample Course Topics


Objective: Discuss impact of SO Certification Maintenance topics

Background:


5. SOCCED Update

PCGC Member(s): Melinda Nicius

Objective: Provide SOCCED Update. Status of Phase 2A Requirements and SOW. Automated emails.

Background: PowerPoint presentation will be provided during the meeting.


6. Task Force Working Sessions Duration: 3 hours

a. Exam Development & Delivery

b. Implementation

c. Policy Oversight

d. Nomination

7. Task Force (TF) Reports

a. Nomination Task Force: Exam Working Group (EWG) and PCGC

PCGC Member(s): Bill Ellard, TF Chair; John Kerr; Don Oatman; EWG Chair

Objective: Approval of EWG Vice Chair by full committee.

Background:


b. Budget Task Force:

PCGC Member(s): Dave Carlson, TF Chair; Mike Anderson, Jake Burger, Brett Hallborg

Objective: Review 2014 YTD Budget and Projected Budget for 2015-2017

Background: Budget TF will provide an update during the meeting.


c. Record Retention, Storage and Transmittal Task Force:

PCGC Member(s): Dave Carlson, TF Chair; Don Oatman, Bill Ellard

Objective: Develop training for NERC staff and Implementation process.

Background: Dave Carlson to provide update on training process during the meeting.


d. Implementation Task Force:

PCGC Member(s): Jake Burger, TF Chair; Mike Anderson, Bill Ellard

Objective: Status of transition; PSI test center complaints.

Background: Implementation of New Exam Development and Testing Vendor: Website cleanup. TF to provide update during the meeting.


Agenda – Personnel Certification Governance Committee Meeting May 5-6, 2014 3

e. Policy Oversight Task Force:

PCGC Member(s): Jake Burger, TF Chair; Eric Dietzman; Mike Scott; Dave Carlson

Objective: Review of SOC Program Manual update; “Name Change” on Certificates; Leave of Absence

Background: TF to provide update during the meeting. Update test center vendor information and policies.


f. Exam Development and Delivery Task Force:

PCGC Member(s): John Kerr, TF Chair; Dave Carlson; Eric Dietzman; Mike Scott

Objective: Item Writing Workshop Update

Background: Provide expectations of upcoming IWW.


g. SOCCED Joint Task Force:

PCGC Member(s): Eric Dietzman; Dave Carlson

Objective: Provide support to Phase 2A Update with PS members

Background:


8. Task List Review –


Objective: Review task lists and ensure items are properly documented.

Background: Included in the meeting minutes.


9. Future PCGC Meetings:

Dates: Location: Comments:

August 12-13, 2014 Seattle, WA

November 11-12, 2014 Atlanta, GA Annual Board Update

February 10-11, 2015 Tampa, FL Joint Meeting with PS

10. Roundtable

11. Closing Comments and Adjourn – Brett Hallborg, Chair

Attachment B

Action Items Personnel Certification Governance Committee May 5 - 6, 2014

Item# Task Assignment NERC Support Staff Due Task Force

2013-2 Data Storage, Retention, Transmittal NERC staff training and transmittal implementation. Dave will provide onsite training to NERC staff in Atlanta during 2nd-3rd quarter.

Dave Carlson NERC Staff and IT Next Meeting Record Retention, Storage

2013-2.1 PCGC Secure Site to SharePoint Trion King Trion King; NERC IT Next Meeting NERC

2013-4 Mass Email Option (Ability to contact operators through the use of SOCCED) - Proceed. Need delivery date from MCG.

Trion King/ Melinda Nicius

NERC IT/MCG Will be included in SOCCED Phase 3 Update

PCGC

2013-11 Review PCGC web page on NERC.com. Jake Burger Trion King Completed Policy Oversight

2013-19 Metrics: change pass rate to bar chart; add number of exams taken by type along with pass rate; 2nd attempt to bar chart; add details on PCGC budget is independent… take language from ROP.

Trion King Completed

2013-26 Track issues: Testing Trion King Completed

2013-32 Update EWG Scope: Renewal Fee Policy Trion King Completed Policy

2013-36 Confidential vs Closed Trion King Legal Completed

2013-37 Add Active Graph to BOT Report- vet Trion King Completed Policy

2013-38 Budget YTD Budget On-going Budget

2014-1 Early Renewal Email response Process Trion King Melinda Nicius Will be included in SOCCED Phase 2A

Meeting Action Items – Personnel Certification Governance Committee – May 5-6, 2014 2


2014-2 BOT Report: Update to include Exam Taken Pass Rate; Active Credentials


2014-3 PCGC Budget Data from Susan 2015-2017

Trion King Susan Turpen Budget

2014-4 Develop Budget Proposal for NERC Budget TF Completed

2014-6 Proctor Guidelines from PSI Trion King Completed Implementation

2014-7 Map Test centers - PSI Trion King Melinda Nicius Completed Implementation

2014-8 PSI to provide quarterly report. Track comments associated with score of Poor.

Trion King Completed EDD

2014-9 Finalize Policy Oversight Scope Policy Oversight

Policy Oversight

2014-10 SOC Program Manual update Policy Oversight

Completed Policy Oversight

2014-11 Develop invite and registration and agenda for IWW

Trion King Vee Cohens/NERC IT Completed EDD

2014-12 Communicate expectations of SOCCED TF to PS officers

Brett Hallborg Completed

2014-13 ILA and CEH for IWW Trion King Brenda Boline Completed EDD

2014-14 Work with PS on SOCCED update Trion King Melinda Nicius Completed SOCCED TF

2014-15 Present Nominees for EWG Vice Chair. Trion King Completed Nominations

2014-16 PCGC New member to Board Trion King Completed

2014-17 Invite new member to May meeting Trion King Completed

2014-18 Provide meeting minutes draft to PCGC week after meeting

Trion King February 12, 2014 Completed

2014-21 2015 Joint meeting: Feb. 10th – 11th Trion King Meeting Planner



2014-22 Verify Budget Due Date: 4/15/14 Trion King Susan Turpen Completed Budget

2014-23 Change color of BI on Exams by Type Chart


2014-24 Send draft of BOT report to Chair Trion King Completed

2014-25 Work station notification: Consistency of Pass/Fail

Trion King PSI Completed Implementation

Remove “Draft” from February meeting minutes.

Trion King Sandy Shiflett

Modify Active Credential Report Trion King Brett Hallborg

Verify Standards references are accurate for 2015 Exams and Glossary of Terms for pilot items (2012).

EWG Next meeting EDD

Kick Start of 2018 Exam Development Process

EDD November 2014 EDD

Create fluid process of reviewing exam items with current Standards (2018)

EDD

Long term – for 2018 exams

Update How to Apply for SOC Exam document. Email process has changed.

EDD Trion King Next Meeting EDD

Update “2012 Study Guide” for 2015 Exam Rollout

EWG Next Meeting EDD

Create Hardship TF Scope Hardship TF Next Meeting Policy Oversight

Verify Hardship cases are documented in SOCCED

Trion King Next Meeting

Update SOC Program Manual to reflect current vendor information

Trion King Technical Publications

Next Meeting Implementation

Update SOC/PCGC/EWG website: Policy/ Trion King

Sandy Shiflett Next Meeting Policy Oversight



Delete EWG files except Scope

Delete PCGC files except Charter

Update Fee Schedule

Exam Statistics 2014 – remove

Prometric Cancellation Policy – remove

SOC Exam Announcement – Remove

SOCCED User Guide – Remove

How to Renew - remove

Create Nominations TF Scope Nominations Next Meeting Nominations

Create Re-appointment Slate for PCGC members

Trion King Next Meeting

B. Hallborg to discuss Joint SOCCED TF with L. Hennebury

Brett Hallborg