Role of Account Management at ERCOT Nodal Training Project Debrief for TPTF September 12, 2006 Market Education

Role of Account Management at ERCOTNodal Training ProjectDebrief for TPTF

September 12, 2006

Market Education

September 12, 2006 TPTF Nodal Training Debrief

Agenda

I. Readiness Discussion

II. Course Availability and Schedule

III. Nodal Training Communications

IV. Standard Course Development Process

V. Project Planning and Budgeting Assumptions

VI. Suggestions for Restructuring the Course Curriculum

VII. LMS Update

VIII. Nodal 101 Course Update

VII. LMP 201 Update

VIII. Work in Progress

Market Education

Nodal Education:(Course Title)

(Presenter Name)(Presenter Title)ERCOT

July 17, 2006

Role of Account Management at ERCOTReadiness Discussion

Market Education


Discussion

• Materials have all been distributed– Summarized TAC comments– Market Readiness Requirements

• Validate comments and discuss

Market Education



July 17, 2006

Role of Account Management at ERCOTCourse Availability and Schedule

Market Education


TPTF Curriculum – Proposed Availability Schedule

Group / Course

Pro

po

sed C

ou

rse H

ou

rs

So

urce o

f SM

E

No

dal T

rainin

g

Su

pp

ort

Delivered

By

NE

RC

Certified

Test R

equ

ired

Visu

al Pro

cess Map

Face to

Face

Availab

le

Web

-Ex A

vailable

Web

-based

A

vailable

Basic Training Program QSE 40 Nodal Training Standard Process Nodal Training Yes Yes Yes June, 2007 June, 2007 Aug, 2007CRRs, PCRRs, and Supply Hedging 8 Nodal Training Standard Process Nodal Training Yes Yes Yes Nov, 2007 Nov, 2007 Dec, 2007ERCOT 101 for Wind Generation 8 Nodal Training Standard Process Nodal Training Yes Yes Yes Jan, 2008 Feb, 2008 Mar, 2008ERCOT LMP 201 (Economics) 8 Prof. Baldick TBD Prof. Baldick No TBD No Oct, 2007 NA NAERCOT Nodal 101: The Basics 8 Nodal Training Standard Process Nodal Training Yes Yes Yes Available Dec, 2006 Dec, 2006Extended Training - System Dynamics 8 Nodal Training Standard Process Nodal Training Yes Yes Yes Sep, 2009 NA NAGeneration 101 16 Nodal Training Standard Process Nodal Training Yes Yes Yes Dec, 2007 Jan, 2008 Feb, 2008Generation 201 16 Nodal Training Standard Process Nodal Training Yes Yes Yes Dec, 2007 Jan, 2008 Feb, 2008Generation 301 8 Nodal Training Standard Process Nodal Training Yes Yes Yes Dec, 2007 NA NALoad Serving Entity 201 20 Nodal Training Standard Process Nodal Training Yes Yes Yes May, 2007 May, 2007 July, 2007Market Settlements 301 16 Nodal Training Standard Process Nodal Training Yes Yes Yes Sep, 2007 Oct, 2007 NAMIS 101: Market Information Systems 8 Nodal Training Standard Process Nodal Training Yes Yes Yes Dec, 2007 Jan, 2008 NANetwork Model Management 16 Nodal Training Standard Process Nodal Training Yes Yes Yes Nov, 2007 Nov, 2007 NANOIE QSE Operations 8 Nodal Training Standard Process Nodal Training Yes Yes Yes May, 2007 July, 2007 NAScenario-based Training for Generators 8 Nodal Training Standard Process Nodal Training Yes Yes Yes Oct, 2008 NA NAScenario-based Training for LSEs 8 Nodal Training Standard Process Nodal Training Yes Yes Yes Oct, 2008 NA NAScenario-based Training for TSPs and DSPs 8 Nodal Training Standard Process Nodal Training Yes Yes Yes Oct, 2008 NA NASystem Operator Seminar (Support for) 24 Nodal Training Standard Process Nodal Training Yes Yes Yes 2009 NA NATransition to Nodal Markets & Start-up Testing 16 Nodal Training Standard Process Nodal Training Yes Yes Yes April, 2007 May, 2007 NATransmission 101 20 Nodal Training Standard Process Nodal Training Yes Yes Yes Nov, 2007 Nov, 2007 Jan, 2008

Available dates are based on current information, staffing and plan assumptions.


ERCOT Requested Supported Courses and Capabilities

Group / Course

Pro

po

sed C

ou

rse H

ou

rs

So

urce o

f SM

E

No

dal T

rainin

g

Su

pp

ort

Delivered

By

NE

RC

Certified

Test R

equ

ired

Visu

al Pro

cess Map

Face to

Face

Availab

le

Web

-Ex A

vailable

Web

-based

A

vailable

EMS Product Training 16 Vendor Templates, LMS Upload Vendor Yes Yes No TBD NA NA

MMS Product Training 16 Vendor Templates, LMS Upload Vendor Yes Yes No TBD NA NA

OTS Product Training 16 Vendor Templates, LMS Upload Vendor Yes Yes No TBD NA NA

NMMS Prodcut Training 16 Vendor Templates, LMS Upload Vendor Yes Yes No TBD NA NA

CRR Product Training 32 Vendor Templates, LMS Upload Vendor Yes Yes No TBD NA NA

ROME Product Training 40 Vendor Templates, LMS Upload Vendor No Yes No TBD NA NANodal Operating Guide Training 16 Collen - IRT Standard Process Nodal Training Yes Yes No May, 2007 NA NAControl Room Desk Procedure Training 20 OTS Trainer Standard Process OTS Trainer Yes Yes Yes Dec, 2007 NA NA

EMS Application Support Training 40 Vendor Templates, LMS Upload Vendor No Yes No TBD NA NA

MMS Application Support Training 40 Vendor Templates, LMS Upload Vendor No Yes No TBD NA NA

OTS Application Support Training 40 Vendor Templates, LMS Upload Vendor No Yes No TBD NA NA

NMMS Application Support Training 40 Vendor Templates, LMS Upload Vendor No Yes No TBD NA NA

CRR Application Support Training 40 Vendor Templates, LMS Upload Vendor No Yes No TBD NA NA

ROME Application Support Training 24 Vendor Templates, LMS Upload Vendor No Yes No TBD NA NA

Outage Scheduler Application Support Training 40 Vendor Templates, LMS Upload Vendor No Yes No TBD NA NAMIS Application Support Training 16 ERCOT team 25% Support ERCOT team No Yes No TBD NA NAEDW Application Support Training 16 ERCOT team 25% Support ERCOT team No Yes No TBD NA NAMP Registration Application Support Training 16 ERCOT team 25% Support ERCOT team No Yes No TBD NA NATIBCO / EAI Application Support Training 24 ERCOT team 25% Support ERCOT team No Yes No TBD NA NA


Scheduling Assumptions

• The available dates are based on the current EDS delivery schedule and phasing

(prior to Sept, 2006) and the Curriculum descriptions regarding Training Readiness.

• ERCOT Team refers to ERCOT staff and contractors who are developing an

application.

• Vendor assumes that the vendor developing the software is contracted to create

and deliver the courses (including tests) in approved format.

• Individual course delivery schedules are not currently scheduled other than for 101

The Basics (Typically delivery schedules are finalized 30 days before date

available, rooms are preliminarily assigned at the beginning of course

development).

• WebEx productions are currently not scheduled.

• All courses (other than LMP 201 which replaced LMP 101) will be hosted on LMS.

Market Education



July 17, 2006

Role of Account Management at ERCOTNodal Training Communication

Market Education


Proposed Nodal Training Site

http://nodal.ercot.com/training/index.html


Proposed Course Pages


Communication Plan

For course updates and other nodal training information there are

several ways is which information will be communicated.• Nodal training website• Nodal project news letter• Word of mouth• E-mail exploder lists – actual lists to be determined• During course delivery• TPTF presentations

Market Education



July 17, 2006

Role of Account Management at ERCOTCourse Development Process

Market Education


8 Hour Course Development Cycle – One Day Course

4 Days

12 Days

12 Days

24 Days

5 Days

12 Days

12 Days

54 Days

TPTF subgroup review of

abstract and outline

Publish Course

TPTF subgroup review of

preliminary module

BetaTPTF workshop

Interactive Visual

Training Map

Schedule Course

Design Course

Develop Preliminary

ModuleDevelop Course

Structure F.A.Q.

Document

Develop Quick

Guides

Compose WebEx

Materials

Migrate Materials to Web-Based Training

Deliver Courses

Market Education



July 17, 2006

Role of Account Management at ERCOTCurrent Nodal Training Budget and Plan Assumptions

Market Education


Nodal Training Planning and Budgeting Assumptions

• The training plan does not include the current ERCOT payments for WebEx sessions. ERCOT

departments are allowed to use WebEx sessions without specifying the actual charge in

budgets. WebEx sessions will not exceed 125 students per class session.

• The training plan assumes the use of the current LMS with the purchase of Lectora to provide a

testing platform.

• Any required self-study presenters are provided from resources already allocated to the

project.

• For budgeting purposes we have used the lowest estimated cost ($6 per booklet) for

production of the course booklets. This is not the current preferred ERCOT vendor for printing -

Document Engine – which has estimated three times this much. Printing is black & white only.

• Based on the Curriculum seven courses need to be transcribed for narration for web-based.

• The number of students trained per active course per month by face to face is estimated at 50.

• There are no charges included for ERCOT provided facility, software & supplies or for

internally printed material.



• The level of effort is based on market-focused courses identified by the Market

Participants in the “Curriculum" document, and those identified by the IRT group

as required for ERCOT Readiness. Any additional training at ERCOT would be

the responsibility of the particular department, though it is suggested that the

standard formats and process being used by the Nodal Training Project may be

of value. The market-focused course schedule and scope is to support the

requirements dictated by the TPTF as documented in the Curriculum documents.

Scaling of development effort is based on the course creation section of the

process.



• PJM training is not being purchased.

• Contractor travel expenses are included in vendor rates.

• Other than LMP 201, all courses are presented by ERCOT staff or Training contractors.

• The LMS contract is good for two years there is no guarantee of the same price from the

vendor – OnLine Training – for the 3rd year. The cost of software licenses includes est. 10%

sales tax.

• All scenario based training is one day.

• Nodal Program Office will provide a mechanism to monitor budget & time.

• Training external to Texas can be provided at time and material reimbursement.

• The cost of course delivery through 2009 is included in the Nodal Training budget with the

assumption of using internal ERCOT resources. No assumption is made as to the provision of

these resources. Approximately 14,000 hours have been budgeted using these internal

resources at $65 hour. Should these resources not be available cost of contractors for training

delivery could be as high as $165 an hour (including expenses). The current assumption is that

each course, once available, will be presented once a month through market opening.



• The Annual ERCOT Operations Training Seminar will include at least 2 Hours of nodal

material in the 2007 Seminar. The Seminar Working Group is expected to include more in

the Seminar to be held in the fall of 2008; how much is yet to be determined. Since the

2009 Seminar is to be held in the fall of 2009, post Nodal implementation, operational

implementation issues will be considered by the Seminar Working Group. Nodal Training

will provide support and material to the Seminar as required. The current Early Delivery

System (EDS) development timelines must consider all available options to prepare target

populations for Nodal implementation.

• The Extended Training – System Dynamics course is budgeted for development but not

delivery as delivery (according to the Curriculum) occurs after 1/1/09.

• The Nodal Training Budget does not include any activity or expenses after 1/1/09.


Training Travel Discussion

• Training external to Texas can be provided at time and material reimbursement.

• Training has currently budgeted for in-Texas training at four separate locations (North, South, East and West) at an ERCOT approved facility.– Market Participant support & travel is not included.

– Budget assumes course travel for three people per month in Texas, with two nights out (three days) at one of the four locations.

– Out-of-Austin facilities are assumed free and include projection and sound capability supporting an audience from 25-50 people.

• Laptop is purchased for training use in out-of-Austin facilities, includes tax, support, and warranty.

Market Education



July 17, 2006

Role of Account Management at ERCOTSuggestions for Reorganization of Nodal Training Curriculum for Budget Reduction

Market Education


Nodal Training Budgeting

Background• Can the scope be reduced

• Can we simplify the process

• Can the scale be reduced

• Can we reorganize the schedule

• Must maintain the quality and content

Perspective• Nodal 101 Training in production

• Development of five day Basic Training Course outline/abstract for QSEs

• Familiarity with other components of the Nodal Project


Courses with no suggestions for curriculum changes

• NOIE QSE Operations

• Basic Training Program

• Extended Training System Operations

• ERCOT LMP 201

• Market Settlements 301

• CRRs, PCRRs etc

• Support for Annual System Operations Seminar


Reduction of Face to Face Course Delivery

Rationale: The provision of Web-Based Training with electronic testing capability could provide the necessary coverage. Nodal Training will host a once-a-month question and answer conference-call for questions. In addition several of the course durations could be reduced.

Courses Impacted Savings

ERCOT Nodal 101 After 2006 $ 75,000

Transition to Nodal Markets $250,000• Reduce to 4 hours (from 16 hours) as content is largely project information that might

be provided in updates to TPTF and circulated on a regular basis. Course content still includes a description of the plans, major components and anticipated interaction points of the Market.

Transmission 101 $300,000• Reduce to 10 hours (from 20 hours) as content is very specific to TSPs and DSPs

and they are already required to attend the Basic Training QSE course per the Readiness discussions.


Consolidation of Generation Courses

Rationale: There are currently four courses that are generation focused for a total duration of 48 hours of classroom time. Based on the fact that this material will build on the Basic Training QSE course generation specific material could be provided over 16 hours of training (vastly reducing development and delivery time) with one possible format as follows:Day 1, Morning Generation 101

Day 1, Afternoon Generation 101, test 101

Day 1, Afternoon Generation 201

Day 2, Morning Generation 201, test 201

Day 2, Afternoon Generation 301, test 301

Day 2, Afternoon Wind Generation, test wind generation


Generation 101,201,301 $910,000

Wind Generation 101 $290,000


Reduction of Course Duration

Rationale: Based on the current anticipated depth of material being created for the

5 Day Basic Training Program, the following courses could have their duration

reduced.


Load Service Entity 201 $142,000

(Reduce from 20 to 16 hours)

Network Model Management $ 76,000

(Reduce from 16 to 8 hours)


Removal of courses from Nodal Training Plan and Budget

Rationale: This course can be scheduled for production and delivery after the

1/1/2009 date used for the Texas Nodal Training budget and plan.

Course Impacted Savings

Scenario-Based Training $737,000

Rationale: This course will be provided by the ERCOT Software Development

Team in the form of presentations.

Course Impacted Savings

MIS 101 $292,000


Additional Potential Savings

Item Impacted Savings

Remove WebEx $ 78,000

Remove Off-site Texas Training $ 90,000

Remove support for vendor supplied training $100,000

Drop Interactive Visual Training (Quality Impact) $100,000

Drop Process Maps (Quality Impact) $429,000

Drop Quick Guides (Quality Impact) $273,000

Drop FAQ (Quality Impact) $ 55,000


Budget Saving Summary

Items Savings

Curriculum Restructuring $3,072,000

No off-site Texas training $ 90,000

Drop WebEx $ 78,000

Drop Vendor Training Support $ 100,000

Reduce Training Quality $ 857,000

Market Education



July 17, 2006

Role of Account Management at ERCOTLMS Update

Market Education


• Tracking and reporting on Nodal Market Readiness

• Automation of as many processes as possible

• Support for distance learning

• MPs need to be able to manage their own students

• Mapping Market Protocols to learning objectives

• Support for NERC Certification and CEH

• Flexible and secure system

Business Drivers


Technology

Functional Requirement Categories

• User Accounts

• Publishing & Scheduling

• Student Enrollment

• Attendance & Tracking

• Reporting

• Certification

• Protocol Mapping

Prospective software

• WBT Manager (LMS), hosted by external vendor

• Lectora courseware and exam creation tools, hosted by external vendor

• Requisite Pro for protocol mapping


LMS Readiness Schedule

9/15 – Complete Requirements / UI Design

9/22 – Licensing & Software Procurement (dependency)

10/20 – Requirements, UI, & Template Design Approval (dependency)

10/20 – LMS Configuration Completed

11/10 – UI & Templates Completed

11/22 – System Testing & Data Migration Completed

12/22 – Flight Testing Completed

12/26 – Go Live


Proposed Features

Reporting:

• Reporting access provided at the overall system and business domain (MP) levels.

• Report generation on student performance:

– By student

– By Market Participant and Market Participant Division

– By Market Segment (QSE, LSE, etc.)

• For the market at large with a wide range of additional reports

Publishing & Scheduling:

• Organization of content by course and module

• Class schedule, instructor, and facility management tools

• Support for launching self-directed, electronic deliverables (courseware, exams, and

other course aids)

• Publicly accessible via the Internet


Proposed Features

User Account:

• MP-centric student hierarchy

• Local MP administrator support at both the company and division level

• Mandatory student account creation

• Self-service, managed, and bulk creation of accounts

• Self-service and managed account maintenance

• Self-service support for transition between Market Participants

Certification:

• Tracking of certification programs to facilitate NERC requirements

• Support for organizing curricula to meet certification program needs

• Application of training time towards student Continuing Educational Hours (CEH)


Proposed Features

Student Enrollment:

• Access to all course curricula via a global course catalog

• Self-service and managed student class enrollment

• Alerts to facilitate communication between Training and students

Attendance and Tracking Features:

• Manual attendance management for class sessions

• Automatic attendance management for self-service, distance learning

• Tracking of all AICC- or SCORM-compliant electronic courseware

• Exam data gathering, by question and learning objective

• Optional support for pre-requisite constraints on certain curricula

• Optional support for “testing out” of certain curricula


Proposed Features

Protocol Mapping:• Mapping of Market Protocols and derived Nodal system application features to

applicable learning objectives

Market Education



July 17, 2006

Role of Account Management at ERCOTNodal 101 Update

Market Education


ERCOT Nodal 101: The Basics – Evaluation Results

Four MP classes held (150 attendees, 90 surveys completed – 60%)

• July 26 – 37 attendees, 20 surveys completed

• July 27 – 33 attendees, 21 surveys completed

• Aug 8 – 35 attendees, 26 surveys completed

• Aug 23 – 45 attendees, 23 surveys completed

Survey asked attendees:

• Name (optional)

• Participant Type– QSE, LSE/REP, TDSP, Generation, PUCT, Other

• Role– Accounting/Settlements, Power Operations, Engineering, Management, Consultant,

Other

• Nodal Market Knowledge– Beginner, Intermediate, Advanced


ERCOT Nodal 101: The Basics – Surveys by Type

0% 5% 10% 15% 20% 25% 30% 35% 40% 45%

QSE

LSE/REP

TDSP

GEN

Other

26-Jul 27-Jul 8-Aug 23-Aug


ERCOT Nodal 101: The Basics – Surveys by Role

0% 5% 10% 15% 20% 25% 30% 35%

Acct/Settlements

PowerOp

Engineer

Management

Consultant

Other



ERCOT Nodal 101: The Basics – Prior Nodal Knowledge

0% 10% 20% 30% 40% 50% 60% 70%

Beginner

Intermediate

Advanced



ERCOT Nodal 101: The Basics – Topic Importance

Topics

• Market Overview• LMP• Adjustment Period and Real-

Time Operations• Day-Ahead Operations• Congestion Revenue Rights• Settlements

Participants asked to rate each topic using the following scale:

• No Opinion (0 – Not Counted)• Very Unimportant (1)• Unimportant (2)• Important (3)• Very Important (4)

All topics received ratings between 3.02 and 3.44


ERCOT Nodal 101: The Basics – Content/Presentation Ratings

Adapted to reflect generic NERC survey

• Met learning objectives• Prerequisites were appropriate• Materials were accurate and

relevant• Instructional method was

appropriate• Time allotted was appropriate• Instructor(s) were effective• Facilities and/or tech equipment

were appropriate• Materials issued before/during

training were appropriate• A/V used were effective• Recommend course to others

Participants asked to rate each item using the following scale:

• No Opinion (0 – Not Counted)• Strongly Disagree (1)• Disagree (2)• Agree (3)• Strongly Agree (4)

All categories received ratings between 3.16 and 3.48 except

Time Allotted - 2.86 (Large amount of new material)


ERCOT Nodal 101: The Basics – Test Procedures

• Attendees were told that they need to take the test, this is now documented on the web site and in the course materials.

– 150 attendees– 87 took test (58% of those who attended)– 69 passed (79% of those who took the test)

• ERCOT Attendees– 26 attendees– 10 (38%) took test and all passed

• Test was open book/notes

• Multiple choice questions– Some questions removed as a result of 7/26 and 7/27 classes

• Passing score = 70%

• “Test out” option to be in place Q4 2006 when LMS is operational


ERCOT Nodal 101: The Basics – Intro Slide on Testing

• Market Participant – An Entity that engages in any activity that is in whole or in part the subject of the Nodal Protocols, regardless of whether that Entity has signed an Agreement with ERCOT.

• Market Participants are required as part of the Nodal Transition Plan to demonstrate Market Readiness. – Training is considered one component of Market Readiness.

– As such each Market Participant, as defined above, will be expected to take the test for a given course.

– ERCOT will maintain such information and report to the Market as required on the status of Training Readiness.


ERCOT Nodal 101: The Basics – Test Performance, all Attendees

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

Passed

Failed



ERCOT Nodal 101: The Basics – 2006 Face to Face Schedule

• September 21, 2006 in Austin• October 11, 2006 in Taylor• October 25, 2006 in Austin• November 7, 2006 in Austin• November 29, 2006 in Taylor

Market Education



July 17, 2006

Role of Account Management at ERCOTLMP 201 Update

Market Education


Summary of comments

• Very strong content that was broader than LMP, include a lot of basic supply and demand theory.

• Had some material that was commentary on other markets that could be removed.

• Material delivered as if in a university – Professor Baldick presents the information without facilitating the learning of the information.

• Professor Baldick is highly knowledgeable about the subject matter.• Professor Baldick has certain opinions that are not necessarily those of

ERCOT and is not afraid to voice them.• Audience did not perceive this as a 200 level course.• Material was in Professor Baldick’s own format.• Recommendation from an audience member was that the course

should be mandatory.


Decisions required for LMP 201

• Should this course be included in the “Nodal Training” Curriculum in standard format with or without test as a replacement for LMP 101?– Professor Baldick owns the material completely.

• Should this course be mandatory and a prerequisite to Nodal Basics 101?

• Should this course be called “The Economics of LMP”?

Market Education



July 17, 2006

Role of Account Management at ERCOTWork in Progress

Market Education


Accomplishments and On-going Work

Recent Accomplishments• Finalization of ERCOT Nodal 101 Development including quick guides• Transition of delivery of Nodal 101 to Wholesale Client Services• Completion of initial budget and plans• Beta Run of LMP 201 • Training development standards• Completion of Protocol Mapping

On-going Work• Finalization of LMP 201 • Development of 5-Day Basics Training Program

– Release of abstract and outline to TPTF Sub-Group 9/15/2006

• Development of Interactive Visual Process Map has commenced• Completion of LMS requirements validation• Development of Transition to Nodal Testing

– Release of abstract and outline to TPTF Sub-Group 10/14/2006

Documents

Role of Account Management at ERCOT Nodal Training Project Debrief for TPTF September 12, 2006 Market Education