Market Trials

Weekly Update – Friday MeetingNovember 22, 2013

Agenda• IDT Preliminary Debrief: 9:00 – 10:00A

• Analysis: 10:00 – 10:20A

• Scenario Analysis

• Look-Ahead: 10:20 – 10:40A

• Announcements

• Release Updates

• Look-Ahead

• Updates• Release Scorecard• Migration Report• Known Issues Log

• Settlements: 10:40 – 11:00A

• CWG Checkpoint: 11:00 – 11:45A

2

Announcements

3

Please submit IDT ‘Management’ suggestions via RMS•Next IDT is Dec 3 (Prep Call Dec 2).

IDT Reports•IDT reports to be posted to spp.org here 7 calendar days after each IDT

spp.org > Integrated Marketplace > Market Trials > Integrated Deployment Test

– Reports will be “summarized” on Friday calls one week in arrears

Ginny Watson

Test Date Report Date Review Date

11/19/13 11/26/13 12/6/13 (Holiday 11/29)

11/21/13 11/28/13 12/6/13 (Holiday 11/29)

12/3/13 12/10/13 12/13/13

12/5/13 12/12/13 12/13/13

And so on…

Deployment Test Reminders• Please follow IM setpoints

– 11/21: NDVERs and DVERs chasing prices 11/21 IDT timeframe was selected because load was flat and NSI changes

are minimum. SPP systems worked correctly but, ACE oscillation caused by Resources who do not follow IM setpoints

During the 12/3 IDT, SPP expects to get load pick up; NDVERs and DVERs need to not run at all or make every effort follow IM setpoints

• BAs should not be following or concerned about your ACE Please ensure systems are following IM setpoints and not trying to control

your own ACE

4CJ Brown

Deployment Test Follow Up• ICCP

– DVERs: Can receive follow flag for economic reasons. Please follow it.

Additionally, if DVERS are being “OOMEd,” they will receive a corresponding follow flag, which we except you to follow. DVERs will always receive OOME and follow flag at the same time.

– NDVERs: No such thing as a follow flag

If you are being “OOMEd”, you will receive an OOME only

5Carrie Simpson

11-19-13 Deployment Test Debrief• Overall the First Deployment test was a great success

– Overall SPP BA ACE and frequency were in acceptable ranges throughout the duration of the test

– EIS BA’s had no problems switching from EIS setpoints to IM setpoints and back again – this was the first time this was fully tested

– No major problems were reported by SPP or the MPs during the test

– Able to start and stop the test on time as scheduled with no delay, reduction or extension

6

CJ Brown

11-19-13 Deployment Test Debrief• Though a “Success” there are things we can improve upon

– MATCHING EIS and IM Offers MPs need to start matching offers 4 hours prior to IDTs to allow time to work

out any mismatches

Up to 30 Minutes prior to the operating hour of the IDT, SPP was still working with MPs to get the A/S offers corrected to assure there was enough Regulation and Operating Reserves available during the test

Regulation needs to be provided by the same Resource it was offered into the EIS market on

– Coordinating the cutover better Some MPs switched to IM setpoints too early – SPP will communicate the

cutover timing more clearly

The desire for every IDT is to have the EIS BA’s and MP’s with Resources to be following IM setpoints by the start of the first five-minute interval of the test

7CJ Brown

11-19-13 Deployment Test Debrief• Though a “Success” there are things we can improve upon

– Minor MP problems were addressed quickly and didn’t create any reliability concerns Need to assure that MPs know what desk to call for what questions

Added additional phones to make sure we can take the call volume of changes during deployment tests

– 4 RTBM intervals solved but timed out before being approved System loading is believed to be part of the root cause. Changes have been

made to help address this issue. Issue #504 on the Known Issues Log.

RTGEN continued to issue setpoints based on the last good RTBM regulating for ACE. This worked as designed.

MPs need to know what to do in the case of a failed RTBM interval – they should follow setpoints – During the IDT some MPs reported getting a “zero” XML notifications where they should have not received any updates

8CJ Brown

11-19-13 Deployment Test Debrief• Though a “Success” there are things we can improve upon

– Some MPs had XML Listener problems Need to assure that MPs check them early in the day then again right

before the tests

– One participant had to manually enter setpoints for the first hour of the test due to problems with their EMS Lead to a slight lag in managing ACE, but didn’t cause any excursions

– MPs with DVERs need to know that if they are dispatched down by the IM setpoints they are expected to follow that setpoint

– There were 59 non-wind Resources that had turnaround ramp rate of 0.1 or less – we need to discuss relaxing this for subsequent tests

9CJ Brown

11-21-13 Deployment Test Debrief• Overall the Second Deployment test was also a great success

– Overall SPP BA ACE and frequency were in acceptable ranges throughout the duration of the test

– All RTBM intervals solved and were approved on time

– No major problems were reported by SPP or the MPs during the test

– Able to start and stop the test on time as scheduled with no delay, reduction or extension

10CJ Brown

11-21-13 Deployment Test Debrief• Though a “Success” there are things we can improve upon

– MPs did much better matching EIS and IM Offers – much improved from the 1st IDT but can still get better

– Flowgate loading was negatively affected by DVERs who were responding to EIS price signals – once this was communicated during the test it improved quickly

– SPP’s Control room displays need some fine tuning

– Had a single EIS BA that had some problems following IM regulation setpoints with three EIS BAs reporting some type of EMS issue during the test

11CJ Brown

Preliminary Results

12

• Scenario 7 Must Offer*Scenario document on spp.org herespp.org > Integrated Marketplace > Market Trials > Parallel Operations Test> Integrated Marketplace Parallel Operations Project Documents > Scenarios in Parallel Ops

• From initial analysis it appears that most MPs were short of their demand bids and were in emergency conditions that should trigger Must Offer penalties

• From here, it is a manual process for the Market Monitor to determine if they were actually short and send out all of the notifications

• SPP filed tariff language on November 15th to reflect the determination of Day-Ahead Must Offer compliance at the Asset Owner level. This is important change and SPP will reflect this change in Must Offer Scenario 7. The impact of this change is that the calculations of Net Resource Capacity, Reported Load, and Reserve Obligation will all be made at the Asset Owner level rather than the Market Participant level.

• Other updates in the document reflect the timing of the metered load data. While this data is available in the SPP Settlements system on OD+4, it is not available to the MMU until OD+8, after the issuance of initial statements. The updates in Section 1.2 reflect a shift of four days in the expected timing of the Scenario.

John Hyatt

Preliminary Results

13

• Scenario 28a Nonfunctional Testing: CMS Outage CMS was disabled at 10:45 for about 15 or 20 minutes while the MPs continued to

submit virtual transactions from the Markets User Interface.

When CMS was disabled, the credit check from Markets was disabled.

When the scenario was over, SPP brought CMS back online, and the DA Markets Desk approved the pending transactions and re-enabled the credit check from the Markets side. That triggered the auto-approve and re-synch process between Markets and CMS, and all processed successfully.

9 virtual transactions were pending credit approval for 6 asset owners. All 9 transactions approved with the operator manual approval action.

The manual notifications were sent successfully

There is a known defect with Enable Link button command. SPP believes this should be corrected with the next PRD release.

For Reference Only

14

• Summary – Scenarios in Parallel Ops:– 1 Scenario and 1 Partial Scenario Executed to date:

Scenario 7 Must Offer pending Settlements results

Scenario 28a Successful

• Summary – Retests Pending:– 14 individual retest requests pending

1 MP retesting Scenario 17.3 on 11/25

6 MPs requested retest for Scenario 34

3 MPs requested retest for Scenario 2

1 MP requested retest for Scenario 3

2 MPs requested retest for Scenario 12 (SPP waiting on MPs)

1 MP requested retest for Scenario 33 (SPP waiting on MP)

Scenario Tracker as of Nov 21

Alan Adams

Announcements

15

Modeling Reports•The following reports were available on spp.org during SMT and have been moved permanently to the Marketplace portal:

marketplace.spp.org > Public > Marketplace > Modeling ReportsSL to Pnode to Zone Mapping SL EIR Mapping SMT FAQ SMT Quick Facts

Scheduling Changes•3 Hour IDT Thursday, Dec 12 5:30 – 10:30A CPT (IDT Test 6:30 – 9:30A)

Actual test is now 3 hours instead of 4 hours

•Web mirroring turns off Dec 16 For details, Reference Market Trials Meeting_20131115 on spp.org.

spp.org > Integrated Marketplace > Market Trials > Parallel Operations Test> Integrated Marketplace Parallel Operations Project Documents > Parallel Ops Presentations, Statistics, MP Meetings

Ginny Watson

Announcements

16

IDT Reports•IDT reports to be posted to spp.org here 7 days after each IDT

spp.org > Integrated Marketplace > Market Trials > Integrated Deployment Test

– Reports will be “summarized” on Friday calls one week in arrears

Ginny Watson

Test Date Report Date Review Date

11/19/13 11/26/13 12/6/13 (Holiday 11/29)

11/21/13 11/28/13 12/6/13 (Holiday 11/29)

12/3/13 12/10/13 12/13/13

12/5/13 12/12/13 12/13/13

And so on…

Announcements

17

• Settlements Calendar marketplace.spp.org > Public > Settlements > Calendar

– The Settlements Parallel Operations calendar on the portal has been updated to reflect the accelerated posting schedule (OD+14)for finals during parallel operations. The accelerated meter and BSS submittal timelines have also been updated accordingly.

Ginny Watson

Look-Ahead: Nov 25 – Dec 1

18

• Market Trials Hotline closes Wednesday Nov 27• Contact the SPP desks (501-614-3900) critical or high issues• DA, RUC, RTBM , Balancing desks supported 24 x 7. See the following slide.

• Parallel Ops Calendar on spp.orgspp.org > Integrated Marketplace > Market Trials > Parallel Operations Test> Integrated Marketplace Parallel Operations Project Documents• Settlements details on the Settlements Calendar marketplace.spp.org > Public > Settlements > Calendars

• Mon Nov 25 for OD Tues Nov 26: Scenario 15 – Commitment Mismatch• Thurs Nov 28 & Fri Nov 29: SPP Holidays. No SPP staff running DA Market or DA

RUC. Quality of solutions may be less than a normal supported day.

DA Market and DA RUC: run studies on Wed 11/27 for Thurs 11/28, Fri 11/29ID-RUC (COP True-UP): run at least every 4 hoursRTBM: maintain as normalRTGEN/EMS: maintain as normal

Ginny Watson

Parallel Ops / IDT Communication Plan

19Ginny Watson

Availability

501-614-3900

Call Option When should I call a specific desk?

Balancing Desk 24x7 2

I have any real time issues or questions concerning NSI, Tie Lines or Frequency.I have a real time Load related issue that might impact capacity or operating reserves.I have an ICCP real time related issues.I have a Real Time Reserve Sharing Event questions.Any activities of EIS BAs that could impact SPP BA.Events that impact real time reporting requirements.

RTBM Desk 24x7 5

I have a real time Resource related issue that might impact capacity, operating reserves and/or economic dispatch.I have any real time operational discrepancies concerning economic dispatch (XML dispatch, resource plan information, AS offer information).

Tariff and Interchang

e Desk8am-5pm

Monday-Friday 6

My tags are getting denied but I have to have them in by a certain deadline.My transmission requests are getting denied but I have to have them in by a certain deadline.

RUC Desk 24x7 7

I have a unit commitment related question or information to communicate to SPP (new commitments, changes to existing commitments, startup issues, shutdown issues).I have a real time Contingency Reserve Deployment question/issue.

DA Desk6am-6pm

7 days a week 8I can't get my DA Bid Offer in by a certain deadline.I have a question concerning the current day's DA Market clearing process.

Reliability Coordinati

on DeskDuring IDTs

Only N/AFor Reliability Coordination issues during IDT the IDT conference calls and normal process will be leveraged.

Scenario 15 pt 2 Commitment Mismatch

20

*Scenario document on spp.org herespp.org > Integrated Marketplace > Market Trials > Parallel Operations Test> Integrated Marketplace Parallel Operations Project Documents > Scenarios in Parallel Ops

Scenario Description• A commitment mismatch occurs when a resource outaged in the RTBM commit

status conflicts with the COP status for the resource as committed by the DA market.

• When there is a commitment mismatch between RTBM commit status and the COP, the RTBM commit status outage will prevail in the RUC solution

Expected Result• Outaged resources will not be committed in the RUC process for the outaged

intervals.

• SPP operators will update the COP

• MPs should receive a STOP notification based on when the Resource is to be outaged.

Alan Adams

Scenario 15 pt 2 Commitment Mismatch

21

MP Activities: Section 2.2. “Outage Commit Status• MPs must submit a commit status of “Self” in the DA Market offer to ensure a

resource will be committed in the DA Market and therefore be committed in the COP during the DARUC process

• MPs will submit a Commit Status of “Outage” in the RTBM offer prior to the execution of DARUC (1700) on the resource that cleared in the DA Market

• MPs will need to review the START/STOP notifications to verify that the resources with a Commit Status of “Outage” received a STOP notification for the beginning hour of the Outage period

Please ensure that for Operating Day 11/26, the designated Resource has a Commit Status of “SELF” in the DAMKT Offer. On 11/25, after the closing of the DAMKT, update the RTBM Offer for OD 11/26 to have a Commit Status of “Outage”.

Alan Adams

Schedule

22

• Nov 25 – Dec 1Nov 26: Scenario 15 Commitment Mismatch Part 2Nov 28: SPP HolidayNov 29: SPP Holiday

• Dec 2 - 8Dec 2: IDT Prep Call 3:00PDec 3: IDT 5:30A (test begins at 6:30A)Dec 4: IDT Debrief and Prep Call 3:00PDec 5: IDT Call 6:30A (test begins at 7:30A)Dec 6: IDT Debrief and Market Trials Call 9:00A

Ginny Watson

Migration and Maintenance Report

23

Migration and Maintenance Report posted to spp.orgspp.org > Integrated Marketplace > Market Trials > Parallel Operations Test> Integrated Marketplace Parallel Operations Project Documents

Annette Holbert

24

Release Scorecard posted to spp.org:spp.org > Integrated Marketplace > Market Trials > Parallel Operations Test> Integrated Marketplace Parallel Operations Project Documents

Release Scorecard

Ginny Watson

Known Issues Log

25

• Please review the Market Trials Known Issues Log on spp.orgspp.org > Integrated Marketplace > Market Trials > Structured/Unstructured Test > Integrated Marketplace Market Trials Project Documentation Folder

Carrie Simpson

Issue # Component Synopsis Severity

827Markets

UI

When the Reserve Requirement is recalculated within the same day and a MP queries for their Reserve Requirement, they will receive all entries. The code should be corrected so that only the most recent entry is pulled.

Severity 3: Medium

Market Trials Useful ResourcesIntegrated Marketplace Parallel Operations Project Documents spp.org > Integrated Marketplace > Market Trials > Parallel Operations Test> Integrated Marketplace Parallel Operations Project Documents

•Parallel Ops Presentations, Statistics, MP Meetings Folder•Parallel Ops Reference Documents Folder•Scenarios in Parallel Ops Folder > Scenarios Schedule•Parallel Ops Communication Plan•Parallel Ops MP Guide•Parallel Ops Calendar•Known Issues Log•Release Scorecard•Migration Report

Integrated Deployment Test (as part of Parallel Ops)spp.org > Integrated Marketplace > Market Trials > Integrated Deployment Test

•IDT MP Guide•Parallel Ops Calendar

Additional documents:•Settlements Calendar marketplace.spp.org > Public > Settlements > Calendars•Change Tracker spp.org > Integrated Marketplace > Marketplace Documents•Integrated Marketplace Structured/Unstructured Test Project Documents spp.org > Integrated Marketplace > Market Trials > Structured / Unstructured Test> Integrated Marketplace Structured / Unstructured Test Project Documents

26

Settlements Status

• Transition from SMT to Parallel Operations was executed on schedule

• The first Operating Day of Parallel Operations (11/12) was posted on schedule

• The accelerated posting schedule for Finals and Resettlements was discussed at the CWG and this week’s SUG. The recommendation was accepted so that is out plan moving forward.

• The revised Parallel Operations calendar reflecting the change was posted yesterday.

27 Brent

Wilcox

Market Settlement System (MSS) Known Issues

• KIL# 828

• The invoice date is incorrect. On the Bill determinant report the invoice date matched the operating day---this is not possible. – This is expected to be fixed in January.

28 Brent

Wilcox

Market Settlement System (MSS) Known Issues

• KIL# 838

• The Common Bus Flag is not being correctly pulled into ETSE . This will affect any determinants or flags that use the commonbusflg on OD 11/12/13. (i.e XmptDev5minFlg, RtCRDeplFailAmt.)

This is expected to be fixed in January.

29 Brent

Wilcox

Market Settlement System (MSS) Known Issues

• KIL# 840

• Two decommitment records are being received in table for one commitment period. – Change to POPS so that every decommit has a

corresponding commit

– A message is needed to WARN analyst that a resource could be re-commited and de-commited again and to check for this possibility

This is expected to be fixed in January.

30 Brent

Wilcox

Market Settlement System (MSS) Known Issues

• KIL# 841

• The view that is used to populate the customer info (address, long name, etc.) on the invoice pdf needs to be modified to check the effective date of the mp in the spp_cust_def table in case the mp is in that table multiple times w/ different effective dates.This is expected to be fixed in January.– This is expected to be fixed in January.

31 Brent

Wilcox

Market Settlement System (MSS) Known Issues

• KIL# 842

• Intermittent schema validation issues are occurring when submitting meter data files. MPs may receive error message. Resubmitting usually works fine.– This is expected to be fixed in January.

32

Brent Wilcox

Parallel Operations /Go Live Checkpoint

November 22, 2013

MARKET RELEASE UPDATESCarrie Simpson

• MOS 1.8– Standard core regression testing against 1.8 will wrap up early week of 11/25– Full regression of UI and API is wrapping up – MCE reporting defects found in Certification Testing have been retested– Regression tested the FIT interface improvements with 1.8

• Increased Functionality with 1.8– Wind Resource Forecast displays in MUI (subject to NDA)– Notifications on Secondary Listeners– Special resource logic fixes for BDRs, DVERs, and EDRs– MOI operator usability improvements– Market Control and stability improvements

• FIT Testing – 94% of Markets workstream FIT test cases have passed

Remaining FIT testing is specific commercial model and TLR Level 2 WAUE name issues have been resolved

35

Markets Test Execution

• MOS 1.9 will include:– FERC Mitigation changes– Emergency Logic fixes– Increased UI and API usability– Additional Market Control, MDB and performance updates

• MOS 1.9 Test Execution:– Planned full MCE regression test– MMU team will conduct full regression against revised Mitigation functionality

• Release Schedule:– Delivery to SPP scheduled for Wednesday 11/27– PROD promotion scheduled for Friday 12/20

36

MOS 1.9 Release

SYSTEM FREEZE UPDATESam Ellis

System Freeze Timeline Update

38

MP READINESS CERTIFICATION & SPP READINESS FILING UPDATE

Erin Jester

Timeline: Integrated Marketplace Readiness Filing

40

Readiness Filing Next Steps • SPP’s Integrated Marketplace Readiness Certification

– Filing Due Date: December 27, 2013

– SPP’s Assessment of Operational Readiness

– Market Readiness Metrics

– Status of BA Certification

41

MP Certification Form Process Completion of the MP Readiness Certification is an official Readiness Metric filed with FERC (RGL-03)

•A letter was sent to MOPC Representative (or Executive Sponsor) of each MP on 11/18 to explain the timeline and intent of the MP Readiness Certification

•The official MP Readiness Certification letter and form was sent on 11/20 via RMS to the Readiness Liaison for each MP

– If no Readiness Liaison, then it will be assigned to the CWG Representative

– If no CWG Representative, it will be assigned to the Market Trials Liaison or Registration contact

•Please return the form signed by an authorized signatory of your company as a hard copy by December 20, 2013

– A scanned copy can be emailed in order to make the deadline; a hard copy must be sent after

42

READINESS METRICS UPDATEErin Jester

Color Description

Grey Monitoring of the metric has not begun or not applicable

Blue The metric success criteria has been met and no longer requires monitoring, and/or the metric has been completed by the target due date

Green The metric is meeting identified success criteria, and/or The metric is on schedule to meet the target due date

Yellow The metric has not met the highest level of the identified success criteria, and/orThe metric is at risk not meeting the identified target due date

Red The metric has not met the highest or mid-range success criteria as defined, and/or the metric has not been completed by the specified target due date

• Total Number of Readiness Metrics: 50

• Number Complete (Blue): 9

• Number on Track (Green): 31

• Number at Risk (Yellow/Red): 8

• Number Not Started (Gray): 2

Readiness Metrics Overview

44

Parallel Operations Readiness MetricsID Metric Name Status Variance

OPS-02 State Estimator Availability YSE not solved for certain times of the day due to network issues, which will be patched after further resting

OPS-03 State Estimator Accuracy GState Estimator solves 90% of the time with Mismatch level lower than 50 MW for 7 consecutive days prior to Parallel Ops

OPS-04 STLF Accuracy G

Short term load forecast for the SPP market footprint used for RTBM for conforming loads solves 97% of time with +/- 1% accuracy for seven consecutive days prior to Parallel Ops

OPS-05a MTLF - 4AM ID RUC G

4am Mid-term load forecast for conforming loads for the SPP market footprint solves 97% of time with +/- 3% accuracy for seven consecutive days prior to Parallel Ops

OPS-05b MTLF - 12PM ID RUC G

4am Mid-term load forecast for conforming loads for the SPP market footprint solves 97% of time with +/- 3% accuracy for seven consecutive days prior to Parallel Ops

OPS-05c MTLF - 5PM DA RUC G

4am Mid-term load forecast for conforming loads for the SPP market footprint solves 97% of time with +/- 3% accuracy for seven consecutive days prior to Parallel Ops

45

Parallel Operations Readiness Metrics

46

ID Metric Name Status Variance

OPS-06a Wind Forecast Accuracy - Pre RTBM G

Wind forecast for SPP Market footprint solves with +/-15% accuracy for six out of seven consecutive operating days prior to Parallel Ops

OPS-06b Wind Forecast Accuracy - RTBM GWind forecast for SPP Market footprint solves with +/-10% accuracy for six out of seven consecutive operating days prior to Parallel Ops

OPS-06c Wind Forecast Accuracy - DA RUC YWindfarms in EIS that are not reflected in the IM forecast must be curtailed/directed down. Weather events also cause data to fluctuate

OPS-07 Market Flow Calculations Availability - Current Hour G

Market Flow Calculations successful ninety-seven percent (97%) of 5 minute solutions for seven consecutive days prior to Parallel Ops

OPS-08 Market Flow Calculations Availability - Next Hour G

Market Flow Calculations successful ninety-seven percent (97%) of 5 minute solutions for seven consecutive days prior to Parallel Ops

OPS-09 DA Market Results Posted GDay-Ahead Market results are posted every day by 4pm for 5 consecutive days prior to Parallel Ops

Parallel Operations Readiness Metrics

47

ID Metric Name Status Variance

OPS-10 Generate DA LMPs and Settlement Point Prices G

LMPs are made available for each settlement location every day by 4pm for 5 consecutive days prior to Parallel Ops

OPS-11 RTBM Calculation Availability Y

Real-Time Calculations are not successful ninety-nine percent (99%) of five (5) minute solutions for 7 consecutive days prior to Parallel Ops. (93.8% on 11/10 and 97.9% on 11/16)

OPS-12 Operating Reserve Requirements G Operating Reserves are calculated and posted on time for 7 consecutive days prior to Parallel Ops

OPS-13 Regulation Pre-Certification G Milestones are tracking on time

OPS-14 ACE Performance Accuracy R Data, staffing and testing issues causing this metric to pass only 35% of the intervals

OPS-15 CROW Outage Submission R19/20 of resource-owning MPs have not reported all outages through SCADA and CROW within 30 minutes

Parallel Operations Readiness Metrics

48

ID Metric Name Status Variance

OPS-16 MP Response to Deployment Testing 12/3/2013 Metric not started

OPS-17 ICCP Model Validation Y 7/41 of MPs ICCP models are not validated due to not providing SPP with point name and models

OPS-18 ICCP Control Status Tracking R

61.3% of all MP's control modes submitted correctly between 11/4 and 11/8. Communications have been made to MPs regarding submitting control modes correctly.

RSG-01 RSG Member Integration and Testing Complete G Milestones are tracking on time

TRL-01 Market Trials Participation G 90% - 100% of Target Mass is participating in Market Trials

TRL-02 Structured/Unstructured Test Complete Y

November: 2/3 scenarios passed (28A, 33B passed, 35 failed and will be retested in the vendor DST environment)

Parallel Operations Readiness Metrics

49

ID Metric Name Status Variance

TRL-03 Market Trials - Integrated Deployment Testing G The most recently executive Integrated

Deployment Test (11/19) was successful

CBA-01 CBA Certification Activities G Milestones are tracking on time

CON-01 TCR Accelerated Connectivity Test Complete C TCR Accelerated Connectivity Testing Exit Criteria

has been met

CON-02 Connectivity Test Complete C Target Mass Passed Connectivity Testing

CON-03 Data Exchange Certification Complete C Data Exchange Passed for Target Mass

INT-01 Process & Procedures G All Process Documentation is now on track

Parallel Operations Readiness Metrics

50

ID Metric Name Status Variance

INT-02 Organizational Staffing Readiness G All Integrated Marketplace positions have been filled within 90 days of the due date.

INT-03 SPP Internal Training G96% of SPP staff performing Integrated Marketplace functions have no incomplete training requirements within 30 days of due date

INT-04 SPP Operations Training G 96% of SPP Operations staff have no incomplete training requirements within 30 days of due date

REG-01 MP Registration - SPP Action C 100% Target Mass are modeled

REG-02 MP Registration - MP Action G Milestones are tracking on time for 100% of Target Mass

RGL-01 FERC Tariff Filing G Milestones are tracking on time

Parallel Operations Readiness Metrics

51

ID Metric Name Status Variance

RGL-02 Readiness & Reversion Plan G Milestones are tracking on time

RGL-03 MP Readiness Certification G Milestones are tracking on time for 100% of Target Mass

TCR-01 Validate Annual TCR Auction Timeline C The TCR annual auction results are posted on

time for phase two of TCR Market Trials

TCR-02 Validate Monthly TCR Auction Timeline C

Post results according to established timeline (includes interim posting dates for Verification and ARR Allocation steps)

TCR-03 TCR Production - Abbreviated Annual Auction G

Post results according to established timeline (includes posting dates for Verification, ARR Allocation, and TCR Auction steps)

TCR-04 TCR Production - Two Monthly Auctions 2/14/2014 Metric not started

Parallel Operations Readiness Metrics

52

ID Metric Name Status Variance

TCR-05 TCR Monthly Trials Participation C> 90% of “High Priority” MPs and over 50% of candidate ARRs nominated and participated in the TCR auction either in annual or monthly

TCR-06 TCR Monthly Trials October Monthly Participation C 100% of “High Priority” MPs participated in the

TCR auction per stated intention to participate

STL-01 Access & Download Statements G 100% of Target Mass were able to successfully download a Statement

STL-02 Access & Download Invoices G 100% of Target Mass were able to successfully download an Invoice

STL-03 Submit Meter Data G 100% of meter agents successfully submit meter data for each of their assets

STL-05 Submit Dispute G 100% of Target Mass successfully submitted a dispute

STL-06 Settlement Statement Timeliness GSuccessfully posted Settlement Statements on time, reliably and consistently over Market Trials/Parallel Operations

STL-07 SPP Validation of Settlement Charge Types C

100% of Charge Types with calculated results were utilized by Target Mass. Charge Types with calculated results not utilized by Target Mass are explainable.

PARALLEL OPS EXIT & MARKETPLACE GO LIVE CRITERIA

Erin Jester

Parallel Operations Exit and Go Live Criteria

• 21 DRAFT Go Live Criteria

• 28 DRAFT Parallel Operations Exit Criteria

• Please provide feedback via RMS by December 6, 2013– Be sure to use the “Readiness & Metrics” Quick Pick

• Posted on Readiness Center in the Go/No-Go Criteria Documents folder

• Updates will be posted weekly and reviewed during Checkpoints

54

Future CheckpointsCWG checkpoints are posted on the Parallel Operations Calendar

•December– Friday, December 13

– Friday, December 20 (CWG)

•January− Friday, January 3

− Wednesday, January 8

− Friday, January 17

− Friday, January 24 (CWG)

− Friday, January 31

55

• February− Friday, February 7

− Friday, February 14 (CWG)

− Friday, February 21