ERCOT Lodestar Extracts Timing Recommendation ERCOT May 24, 2005

ERCOT Lodestar Extracts Timing Recommendation

ERCOT

May 24, 2005

Overview

Lodestar Extracts Current Extract Process for Lodestar Extracts from Data Archive ERCOT Recommended Process for Lodestar Extracts from ODS ERCOT Recommended Timing Process for Lodestar Extracts from ODS

Lodestar Extracts

SCR 727 ESIID Extract Resource ID Extract (New) Load Extract Generation Extract Settlements & Billing Extract Settlement Input Data Extract (New) Shadow Pricing Extract Public Reference Table Extract (New)

SCR 727 ESIID Extracts Current Process from Data Archive

Data Type: ESIID level data

Data Flow Process: At 5:00AM, Lodestar Data Archive process runs to capture the existing records with addtimes/timestamps of previous day 4:00AM through current day 3:59:59AM

Frequency: Generation and posting of extract occurs daily and is posted to market participants after being audited

Extract Creation Trigger: Automatically by data inserts/updates/deletes

Load, Generation, and Settlements & Billing Extracts Current Process from Data Archive

Data Type:– MOS and bill determinant trade date (Settlement Input Data) data

as early as two days after operating day and may receive data changes up until the initial settlement

– Data Aggregation and Settlement interval data outputs


Frequency: Extract process occurs Tuesday – Saturday only and is modified as necessary for holidays

Extract Creation Trigger: By market participants scheduling the extract; Not dependent on Data Archive Process

Load, Generation, and Settlements & Billing Extracts Current Process from Data Archive – con’t

Known Reasons for Improvement:– Potential for trade date outputs to be contained in multiple extracts– No dependency on settlement trade date approvals allowing trade

date outputs to be received in multiple extracts with different addtimes/timestamps to account for backouts/reruns of unapproved settlements prior to posting an approved statement

– Inconsistent synchronization of SCR 727 ESIID Extract data in conjunction with corresponding Load & Generation Extract postings

Shadow Pricing ExtractsCurrent Process from Data Archive

Data Type: Contains shadow pricing data


Frequency: Extract process occurs Tuesday – Saturday only and is modified as necessary for holidays

Extract Creation Trigger: Automatically by data inserts

SCR 727 ESIID Extracts and Resource ID Extracts ERCOT Recommended Process from ODS

Data Type: – SCR 727 ESIID Extract contains ESIID level data (granular data)– Resource ID Extract contains Resource ID level data (granular

data) - New

Data Flow Process: Continuous data flow from the Lodestar Source System -> Replicated Source System -> Lodestar ODS capturing intraday data updates and point in time selectivity



Recommended Changes:– Data load time window for the granular data Point in Time is

captured by the initiation of granular data changes (Lodestar Batch usage data loading) at SYSDATE -3 to the next day’s initiation of granular data changes (Lodestar Batch usage data loading)

– Produce on a 2 day lag from the data load time window stop time to accommodate for extract trade date synchronization and Lodestar ODS data synchronization including scheduled & unscheduled environment maintenance and troubleshooting data & system issues

– Data included within daily extract is synchronized to one posting of a complete batch of trade dates for the Load, Generation and Settlements & Billing Extracts

Timing proposal discussed with DEWG at February - May 2005 meetings

SCR 727 ESIID Extracts and Resource ID Extracts ERCOT Recommended Process from ODS – con’t

SCR 727 ESIID Extract and Resource ID Extract Timing Change

Day Extract is Generated and

Published

CurrentData Addtime Time Window

RecommendationData Addtime Time Window

Monday Sunday 4:00AM - Monday 3:59:59AM Friday ~4:00:00 AM – Saturday ~3:59:59 AM

Tuesday Monday 4:00AM - Tuesday 3:59:59AM Saturday ~4:00:00 AM – Sunday ~3:59:59 AM

Wednesday Tuesday 4:00AM - Wednesday 3:59:59AM Sunday ~4:00:00 AM – Monday ~3:59:59 AM

Thursday Wednesday 4:00AM - Thursday 3:59:59AM Monday ~4:00:00 AM – Tuesday ~3:59:59 AM

Friday Thursday 4:00AM - Friday 3:59:59AM Tuesday ~4:00:00 AM – Wednesday ~3:59:59 AM

Saturday Friday 4:00AM - Saturday 3:59:59AM Wednesday ~4:00:00 AM – Thursday ~3:59:59 AM

Sunday Saturday 4:00AM - Sunday 3:59:59AM Thursday ~4:00:00 AM – Friday ~3:59:59 AM

Load, Generation, and Settlements & Billing Extracts ERCOT Recommended Process from ODS

Data Type: – Load and Generation Extracts contain all outputs for a trade date in

one extract– Settlement & Billing Daily Extract contains Settlements input data

used and output data created for each approved trade date for all trade dates within a batch run becoming a complete trade date specific extract

– Provide complete batch run with all Trade Dates approved accounting for subsequent Trade Date reruns prior to Settlements Approval

Data Process Flow: Continuous data flow from the Lodestar Source System -> Replicated Source System -> Lodestar ODS capturing intraday data updates and point in time selectivity

Load, Generation, and Settlements & Billing Extracts ERCOT Recommended Process from ODS – con’t


Extract Creation Trigger: By market participants scheduling the extract

Recommended Changes:– With extract creation and posting delay, No additional logic

necessary to accommodate for receipt of unapproved Trade Date data and extract schedule changes

– Up to an additional 3 day lag to the delivery timeline to accommodate for synchronization of extracts with approved settlement data (weekends & holidays), Lodestar ODS data synchronization including scheduled & unscheduled environment maintenance and troubleshooting data & system issues

Recommended Changes:– Consistent synchronization of SCR 727 ESIID Extract data and RID

Extract data in conjunction with corresponding Load & Generation Extract postings – Granular data necessary to shadow settle is posted 2 days prior to the Aggregation and Settlement extracts

Timing proposal discussed with DEWG at May 2005 meeting

Load, Generation, and Settlements & Billing Extracts ERCOT Recommended Process from ODS – con’t

Load, Generation, and Settlements & Billing Extracts Timing Change

Day Extract is Produced and

Published


Batch Trade Dates for due Settlement Statements

RecommendationData Time Window

Monday No extract produced and published Thursday, Friday, Saturday Wednesday Batch Trade Dates

Tuesday Friday 12:30 AM - Monday 12:29:59AM(Fri, Sat, Sun batch)

Sunday Thursday Batch Trade Dates

Wednesday Monday 12:30AM - Tuesday 12:29:59AM(Mon batch)

Monday Friday Batch Trade Dates

Thursday Tuesday 12:30AM - Wednesday 12:29:59AM(Tues batch)

Tuesday Saturday Batch Trade Dates

Friday Wednesday 12:30AM - Thursday 12:29:59AM(Wed batch)

Wednesday Sunday Batch Trade Dates

Saturday Thursday 12:30AM - Friday 12:29:59AM(Thurs batch)

No approved statements posted

Monday Batch Trade Dates

Sunday No extract produced and published No approved statements posted

Tuesday Batch Trade Dates

Settlement Input Data Extracts (New)ERCOT Recommended Process from ODS

Data Type: Contains all input data passed from MOS to the settlement system or entered by settlement personnel; data that is currently provided in the Settlement & Billing Extracts



Extract Creation Trigger: By market participants scheduling the extract

Settlement Input Data Extracts (New)ERCOT Recommended Process from ODS – con’t

Recommended Changes:– Redefines Settlement & Billing Extract to only provide complete

approved trade date inputs and outputs synchronized with the Load and Generation Extracts

– Data load time window is captured by the initiation of granular data changes (Lodestar Batch usage data loading) at SYSDATE -3 to the next day’s initiation of granular data changes (Lodestar Batch usage data loading)

Settlement Input Data Extracts Timing Change


Published

CurrentSettlements & Billing Extract Data Addtime Time Window

RecommendationNew Extract

Data Addtime Time Window

Monday No extract produced and published Friday ~4:00:00 AM – Saturday ~3:59:59 AM


Saturday ~4:00:00 AM – Sunday ~3:59:59 AM


Sunday ~4:00:00 AM – Monday ~3:59:59 AM


Monday ~4:00:00 AM – Tuesday ~3:59:59 AM


Tuesday ~4:00:00 AM – Wednesday ~3:59:59 AM


Wednesday ~4:00:00 AM – Thursday ~3:59:59 AM

Sunday No extract produced and published Thursday ~4:00:00 AM – Friday ~3:59:59 AM

Shadow Pricing ExtractsERCOT Recommended Process from ODS

Data Type: Contains shadow pricing data



Extract Creation Trigger: Automatically by data inserts

Recommended Changes:– Data load time window is captured by the initiation of granular data

changes (Lodestar Batch usage data loading) at SYSDATE -3 to the next day’s initiation of granular data changes (Lodestar Batch usage data loading)

Settlement Input Data Extracts Timing


Published


RecommendationData Addtime Time Window

Monday No extract produced and published Friday ~4:00:00 AM – Saturday ~3:59:59 AM


Saturday ~4:00:00 AM – Sunday ~3:59:59 AM


Sunday ~4:00:00 AM – Monday ~3:59:59 AM


Monday ~4:00:00 AM – Tuesday ~3:59:59 AM


Tuesday ~4:00:00 AM – Wednesday ~3:59:59 AM


Wednesday ~4:00:00 AM – Thursday ~3:59:59 AM

Sunday No extract produced and published Thursday ~4:00:00 AM – Friday ~3:59:59 AM

Public Reference Table Extracts (New)ERCOT Recommended Process from ODS

Data Type: Contains full tables of non-interval market level data (ex. TDSP, STATION, CMZONE, etc.) from Lodestar extracts


Frequency: Generation and posting of extract occurs daily and is posted to market participants after being audited; only one daily extract available (no historical postings); delivery timeline same as SCR 727 ESIID and Resource ID Extracts


What’s Next?

Comments, Feedback, Questions

ERCOT complete Analytical Designs of Execution Phase for the following projects:

– PR 30026 – Resource ID Extract – PR 40007 – SCR 727 Phase II (SCR 727 ESIID Extract and Public

Reference Table Extract)– PR 40042_06 – Lodestar Batch Extracts (Load, Generation,

Settlements & Billing, Settlement Input Data, and Shadow Pricing Extracts)

Documents

ERCOT Lodestar Extracts Timing Recommendation ERCOT May 24, 2005