Coordinated Ramping Product and Regulation Reserve ... Solar-Forecast… · • P-P Plot score and MAPE comparison (normalized by max GHI, daylight hours only) • Watt-Sun 1.0 outperforms

Coordinated Ramping Product and Regulation Reserve Procurements in CAISO and MISO using

Multi-Scale Probabilistic Solar Power Forecasts (Pro2R)

The Johns Hopkins UniversityAward # DE-EE0008215

Solar Forecasting 2 Annual Review & WorkshopOctober 8, 2019

:

Funded by:

Principal Investigator: B.F. Hobbs, The Johns Hopkins University

This presentation may have proprietary information and is protected from public release.

Pro2R Project Team

Johns Hopkins: Benjamin Hobbs (PI), Qingyu Xu, Josephine Wang

NREL: Venkat Krishnan (Co-PI), Elina Spyrou, Paul Edwards, Haiku SkyIBM: Hendrik Hamann (Co-PI), Rui ZhangUniversity of Texas Dallas: Jie Zhang (Co-PI), Binghui Li

Industry Partners: • Amber Motley, Clyde Loutan, Rebecca Webb, Guillermo Bautista

(California ISO)• Blagoy Borissov, Stephen Rose (Midcontinent ISO)

Introduction

2


Pro2R Project Summary

Objective: Integrate probabilistic short- (2-3 hr ahead) and mid-term (day-ahead) solar power forecasts into operations of CAISO & MISO

Thrust 2: Integrate probabilistic forecasts in ISO operations for ramp product &regulation requirements

Thrust 3: Provide situational awareness via visualizations of probabilistic ramp forecasts & alerts

Approach:

Thrust 1: Advanced big data-driven“probabilistic” solar power forecasting technology using IBM Watt-Sun & PAIRS (Big data information processing and machine learning approaches to blend outputs from multiple models).

Introduction

3


Thrust 1: SFII Built Upon IBM PAIRS Geoscope Platform

• Distributed computational system• Scales to many hundreds of Petabytes (PB)• Data processing rate at PB/day.

Thrust 1

NAM HRRR-Subhour GOES-R

spatial resolution 12 km 3 km 500 m

temporal resolution 1 hr 15 min 5~10 min

daily data volume 9.4 GB 86 GB 203 GB

4

A: NegativeError

Forecast ErrorGHI

(W/m^2)

C:PositiveError

B:PositiveError

D

16.6

90

Zenith(Deg)

GHI Forecast (W/m^2)0 1.02e3Question: What is the error of the models, when, where, under what weather situation?

NAM GHI Forecast Error (Surfrad BND)– Strongly depends on zenith angle and forecast

irradiance. The two parameters create 4 categories of situations:

Hurricane Ike path forecasts from 8 different weather models*

Situation-Dependent Model Blending for Solar Forecasting

Thrust 1

5


Utilization of GOES-R Satellites –In progress– Expected in Watt-Sun 2.0, 2019-20 Q7• Blue band, 0.47 μm: monitor dust, haze, smoke and clouds• Red band, 0.6 μm: detect fog, estimate solar insolation, depict diurnal aspects of clouds• Near-infrared, 0.86 μm: detect daytime clouds, fog, and aerosols; calculate normalized difference vegetation index• Infrared, 10.3 μm: correct atmospheric moisture, estimate cloud particle size, characterize surface properties

Plan:• Translate GOES imagery

to Solar Irradiance raster map

• Use Deep Learning combined with Optic Flow to forecast cloud movement, then translate into forecast solar irradiance map.

Thrust 1

6

7

7Overall Flowchart

Numerical Weather Prediction Models

Model Training



FANOVA Parameter (feature) selection

Random Forest model to prediction the forecast

error (Train)

Gaussian Mixture Model for clustering into number of weather categories (Train)

Quantile regression

Train

Solar irradiance measurements

Model Forecasting




Use trained Random Forest model to predict

forecast error

Use trained Gaussian Mixture Model & predicted forecast

error to predict weather categories

Use trained Quantile

regression for each category

for probabilistic forecast







Quantile regression

Train

Quantile regression

Train

Thrust 1

Used quantile regression to deploy probabilistic forecast models

• Quantiles of solar as function of independent variables• Example results for 2 hr-ahead forecasts

• Distributions are asymmetric need quantile regression techniques• Adjacent days have different distributions; but present CAISO flexiramp requirements are very stable day-to-day

because they don’t reflect weather forecasts need to integrate probabilistic forecasts in requirements

Thrust 1

8

P-P plot-based Error Metric• A P–P plot (probability–probability plot) assesses how closely two data sets agree• By plotting the two cumulative distribution functions against each other

P-P plot (2 hr forecast)Mean absolute difference: 0.0543

Example Empirical CDF Curves (2 hr forecast)

CDF

RankGHI

CDF

Thrust 1

9

P-P plot 2 Hr Ahead Error Metric: Comparison with Bias Corrected HRRR

Empirical CDFMean absolute difference: 0.054

Bias Corrected HRRR (normal distribution)

Mean absolute difference: 0.086

CDF

Rank RankThrust 1

10

• P-P Plot score and MAPE comparison (normalized by max GHI; daylight hours only)• Watt-Sun 1.0 outperforms HRRR Bias Corrected in all sites in terms of MAPE, in most

sites in terms of P-P Plot score

Thrust 1: Forecast Error Report -- CAISO

Thrust 1

11

• P-P Plot score and MAPE comparison (normalized by max GHI, daylight hours only)• Watt-Sun 1.0 outperforms HRRR Bias Corrected in all sites in terms of MAPE, in

most sites in terms of P-P Plot score

Thrust 1: Forecast Error Report -- MISO

Thrust 1

12

Baseline approach CAISO bases FRP requirements on confidence intervals for net load uncertainty by analyzing a rolling window of last 20-40 days for the same hour

0

5,000

10,000

15,000

20,000

25,000

30,000

01.

25 2.5

3.75 5

6.25 7.

58.

75 1011

.25

12.5

13.7

5 1516

.25

17.5

18.7

5 2021

.25

22.5

23.7

5

MW

hour

CAISO RTPD forecast for Net load

5/14/2019 5/15/2019

0

500

1000

1500

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

MW

hours

Uncertainty component (direction down)

5/14/2019 5/15/2019

Thrust 2

Thrust 2: Integration of probabilistic forecasts into ISO operations: Flexible Ramping Product

13


Baseline method for FRU uncertainty component potentially leads to under-procurement & price spikes

0

500

1000

1500

2000

2500

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

MW

Max RTD net load – RTPD net load

5/14/2019 5/15/2019 fru

Two days had different net load errors between RTPD & RTD different RTD prices

Source: CAISO OASIS

Estimates of FRP uncertainty component do not necessarily correspond to the target confidence interval

0%

20%

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24hours

Percent of 15-min intervals in March-May 2019 that underestimated flexible ramping need (CAISO only)

RTPD FRU < max RTD net load -RTPD net loadRTPD FRD < RTPD net load -min RTD net loadReference: 2.5%

Thrust 2

14

-100

400

900

$/M

Wh

RTD prices 5/15/2019 5/14/2019


I. AggregationII. Net load (NL)III. NL forecast errorIV. FRP Requirements

NL forecast error (RTD)

Net load(Binding RTD)

Net load(Advisory RTD)

Convolution

Convolution

Site-level probabilistic solar power

(Advisory RTD)

System-level probabilistic solar power

(Advisory RTD)

Site-level probabilistic solar irradiance

(Advisory RTD)PVlib Convolution

System-level probabilistic load, wind power (Advisory

RTD)

Site-level probabilistic solar power

(Binding RTD)

Site-level probabilistic solar irradiance

(Binding RTD)PVlib Convolution

System-level probabilistic load, wind power (Binding

RTD)

System-level probabilistic load, wind power (Binding

RTD)

Convolution

FRP in RTD

Thrust 2

Probabilistic net load forecast-based ramping product 15


Value of probabilistic forecast-based FRP procurement

Compare baseline to probabilistic requirements

Probabilistic 𝐹𝐹𝐹𝐹𝐹𝐹≈ 𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵 𝐹𝐹𝐹𝐹𝐹𝐹

Probabilistic 𝐹𝐹𝐹𝐹𝐹𝐹> 𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵 𝐹𝐹𝐹𝐹𝐹𝐹

Actual RTD flex need≤ 𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵 𝐹𝐹𝐹𝐹𝐹𝐹

Actual RTD flexibility need> 𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵 𝐹𝐹𝐹𝐹𝐹𝐹

Probabilistic 𝐹𝐹𝐹𝐹𝐹𝐹< 𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵 𝐹𝐹𝐹𝐹𝐹𝐹

Actual RTD flex need> 𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝐵𝐵𝑃𝑃𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝑃𝑃𝐵𝐵𝑃𝑃 𝐹𝐹𝐹𝐹𝐹𝐹

Actual RTD flex need≤ 𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝐵𝐵𝑃𝑃𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝑃𝑃𝐵𝐵𝑃𝑃 𝐹𝐹𝐹𝐹𝐹𝐹

Production cost ≈ Increase DecreasePower balance violations (scarcity events)

≈ Decrease Increase

a b c d e

F: Frequency

Fa

Fb Fc Fd Fe

Compare requirements

Actual need (realization)

Performance of prob. forecasts

Thrust 2

Under procurement (Risky)

Over procurement(Conservative)

16


Daily profile BaselineExtreme ramps

realized?(Prob Req -Baseline

Req) / (Baseline Req)19th May: (b) intervals Risky Occurred (11%) +40%

6th May: (c) intervals Risky Did not occur

(42%) +79%

12th May: (e) intervals Conservative Did not occur

(44%) -13%

Market simulations: Probabilistic vs. Baseline

Test system: Modified 118 bus IEEE Reliability Test System, mimicking CAISO gen mix (1/10th CAISO system)

Thrust 2.2

Reliability (# Gen. scarcity)

Production cost

Uncertainty-induced costs

Decrease(2713)

Greater(+$2,601) Lower (-60%)

Same(5 5)

Greater(+$920) Higher (+5%)

Same(0 0)

Lower(-$497) Lower (-19%)

• When baseline is riskier we expect reliability benefits when high ramps realized• When baseline is conservative we expect production cost reductions when ramps not realized

0

2

4

6

0

2000

4000

6000

8000

6.50 6.58 6.67 6.75 6.83 6.92 7.00 7.08 7.17

MW

h

$/M

Wh

RTD market results (Baseline)

RTD price RTD unserved energy

Power balance violations during 7 RTD market intervals:Baseline Probabilistic

00.20.40.60.81

-

20

40

60

6.50 6.58 6.67 6.75 6.83 6.92 7.00 7.08 7.17

MW

h

$/M

Wh

RTD market results (Probabilistic)

RTD price RTD unserved energy

No power balance violations in RTD market:

NREL | 18

The Ramp Visualization for Situational Awareness (RaViS) tool provides situational awareness and visualization capabilities using probabilistic solar and net-load time series.

Features:• Integrates IBM forecast data• Refresh rate of 60 seconds• User interface: Single page web application and

open source• Shows site specific metadata via hover• Highly flexible, easily configurable

Future work:• Net-load ramps• Adaptable to other kinds of events: outage/trip,

cyber threatsEvent Outlook

Geolocation All-regions

Geolocation Region Detail

Forecast Detail

Thrust 3: RaViS - Visualization tool

Thrust 3

18


Pro2R: What’s Ahead

• Thrust 1: Probabilistic Watt-Sun Versions 2.0 & 3.0: integrates GEOS-R, multi-expert machine learning, & blended models

• Thrust 2:• Statistical/ML modeling of relationship of probabilistic solar forecasts

to net load uncertainty on regulation & FRP timescales• Simulation-based testing on ISO-scale systems of improved

requirements: cost & reliability• Interaction with CAISO & MISO on data, method value, simulations, &

implementation pathways• Thrust 3: RaVIS development, including integrating latest Watt-Sun

methods, net load data, market information, and ISO feedbackConclusion

19


Wrap-Up: Pro2R Impact

• Identified the major methodological issues in integrating probabilistic forecasts into system products• BP2 will address those issues and possible improvements in forecasting and

industry practice (e.g., product definition, timeline) • Expected outcome: a blueprint for research in forecasting and industry practice

• Project results expected to highly influence industry practice • ISO staff confirmed that they are evaluating potential improvements to their

existing requirements approach• Integrating probabilistic forecasts is the most promising way to address needs

for requirements to reflect up-to-date weather forecasts

20

Conclusion


Questions? 21


EXTRA SLIDES


Percentage of time when reduction is over 10% FRU FRD

May 27, 2019 38% 25%May 28, 2019 42% 50%May 29, 2019 46% 50%May 30, 2019 58% 38%May 31, 2019 42% 46%

• RTPD• Requirements are increased occasionally due to greater

uncertainties of NL forecast errors• 25%-58% of hours see >10% reductions in FRU and FRD

requirements

Thrust 2.1: Requirements for ramping product

00:00 06:00 12:00 18:00 00:00

May 27, 2019

-2000

-1000

0

1000

2000

MW

Prob

Baseline

00:00 06:00 12:00 18:00 00:00

May 27, 2019

10-2

10-1

100

101

102

Rat

io o

f Pro

b/Ba

selin

e

FRD

FRU

Thrust 2.1

Documents

Coordinated Ramping Product and Regulation Reserve ... Solar-Forecast… · • P-P Plot score and MAPE comparison (normalized by max GHI, daylight hours only) • Watt-Sun 1.0 outperforms