Robust unit commitment using the parametric cost function ... presentation FERC June 2017.pdfSlide 1 Robust unit commitment using the parametric cost function approximation FERC, Washington,

Slide 1

Robust unit commitment using the parametric cost function approximation

FERC, Washington, D.C.

June 25-26, 2017

Raymond PerkinsWarren B. Powell

CASTLE LabsPrinceton University

http://www.castlelab.princeton.edu

Slide 1© Warren B. Powell, 2014

Mission statement Main points:

» Classical view is that ISOs use a “deterministic model” for unit commitment.

» Considerable research has been done on the “stochastic unit commitment problem” that uses a stochastic lookahead model using scenario trees.

» Our thesis: • “stochastic programming” is a form of policy (a stochastic lookahead

model) for solving a stochastic problem (the real world).• ISOs use a modified deterministic lookahead model, where the

modifications enforce reserve requirements to ensure a robust solution.• We call this a parametric cost function approximation, and argue that this

is also a form of policy for solving stochastic unit commitment problems.» We will describe weaknesses in the use of scenario trees, and argue

why the parametric CFA (which is current industry practice) is likely to be much more effective for handling uncertainty in this context.

Slide 2

Variability and uncertainty

Slide 3

Win

d

So

lar


Illustration of forecasted wind power and actual» The forecast (black line) is deterministic (at time t, when the forecast

was made). The actuals are stochastic.

This is our forecast of the wind power at time t’, made at time t.

'ttf

This is the actual energy from wind, showingthe deviations from forecast.

Slide 4Current timet ' Some point in the futuret

Variability and uncertainty Forecasts evolve over time as new information arrives:

Slide 5

Actual

Rolling forecasts, updated each hour.

Forecast made at midnight:


ISOs handle uncertainty using a sequence of decisions» Day-ahead, intermediate term and real-time planning

each address different types of decisions

Slide 6

© 2010 Warren B. Powell Slide 7

Lecture outline

General modeling framework Stochastic lookahead policies The parametric cost function approximation Conclusions

Slide 7


Lecture outline


Slide 8

General modeling framework The objective function

Given a system model (transition function)

We refer to this as the base model to avoid confusions with lookahead models we will introduce later.

00

min , ( ) |

T

t t tt

C S X S S

Decision function (policy)State variableCost function

Finding the best policy

Expectation over allrandom outcomes

1 1, , ( )Mt t t tS S S x W

Slide 9

General modeling framework

There are two fundamental strategies for solving sequential decision problems:» Policy search – Search over a parameterized class of

functions for making decisions to optimize some metric.

» Lookahead approximations – Approximate the impact of a decision now on the future.

00

min , ( | ) |

T

t t tt

E C S X S S

Policy search Policy search – Two types of policies:

» Analytical functions that directly map states to actions (“policy function approximations”)

• Lookup tables – “when in this state, take this action”

• Parametric functions– Order-up-to policies: if inventory is less than s, order up to S.

• Locally/semi/non parametric– Release rate from a reservoir as a function of reservoir level

» Minimizing analytical approximations of costs and/or constraints (“cost function approximations”)

• Optimizing a deterministic model modified to handle uncertainty (buffer stocks, schedule slack)

( )( | ) arg min ( , | )

t t

CFAt t tx

X S C S x

X

Lookahead policies Lookahead approximations – Approximate the impact of a

decision now on the future:» Approximate lookahead models – Optimize over an approximate

model of the future:• Replace uncertain future with a deterministic approximation• Model future with a small sample of uncertain outcomes

» Approximating the value of being in a downstream state using machine learning (“value function approximations”)

( ) arg min ( , ) ( , )t

VFA x xt t x t t t t t tX S C S x V S S x

*' ' ' 1

' 1

( ) arg min ( , ) min ( , ( )) | | ,

t

T

t t x t t t t t t t tt t

X S C S x C S X S S S x

*' ' ' 1

' 1( ) arg max ( , ) max ( , ( )) | | ,

t

T



Lookahead policies

The ultimate lookahead policy is optimal

Slide 13

Expectations that we cannot compute

Maximization that we cannot compute

Designing policies

The ultimate lookahead policy is optimal

Instead, we have to solve an approximation called the lookahead model:

» A lookahead policy works by approximating the lookahead model.

Slide 14

*' ' ' 1

' 1( ) arg max ( , ) max ( , ( )) | | ,

t

T



*' ' ' , 1

' 1( ) arg max ( , ) max ( , ( )) | | ,

t

t H

t t x t t tt t tt t t tt tt t


Lookahead policies

We use a series of approximations:» Horizon truncation – Replacing a longer horizon

problem with a shorter horizon» Stage aggregation – Replacing multistage problems

with two-stage approximation.» Outcome aggregation/sampling – Simplifying the

exogenous information process» Discretization – Of time, states and decisions» Dimensionality reduction – We may ignore some

variables (such as forecasts) in the lookahead model that we capture in the base model (these become latentvariables in the lookahead model).

1) Policy function approximations (PFAs)» Lookup tables, rules, parametric/nonparametric functions

2) Cost function approximation (CFAs)»

3) Policies based on value function approximations (VFAs)»

4) Direct lookahead policies (DLAs)» Deterministic lookahead/rolling horizon proc./model predictive control

» Chance constrained programming

» Stochastic lookahead /stochastic prog/Monte Carlo tree search

» “Robust optimization”

Four (meta)classes of policies

( )( | ) arg min ( , | )

t t

CFAt t tx

X S C S x

X

,' ',..., ' 1

( ) arg min ( , ) ( , )tt t t H

TLA Dt t tt tt tt ttx x t t

X S C S x C S x

( ) arg min ( , ) ( , )t


' '' 1

( ) arg min ( , ) ( ) ( ( ), ( ))t

TLA St t tt tt tt tt

t tX S C S x p C S x

xtt , xt ,t1,..., xt ,tT

,' ',..., ( ) ' 1

( ) arg min max ( , ) ( ( ), ( ))tt t t H t

TLA ROt t tt tt tt ttx x w W t t

X S C S x C S w x w

[ ( )] 1t tP A x f W

Polic

y se

arch

Look

ahea

dap

prox

imat

ions

Lookahead policies

Lookahead policies peek into the future» Optimize over deterministic lookahead model

The base model

. . . .

The

look

ahea

dm

odel

t 1t 2t 3t

Slide 17

Lookahead policies


. . . .

The

look

ahea

dm

odel

t 1t 2t 3t The base model

Slide 18

Lookahead policies


. . . .

The

look

ahea

dm

odel


Slide 19

Lookahead policies


. . . .

The

look

ahea

dm

odel


Slide 20


Lecture outline


Slide 21

Stochastic lookahead policies

Stochastic lookahead» Here, we approximate the information model by using a

Monte Carlo sample to create a scenario tree: 1am 2am 3am 4am 5am …..

Change in wind speed

Change in wind speed

Change in wind speedSlide 22


We can then simulate this lookahead policy over time:

. . . .

t 1t 2t 3t

The

look

ahea

dm

odel

The base modelSlide 23

. . . .

t 1t 2t 3t



The

look

ahea

dm

odel


. . . .

t 1t 2t 3t



The

look

ahea

dm

odel


. . . .

t 1t 2t 3t



The

look

ahea

dm

odel



Two stage lookahead approximation

Slide 27

0x

1) Schedule steam

1

2

N

1W

2) See wind:

1 2, ,..., Tx x x3) Schedule turbines


Creating wind scenarios (Scenario #1)

Slide 28



Slide 29



Slide 30



Slide 31



Slide 32


The two-stage approximation

0

1) Schedulesteam

x

2) See wind:

3) Schedule turbinesSlide 33


Slide 34

Load shedding event:


Slide 35



Slide 36


Actual wind

Hour ahead

forecast


Slide 37

Note the dip in steam just when the outage occurs – This is because we forecasted that wind would cover the load.


Slide 38

When we allow the model to see into the future (the drop in wind), it schedules additional steam the day before. But only when we need it!


The two-stage approximation

0

1) Schedulesteam

x

2) See wind:

3) Schedule turbinesSlide 39

Downward wind shift


Lecture outline


Slide 40

1) Policy function approximations (PFAs)» Lookup tables, rules, parametric/nonparametric functions

2) Cost function approximation (CFAs)»

3) Policies based on value function approximations (VFAs)»

4) Lookahead policies» Deterministic lookahead/rolling horizon proc./model predictive control

» Chance constrained programming

» Stochastic lookahead /stochastic prog/Monte Carlo tree search

» “Robust optimization”

Four (meta)classes of policies

( )( | ) arg min ( , | )

t t

CFAt t tx

X S C S x

X

,' ',..., ' 1

( ) arg min ( , ) ( , )tt t t H

TLA Dt t tt tt tt ttx x t t

X S C S x C S x

( ) arg min ( , ) ( , )t


' '' 1

( ) arg min ( , ) ( ) ( ( ), ( ))t

TLA St t tt tt tt tt

t tX S C S x p C S x

xtt , xt ,t1,..., xt ,tT

,' ',..., ( ) ' 1

( ) arg min max ( , ) ( ( ), ( ))tt t t H t

TLA ROt t tt tt tt ttx x w W t t

X S C S x C S w x w

[ ( )] 1t tP A x f W

A deterministic lookahead model» Optimize over all decisions at the same time

» In a deterministic model, we mix generators with different notification times:

• Steam generation is made day-ahead• Gas turbines can be planned an hour ahead or less

Parametric cost function approximation

Gas turbines

' ' 1,...,24

' ' 1,...,24

' ''

( )

min ( , )

tt t

tt t

t H

tt ttx t ty

C x y

Slide 42

Steam generation

A deterministic lookahead policy» This is the policy produced by solving a deterministic

lookahead model

» No ISO uses a deterministic lookahead model. It would never work, and for this reason they have never used it. They always modify the model to produce a robust solution.


Steam generation

Slide 43

' ' 1,...,24

' ' 1,...,24

' ''

( )

( )= min ( , )

tt t

tt t

t H

t t tt ttx t ty

X S C x y

Gas turbines

A robust CFA policy» The ISOs introduce reserves:

» This modification is a form of parametric function (a parametric cost function approximation). It has to be tuned to produce a robust policy.


Slide 44

max, ' , ' '

max, ' , ' '

Up-ramping reserve

Down-ramping reserve

upt t t t tt

downt t t t tt

x x L

x x L

' ' 1,...,24

' ' 1,...,24

' ''

( )

( | )= min ( , )tt t

tt t

t H

t t tt ttx t ty

X S C x y


An energy storage problem:

Slide 45


Benchmark policy – Deterministic lookahead

Slide 46


Parametric cost function approximations» Replace the constraint

with:» Lookup table modified forecasts (one adjustment term for

each time in the future):

» Exponential function for adjustments (just two parameters)

» Constant adjustment (one parameter)

Slide 47

't t

' ' ' 'wr wd Ett tt t t ttx x F

'wrttx '

wdttx

2 ( ' )

' ' 1 '

t twr wd Ett tt ttx x e F

' ' 'wr wd Ett tt ttx x F


Optimizing the CFA:» Let be a simulation of our policy given by

» We then compute the gradient with respect to

» The parameter is found using a classical stochastic gradient algorithm:

We tested several stepsize formulas and found that ADAGRAD worked best:

Slide 48

( ) ( , )F F

( , )F

0

( , ) ( ), ( ( ) | )T

t t tt

F C S X S

1 1( , )n n n nnF


Optimizing the CFA:» We compute the gradient by applying the chain rule

» Where the interaction from one time period to the next is captured using

» Assuming there are no integer variables, these equations are quite easy to compute.

» For real stochastic unit commitment problems, we are going to need to use a derivative-free algorithm.

Slide 49


Optimal adjustment parameters for each model

Slide 50

Lookup tableConstant parameter

Constant parameter

Lookup table

Exponential function

Exponential function

More accurate forecasts Less accurate forecasts

1.0

0.0

1.0

0.0


Improvement over deterministic benchmark:

» Significant improvement over deterministic benchmark (an untuned lookahead policy).

Slide 51

Decreasing forecast accuracy:


0

20000

40000

60000

80000

100000

120000

140000

1 36

71

106

141

176

211

246

281

316

351

386

421

456

491

526

561

596

631

666

701

736

771

806

841

876

911

946

981

1016

10

51

1086

11

21

1156

11

91

1226

12

61

1296

13

31

1366

14

01

1436

14

71

1506

15

41

1576

16

11

1646

16

81

1716

17

51

1786

18

21

1856

18

91

1926

19

61

1996

MW

5‐min Time Intervals

SMART‐ISO ‐ Unconstrained Grid ‐ 22‐28 Jul 2010 Wind Buildout 4 ‐ No ramping reserves

Actual Demand (Exc) Simulated (Used) Wind Simulated Storage Power Simulated Fast Power Simulated Slow Power

Slide 52


0

20000

40000

60000

80000

100000

120000

140000

1 36

71

106

141

176

211

246

281

316

351

386

421

456

491

526

561

596

631

666

701

736

771

806

841

876

911

946

981

1016

10

51

1086

11

21

1156

11

91

1226

12

61

1296

13

31

1366

14

01

1436

14

71

1506

15

41

1576

16

11

1646

16

81

1716

17

51

1786

18

21

1856

18

91

1926

19

61

1996

MW


SMART‐ISO ‐ Unconstrained Grid ‐ 22‐28 Jul 2010 Wind Buildout 4 ‐ No ramping reserves


0

20000

40000

60000

80000

100000

120000

140000

1 36

71

106

141

176

211

246

281

316

351

386

421

456

491

526

561

596

631

666

701

736

771

806

841

876

911

946

981

1016

10

51

1086

11

21

1156

11

91

1226

12

61

1296

13

31

1366

14

01

1436

14

71

1506

15

41

1576

16

11

1646

16

81

1716

17

51

1786

18

21

1856

18

91

1926

19

61

1996

MW


SMART‐ISO ‐ Unconstrained Grid ‐ 22‐28 Jul 2010 Wind Buildout 4 ‐ Ramping reserves 9GW


Notice that we get uniformly more gas turbine reserves at all points in time (because we asked for it)

Slide 53

Slide 54

Paper available on arXiv:


Lecture outline


Slide 55

Conclusions:» Weaknesses of stochastic programming (with scenario trees)

• Computationally intensive• It does not ensure robust solutions (needs too many scenarios)• Many approximations are required (e.g. two stage).

» Parametric cost function approximations:• Generalizes standard industry practice• Uses domain knowledge to ensure robust policies across a much wider

range of scenarios.• Resulting models can be implemented using existing commercial

software.

» Parametric CFAs open up an entirely new research directions for stochastic unit commitment:

• Propose new parameterizations to achieve robustness at lowest cost.• Design algorithms (either derivative-based or derivative-free) to optimize

the parameterized policy.• Need to design algorithms for online learning.

Slide 56

Thank you!

http://energysystems.Princeton.edu

Documents

Robust unit commitment using the parametric cost function ... presentation FERC June 2017.pdfSlide 1 Robust unit commitment using the parametric cost function approximation FERC, Washington,