Smart Power Systems: the Optimization Challenge · 2013-09-17 · Smart Power Systems: the Optimization Challenge Michel DE LARA Cermics, ´Ecole des Ponts ParisTech, France and Pierre

Smart Power Systems:the Optimization Challenge

Michel DE LARA

Cermics, Ecole des Ponts ParisTech, Franceand

Pierre Carpentier, ENSTA ParisTech, FranceJean-Philippe Chancelier, Ecole des Ponts ParisTech, France

Pierre Girardeau, post-doc EDF R & DJean-Christophe Alais, Ecole des Ponts and EDF R & D PhD student

Vincent Leclere, Ecole des Ponts PhD student

Cermics, France

December 19, 2012

Michel DE LARA (Cermics, France) Horizon Maths 2012, Paris, 19 Dec. 2012 December 19, 2012 1 / 112

Outline of the talk

In 2000, the Optimization and Systems team was created at Ecole des PontsParisTech, with stochastic optimization as major theme. The idea was toextend decomposition-coordination optimization methods for large systems

— developed in the deterministic setting with Electricite de France (EDFR&D) — to the stochastic case

During ten years, we have trained PhD students in stochastic optimization,mostly with EDF R&D. Since 2011, we witness a growing demand fromenergy firms for stochastic optimization, fueled by a deep and fast

transformation of power systems

We discuss to what extent optimization is challenged by the smart gridperspective

We shed light on the two main differences of stochastic control with respectto deterministic control: risk and information

We cast a glow on two snapshots highlighting ongoing research in the field


Outline of the presentation

1 Long term industry-academy cooperation

2 The “smart grids” seen from an optimizer perspective

3 Moving from deterministic to stochastic dynamic optimization

4 Two snapshots on ongoing research

5 Conclusion: a need for training


Long term industry-academy cooperation








Long term industry-academy cooperation Ecole des Ponts ParisTech–Cermics–Optimization and Systems


1 Long term industry-academy cooperationEcole des Ponts ParisTech–Cermics–Optimization and SystemsIndustry partners

2 The “smart grids” seen from an optimizer perspectiveWhat is the “smart grid”?Optimization is challenged

3 Moving from deterministic to stochastic dynamic optimizationDam modelsThe deterministic optimization problem is well posedIn the uncertain framework, the optimization problem is not well posedThere are two ways to express the information constraints

4 Two snapshots on ongoing researchDecomposition-coordination optimization methods under uncertaintyRisk constraints in optimization




Russian dolls



Ecole des Ponts ParisTech

The Ecole nationale des ponts et chaussees was founded in 1747 and is oneof the world’s oldest engineering institutes

Ecole des Ponts ParisTech is traditionally considered as belonging to the5 leading engineering schools in France

Young graduates find positions in professional sectors like transport andurban planning, banking, finance, consulting, civil works, industry,environnement, energy, etc.

Faculty and staff

217 employees (including 50 subsidaries).165 module leaders, including 68 professors.1509 students.

Ecole des Ponts ParisTech is part of University Paris-Est



Research at Ecole des Ponts ParisTech

Figures on research

Research personnel: 220About 40 Ecole des Ponts PhDs students graduate each year

10 research centers* CEREA (atmospheric environment), joint Ecole des Ponts-EDF R&D* CEREVE (water, urban and environment studies)* CERMICS (mathematics and scientific computing)* CERTIS (information technologies and systems)* CIRED (international environment and development)* LATTS (techniques, regional planning and society)* LVMT (city, mobility, transport)* UR Navier (mechanics, materials and structures of civil engineering, geotechnic)

* Saint-Venant laboratory (fluid mechanics), joint Ecole des Ponts-EDF R&D* Paris School of Economic PSE



CERMICS, Centre d’enseignement et de recherche en

mathematiques et calcul scientifique

The scientific activity of CERMICS covers several domains in

scientific computingmodellingoptimization

14 senior researchers

14 PhD11 habilitation a diriger des recherches

Three missions

Teaching and PhD trainingScientific publicationsContracts

550 000 euros of contracts per year with

research and development centers of large industrial firms:CEA, CNES, EADS, EDF, Rio Tinto, etc.public research contracts



Optimization and Systems Group

Three senior researchers

J.-P. CHANCELIERM. DE LARAF. MEUNIER

Five PhD students

J.-C. ALAIS (CIFRE EDF)S. SEPULVEDA (foreign co-supervision)V. LECLERE (IPEF)T. PRADEAUP. SARRABEZOLLES

Four associated researchers

P. CARPENTIER (ENSTA ParisTech)L. ANDRIEU (EDF R&D)K. BARTY (EDF R&D)A. DALLAGI (EDF R&D)



Optimization and Systems Group research specialities

MethodsStochastic optimal control (discrete-time)

Large-scale systemsDiscretization and numerical methodsProbability constraints

Discrete mathematics; combinatorial optimizationSystem control theory, viability and stochastic viabilityNumerical methods for fixed points computationUncertainty and learning in economics

Applications

Optimized management of power systems under uncertainty(production scheduling, power grid operations, risk management)Transport modelling and managementNatural resources management (fisheries, mining, epidemiology)

Softwares

Scicoslab, NSPOadlibsim



Publications since 2000

24 publications in peer-reviewed international journals

3 publications in collective works

3 books

Modeling and Simulation in Scilab/Scicos with ScicosLab 4.4(2e edition, Springer-Verlag)Introduction a SCILAB(2e edition, Springer-Verlag)Sustainable Management of Natural Resources. Mathematical Models andMethods (Springer-Verlag)

2 books to be finished soon:

Stochastic Optimization. At the Crossroads between Stochastic Control and

Stochastic Programming

Control Theory for Engineers



Teaching

Masters

Mathematiques, Informatique et Applications

Economie du Developpement Durable, de l’Environnement et de l’Energie

Renewable Energy Science and Technology Master ParisTech

Ecole des Ponts ParisTech

Optimisation et controle (J.-P. CHANCELIER)Modeliser l’alea (J.-P. CHANCELIER)Introduction a Scilab (J.-P. CHANCELIER, M. DE LARA)Modelisation pour la gestion durable des ressources naturelles (M. DE LARA)Economie du risque, effet de serre et biodiversite (M. DE LARA)



Industrial and public contracts since 2000

Industrial contracts

Conseil francais de l’energie (CFE)SETEC Energy SolutionsElectricite de France (EDF R&D)ThalesInstitut francais de l’energie (IFE)Gaz de France (GDF)PSA

Public contracts

STIC-AmSud (CNRS-INRIA-Affaires etrangeres)Centre d’etude des tunnelsCNRS ACI Ecologie quantitativeRTP CNRS


Long term industry-academy cooperation Industry partners









We cooperate with industry partners

As academics, we cooperate with industry partners,looking for longlasting close relations

We are not consultants working for clients

Our job consists mainly in

training Master and PhD students, working in the company and interactingwith us, on subjects designed jointlydeveloping methods, algorithmscontribute to computer codes developed within the companytraining professional engineers in the company



Electricite de France R & D / Departement OSIRIS

Electricite de France is the French electricity main producer

159 000 collaborateurs dans le monde37 millions de clients dans le monde65,2 milliards d’euros de chiffre d’affaire630,4 TWh produits dans le monde

Electricite de France Research & Development

486 millions d’euros de budget2 000 personnes

Departement OSIRISOptimisation, simulation, risques et statistiques pour les marches de l’energieOptimization, simulation, risks and statistics for the energy markets

145 salaries (dont 10 doctorants)25 millions d’euros de budget



The Optimization and Systems Group has trained

8 PhD from 2004 to 2011 + 2 PhD students,the majority related with EDF and energy management

* Laetitia ANDRIEU, former PhD student at EDF, now researcher EDF* Kengy BARTY, former PhD student at EDF, now researcher EDF* Daniel CHEMLA, former PhD student* Anes DALLAGI, former PhD student at EDF, now researcher EDF* Laurent GILOTTE, former PhD student with IFE, researcher EDF* Pierre GIRARDEAU, former PhD student at EDF, now with ARTELYS* Eugenie LIORIS, former PhD student* Babacar SECK, former PhD student at EDF* Cyrille STRUGAREK, former PhD student at EDF, now with Munich-Re* Jean-Christophe ALAIS, PhD student at EDF* Vincent LECLERE, PhD student (partly at EDF)



PhD subjects

Contributions to the Discretization of Measurability Constraints for Stochastic

Optimization Problems,

Optimization under Probability Constraint,

Uncertainty, Inertia and Optimal Decision. Optimal Control Models Applied to Greenhouse

Gas Abatment Policies Selection,

Variational Approaches and other Contributions in Stochastic Optimization,

Particular Methods in Stochastic Optimal Control,

From Risk Constraints in Stochastic Optimization Problems to Utility Functions,

Resolution of Large Size Problems in Dynamic Stochastic Optimization and Synthesis of

Control Laws,

Evaluation and Optimization of Collective Taxis Systems,

Risk and Optimization for Energies Management,

Risk, Optimization, Large Systems,



Other contacts with French small consulting companies

ARTELYS is a company specializing in optimization, decision-making andmodeling. Relying on their high level of expertise in quantitative methods,the consultants deliver efficient solutions to complex business problems. Theyprovide services to diversified industries: Energy & Environment, Logistics &Transportation, Telecommunications, Finance and Defense.

Creee en 2011, SETEC Energy Solutions est la filiale du groupe SETECspecialisee dans les domaines de la production et de la maıtrise de l’energieen France et a l’etranger. SETEC Energy Solutions apporte a ses clients lamaıtrise des principaux process energetiques pour la mise en œuvre desolutions innovantes depuis les phases initiales de definition d’un projetjusqu’a son exploitation.



Summary

The following slides on “smart grids” express a viewpoint

from an optimizer perspective

working in an optimization research group

in an applied mathematics research center

in a French engineering institute

having contributed to train students now working in energy

having contacts and contracts with energy/environment firms


The “smart grids” seen from an optimizer perspective








The “smart grids” seen from an optimizer perspective What is the “smart grid”?









Three key drivers for the “smart grid” concept

Environment

Markets

Technology



Key driver: environmental concern

Strong impulse towards a decarbonized energy system

Expansion of renewable energy sources

⇒ Successfully integrating renewable energy sources has become critical,and made especially difficult due to their inpredictable and highly variable nature



Key driver: deregulation

A power system (generation/transmission/distribution)

less and less vertical (deregulation of energy markets)hence with many players with their own goals

with some new players

industry (electric vehicle)regional public authorities (autonomy, efficiency)

with a network in horizontal expansion(the Pan European system counts 10,000 buses, 15,000 power lines,2,500 transformers, 3,000 generators, 5,000 loads)

with more and more exchanges (trade of commodities)

⇒ A change of paradigm for managementfrom centralized to more and more decentralized



Key driver: telecommunication, metering and computing

technology

A power system with more and more technologydue to evolutions in the fields of computing and telecoms

smart meters

sensors

controllers

grid communication devices, etc.

⇒ A huge amount of data which, one day, will bea new potential for optimized management



The “smart grid”? An infrastructure project with promises

to be fulfilled by a “smart power system”

Hardware / infrastructures / smart technologies

Renewable energies technologiesSmart meteringStorage

Promises

Quality, tariffsMore safetyMore renewables (environmentally friendly)

Software / smart management(energy supply being less flexible, make the demand more flexible)smart management, smart operation, smart meter management, smart distributed generation, load management, advanced distribution

management systems, active demand management, diffuse effacement, distribution management systems, storage management, smart home,

demand side management, etc.


The “smart grids” seen from an optimizer perspective Optimization is challenged









A call from the industry to optimizers

Every three years, Electricite de France (EDF) organizes an internationalConference on Optimization and Practices in Industry (COPI)

At the last COPI’11, Jean-Francois Faugeras from EDF R&D opened theconference with a plenary talk entitled “Smart grids: a wind of change inpower systems and new opportunities for optimization”

He claimed that “power system players are facing high level problems to solverequiring new optimization methods and tools”,with “not only a ’smart(er)’ grid but a ’smart(er)’ power system”and called on the optimizers to develop new methods

EDF R&D has just created a new program Gaspard Monge pour

l’Optimisation et la recherche operationnelle (PGMO)to support academic research in the field of optimization



What is “optimization”?

Optimizing is obtaining the best compromise between needs and

resources

Marcel Boiteux (president d’honneur d’EDF)

Resources: portfolio of assetsproduction units

costly/not costly: thermal/hydropowerstock/flow, predictable/unpredictable: thermal/wind

tariffs options, contracts

Needs: energy, safety, environment

energy usessafety, quality, resilience (breakdowns, blackout)environment protection (pollution) and alternative uses (dam water)

Best compromise: minimize socio-economic costs (including externalities)



Electrical engineers metiers and skills are evolving

Unit commitment, optimal dispatch of generating units:finding the least-cost dispatch of available generation resources to meet theelectrical load

which unit? 0/1 variableswhich power level? continuous variables

subject to more unpredictable energy flows (solar, wind)and demand (electrical devices, cars)

Markets: day-ahead, intra-day (balancing market):dispatcher takes bids from the generators, demand forecasts from thedistribution companies and clears the marketsubject to more unpredictability, more players

Long term planningsubject to more unpredictability (technologies, climate), more players

. . . Without even speaking of voltage, frequency and phase control



Optimization skills will follow the power system evolution

We focus on generation and trading, not on transmission and distribution

Less base production and more wind and photovoltaic fatal generation makessupply more unpredictible→ stochastic optimization

Hence more storage (batteries, pumping stations)→ dynamical optimization, reserves dimensioning

The shape of the load is changing due to electric vehicle penetration→ demand-side management, “peak shaving”, adaptive tariffs

New subsystems emerge with local information and means of action: smartmeters, new producers, micro-grid, virtual power plant→ agregation, coordination, decentralized optimization

Markets (day-ahead, intra-day)→ optimization under uncertainty

Environmental constraints on production (CO2) and resources usages (water)→ risk constraints



A context of increasing complexity

Multiple energy resources: photovoltaic,solar heating, heatpumps, wind, hydraulicpower, combined heat and power

Spatially distributed energy resources(onshore and offshore windpower,solarfarms), producers, consumers

Strongly variable production: wind, solar

Intermittent demand: electrical vehicles

Two-ways flows in the electrical network

Environmental and risk constraints (CO2,nuclear risk, land use)



An optimization birdview

Complex power systems

stocks: water reservoirs, storage,accumulators, fuels, etc.

uncertain demand and production

multiple constraints: costs, environment,blackout risk, etc.

large interconnected networks

multiple producers and consumers

information is not shared by all the actors

Complex decision problems

dynamics

uncertainties

multiple objectives

large scale

multiple decision-makers

decentralized information



Summary

Three major key factors — environmental concern, deregulation,telecommunication, metering and computing technology —drive the power systems mutation

This induces a change of paradigm for management:from vertical centralized predictible “stock” energies tomore horizontal centralized unpredictible “flow” energies

Ripples are touching the optimization community

Specific optimization skills will be required, because optimal solution basedon a nominal forecast (deterministic setting) might perform poorly underoff-nominal conditions

Roger Wets’ illuminating example: deterministic vs. robust

of a furniture manufacturer deciding how many dressers of each type to make withman-hours and time as constraints; when they are uncertain, the stochasticoptimal solution considers all 106 possibilities and provides a robust solution,whereas the deterministic solution does not and does no point in the rightdirection


Moving from deterministic to stochastic dynamic optimization








Moving from deterministic to stochastic dynamic optimization Dam models









At

St

Qt

Rt



A single dam nonlinear dynamical model where turbinating

decisions are made before knowing the water inflows

We can model the dynamics of the water volume in a dam by

S(t + 1)︸︷︷︸

future volume

= minS♯, S(t)︸︷︷︸

volume

− q(t)︸︷︷︸

turbined

+ a(t)︸︷︷︸

inflow volume

S(t) volume (stock) of water at the beginning of period [t, t + 1[

a(t) inflow water volume (rain, etc.) during [t, t + 1[

decision-hazard:a(t) is not available at the beginning of period [t, t + 1[

q(t) turbined outflow volume during [t, t + 1[

decided at the beginning of period [t, t + 1[supposed to depend on the stock S(t) but not on the inflow water a(t)chosen such that 0 ≤ q(t) ≤ S(t)



A single dam nonlinear dynamical model in decision-hazard:

equivalent formulation


S(t + 1)︸︷︷︸

future volume

= S(t)︸︷︷︸

volume

− q(t)︸︷︷︸

turbined

− r(t)︸︷︷︸

spilled

+ a(t)︸︷︷︸

inflow volume




decided at the beginning of period [t, t + 1[supposed to depend on the stock S(t) but not on the inflow water a(t)chosen such that 0 ≤ q(t) ≤ S(t)

r(t) =[S(t) − q(t) + a(t) − S♯

]

+the spilled volume



When turbinating decisions are made after knowing the

water inflows, we obtain a linear dynamical model


S(t + 1)︸︷︷︸

future volume

= S(t)︸︷︷︸

volume

− q(t)︸︷︷︸

turbined

− r(t)︸︷︷︸

spilled

+ a(t)︸︷︷︸

inflow volume


a(t), inflow water volume (rain, etc.) during [t, t + 1[;

hazard-decision:a(t) is known and available at the beginning of period [t, t + 1[

q(t) turbined outflow volume and r(t) spilled volume

decided at the beginning of period [t, t + 1[supposed to depend on the stock S(t) and on the inflow water a(t)chosen such that

0 ≤ S(t) − q(t) + a(t) − r(t) ≤ S♯



Complexity increases with interconnected dams



Typology of hydro-valleys

(c)(a) (b)

dams in cascade converging valleys pumping



Sketch of a cascade model with dams i = 1, . . . , N

Dam 2

Dam 3

Dam 1

S1,t

S2,t

S3,t

Q1,t

R1,t Q2,t

R2,t Q3,t

R3,t

A1,t

A2,t

A3,t

ai ,t : inflow into dam i at time t (rain, run off water)Si ,t : volume in dam i at time t (water volume)qi ,t : turbined from dam i at time t (valued at price pi ,t)ri ,t : spilled volume from dam i at time t (irrigation. . . )



Water inflows historical scenarios



Description of variables: ai ,t

Inflow water volume

ai ,t : amount of water inflowing without any control into the dam

Inflow water volumes ai ,t are part of the problem data, and may be

either deterministic, in which case the values (ai ,t0 , . . . , ai ,T ) are unique andare exactly known, for every dam i

or uncertain, and many possibilities can be considered

inflows chronicles may be available:(as

i,t0, . . . , as

i,T ) where s belongs to a set S of historical scenariosin addition, the probability πs of scenario s may be knownmore generally, the sequence (ai,t0 , . . . , ai,T ) may be considered as arandom process with known probability distribution

Prices pi ,t can also be part of the problem data

In the same way, the prices of turbined volumes can be part of the problem data,with an uncertain or deterministic status



Description of variables: qi ,t

Turbined water volume

qi ,t : amount of water turbined to produce electricity

Turbined volumes qi ,t are control variables, the possible values of which arein the hands of the decision-maker to achieve different goals

In the deterministic framework, one looks fora single sequence of values (qi ,t0 , . . . , qi ,T−1) for each dam i

In the uncertain framework, the situation is more complex

one has to specify the online available informationupon which decisions are madethe controls qi,t are also uncertain variablesand they can be obtained by means of feedback policies feeding on onlineinformation



Description of variables: ri ,t

Spilled water volume

ri ,t : amount of water spilled from the dam with or without control

The status of the spilled volumes depends on the context,and ri ,t can represent

an amount taken in the dam (for irrigation purposes, for instance), and canbe a data (deterministic or uncertain)

a volume deliberately taken in the dam and released without being turbined,and it is then a control variable

the amount of water which overflows in case of excess

ri ,t = [Si ,t + ai ,t + qi−1,t − qi ,t − S♯i ]+

and it is then an output variable which results from a calculation based uponother variables



The dynamics describes the temporal evolution

of the water stock volume in the dam

Dam i

Qi−1,t

Si,tQi,t

Ai,t

The water volume in the dam evolves whentime goes by and it is given by

Si ,t+1 = Si ,t + ai ,t + qi−1,t − qi ,t

A general form is

Si ,t+1 = Dyni ,t

(Si ,t , qi ,t, ai ,t , qi−1,t

)



The payoffs are decomposed in instantaneous and final

The payoff obtained by turbining the volume qi ,t is

pi ,tqi ,t

We shall use the general form

Utili ,t

(Si ,t, qi ,t , ai ,t , pi ,t

)

To avoid emptying the dam at the end of the optimization process, weintroduce a “water value” term bearing on the final stock volume

UtilFini

(Si ,T

)



The management of the dams cascade can be formulated

as an intertemporal optimization problem

The payoff attached to the dam cascade is

N∑

i=1

( T−1∑

t=t0

Utili ,t

(Si ,t , qi ,t, ai ,t , pi ,t

)+ UtilFini

(Si ,T

))

This payoff must be optimized

under constraints of dynamics

Si,t+1 = Dyni,t

`

Si,t , qi,t , ai,t , qi−1,t

´

, i ∈ J1, NK, t ∈ Jt0, T−1K

and under bounds constraints

qi,t ∈ˆ

qi, qi

˜

, i ∈ J1, NK, t ∈ Jt0, T−1K

Si,t ∈ˆ

S i , S i

˜

, i ∈ J1, NK, t ∈ Jt0, T K


Moving from deterministic to stochastic dynamic optimization The deterministic optimization problem is well posed









The deterministic optimization problem

In the deterministic case, inflows ai :=(ai ,t0 , . . . , ai ,T

)and prices

pi :=(pi ,t0 , . . . , pi ,T

)are unique and are exactly known, for every dam i

With qi :=(qi ,t0 , . . . , qi ,T−1

)and Si :=

(Si ,t0 , . . . , Si ,T

),

the optimization problem is

max(qi ,Si )i=1,...,N

N∑

i=1

( T−1∑

t=t0

Utili ,t

(Si ,t , qi ,t , ai ,t, pi ,t

)+ UtilFini

(Si ,T

))


Si ,t0 given, i ∈ J1, NK

Si ,t+1 = Dyni ,t

(Si ,t , qi ,t, ai ,t , qi−1,t

), i ∈ J1, NK, t ∈ Jt0, T−1K

and under bounds constraints

qi ,t ∈[q

i, qi

], i ∈ J1, NK, t ∈ Jt0, T−1K

Si ,t ∈[S i , S i

], i ∈ J1, NK, t ∈ Jt0, T K



The deterministic optimization problem is well posed

If, to make things simple, we assume allvariables to be scalar, the optimization mustbe made with respect to all control variables(∈ R

N(T−t0)) and to all “state” variables(∈ R

N(T−t0+1))

The optimization yields for every damtrajectories (q⋆

i ,t0, . . . , q⋆

i ,T−1) and(S⋆

i ,t0, . . . , S⋆

i ,T ) of the controls and of the“states”, as well as the optimal payoff



Here are some characteristics

of the deterministic optimization problem

The resolution of the deterministic optimization problem can be made in theframework of mathematical programming, that is, the maximization of afunction over R

n under equality and inequality constraints

The volumes Si ,t are intermediary variables,completely determined by the choice of the qi ,t

(however, such intermediary variables may be used to compute gradients bymeans of an adjoint state)

The optimization problem is often solved after formulating it as alinear problem and with softwares such as CPLEX

In the nonlinear case, the optimization problem can be difficult to solve, dueto its large size, but it is known how to apply decomposition and coordinationtechniques


Moving from deterministic to stochastic dynamic optimization In the uncertain framework, the optimization problem is not well posed









0 50 100 150 200 250 300 350 4000

10

20

30

40

50

60

70

80

90

100

Stock volumes in a dam following an optimal strategywith a final value of water

(time)

(vol

ume)

Figure: Stock volumes in a dam following an optimal strategy, with a final value of waterMichel DE LARA (Cermics, France) Horizon Maths 2012, Paris, 19 Dec. 2012 December 19, 2012 58 / 112


In the uncertain framework,

two additional questions must be answered

Question (expliciting risk attitudes)

How are the uncertainties taken into accountin the payoff criterion and in the constraints?

Question (expliciting available online information)

Upon which online information are turbinating decisions made?

In practice, one assumes

more or less precise knowledge of the past (online information)

statistical assumptions about the future (offline information)



How are the uncertainties taken into account

in the payoff criterion and in the constraints?

In a probabilistic setting, where uncertainties are random variables,a classical answer is

to take the mathematical expectation of the payoff (risk-neutral approach)

E

( N∑

i=1

( T−1∑

t=t0

Utili ,t

(Si ,t, qi ,t , ai ,t, pi ,t

)+ UtilFini

(Si ,T

)))

and to satisfy all (physical) constrainsts almost surely that is, practically,for all possible issues of the uncertainties (robust approach)

P

(

qi ,t ∈[q

i, qi

], i ∈ J1, NK, t ∈ Jt0, T−1K

Si ,t ∈[S i , S i

], i ∈ J1, NK, t ∈ Jt0, T K

)

= 1



Upon which online information

are turbinating decisions made?

On the one hand, it is suboptimal to restrict oneself,as in the deterministic case, to open-loop controls depending only upon time

On the other hand, it is impossible to suppose that we know in advancewhat will happen for all times: clairvoyance is impossible

The in-between is non anticipativity constraint

In our case, the decision qi ,t will be looked after as a

fonction of past uncertainties,

that is, of the water inflows and the prices(aj,τ , pj,τ

)

for j ∈ J1, NK and τ ∈ Jt0, tK



On-line information feeds turbinating decisions:

the non anticipativity constraint

As a consequence, the control qi ,t is looked after under the form

qi,t = φi,t

`

a1,t0 , p1,t0 , . . . , aN,t0 , pN,t0 , . . . ,

a1,τ , p1,τ , . . . , aN,τ , pN,τ , . . . a1,t−1, p1,t−1, . . . , aN,t−1, pN,t−1

´

Denote the uncertainties at time t by

wt =(a1,t , p1,t , . . . , aN,t, pN,t

)

When uncertainties are considered as random variables (measurablemappings), the above formula for qi ,t expresses the measurability of thecontrol variable qi ,t with respect to the past uncertainties, also written as

σ(qi ,t) ⊂ σ(wt0 , . . . , wt−1

)⇐⇒ qi ,t σ

(wt0 , . . . , wt−1

)



On-line information structure can reflect

decentralized information

When the control qi ,t is looked after under the form

qi ,t = φi ,t

(ai ,t0 , pi ,t0 , . . . , ai ,τ , pi ,τ , . . . , ai ,t−1, pi ,t−1

)

this expresses that each dam is managed with local information



The stochastic optimization problem

In a probabilistic setting, where uncertainties are random variablesand where a probability P is given, a possible formulation is

max(qi ,Si )i=1,...,N

E

„ NX

i=1

“

T−1X

t=t0

Utili,t

`

Si,t , qi,t , ai,t , pi,t

´

+ UtilFini

`

Si,T

´

”

«


Si,t0 given, i ∈ J1, NK

Si,t+1 = Dyni,t

`

Si,t , qi,t , ai,t , qi−1,t

´

, i ∈ J1, NK, t ∈ Jt0,T−1K

under bounds constraints

P

qi,t ∈ˆ

qi, qi

˜

, i ∈ J1, NK, t ∈ Jt0,T−1K

Si,t ∈ˆ

S i , S i

˜

, i ∈ J1, NK, t ∈ Jt0,T K

!

= 1

and under measurability constraints

qi,t σ`

wt0 , . . . , wt−1

´

, i ∈ J1, NK, t ∈ Jt0,T−1K



The outputs of a stochastic optimization problem

are random variables

210000 215000 220000 225000 230000 235000 240000 245000 2500000.0e+00

1.0e−05

2.0e−05

3.0e−05

4.0e−05

5.0e−05

6.0e−05

7.0e−05

8.0e−05

9.0e−05

1.0e−04

Histogram of the optimal payoff with a final value of water


Moving from deterministic to stochastic dynamic optimization There are two ways to express the information constraints









There are two ways to express the information constraints

Functional approach qi ,t = φi ,t

(wt0 , . . . , wt−1

)

In the special case of the Markovian framework with white noise, it can beshown that there is no loss of optimality to look for solutions under the form

qi ,t = φi ,t

(S1,t , . . . , SN,t

)

︸︷︷︸

state

Measurability constraints qi ,t σ(wt0 , . . . , wt−1

)

In the special case of the scenario tree approach, uncertainties are supposed tobe observed, and all possible scenarios

`

w st0

, . . . , w sT

´

, s ∈ S , are organized in atree, and controls qi,t are indexed by nodes on the treeEquivalently, qi,t are indexed by s ∈ S with the constraint that if two scenarioscoincide up to time t, so must do the controls at time t

`

wst0

, . . . , wst−1

´

=`

ws′

t0, . . . , w

s′

t−1

´

⇒ qsi,t = q

s′

i,t



General framework

Stochastic Optimal Control (SOC)

Consider a dynamical system in discrete time influenced by exogenous noise

At each time step:

observations are made on the system and kept in memorya decision/control is chosen, and the system evolves

Our aim is to minimize some objective function that depends on the evolution ofsystem



A motivating example

Typical application of SOC: portfolio management

Consider a power producer which has to

manage an amount xt of stock at each time step t

using decision ut for each time step t

while dealing with stochastic data wt , such as the power demand, unitbreakdowns, market prices

All random variables are written using bold letters



Mathematical formulation

Standard problem

minx,u

E(

T−1∑

t=0

Ct (xt ,ut ,wt+1) + K (xT )),

s.t. xt+1 = ft (xt ,ut ,wt+1) , ∀t = 0, . . . , T − 1,

x0 = w0,

P-a.-s. constraints (e.g. bounds on xt and ut),

ut is Ft-measurable,

where Ft := σw0, . . . ,wt



Classical methodsStochastic Programming

Discretize random inputs of the problemusing scenario trees

Write the constraints and objective functionon this structure (time has becomearborescent)

Use standard numerical methods frommathematical programming (linear/convexprogramming to overcome the combinatorialexplosion of the tree)

Carpentier/Cohen/Culioli (1995); Birge/Louveaux (1997); Pflug (2001);

Bacaud/Lemarechal/Renaud/Sagastizabal (2001); Heitsch/Romisch (2003);

Dupacova/Growe-Kuska/Romisch (2003); Barty (2004); Shapiro/Dentcheva/Ruszczynski (2009)

...



Classical methodsStochastic Programming

Discretize random inputs of the problemusing scenario trees

Write the constraints and objective functionon this structure (time has becomearborescent)

Use standard numerical methods frommathematical programming (linear/convexprogramming to overcome the combinatorialexplosion of the tree)

Complexity of multistage stochastic programs

The amount of scenarios needed to achieve a given precision grows exponentially w.r.t. thenumber of time steps

See Shapiro (2006); Girardeau (2010)



Classical methodsDynamic Programming (1)

Markovian framework

Noise variables wt are independent over time

Cost-to-go function, Bellman function, value function

Vt (x) := minu

E

T−1X

s=t

Cs (xs , us ,ws+1) + K (xT ) | xt = x

!

, ∀x ∈ X,

subject to constraints of the original problem




The Dynamic Programming (DP) principle Bellman (1957)

There is no loss of optimality in looking for the optimal strategy as feedback functionson xt only.Moreover, DP provides a way to compute Bellman functions backwards:

VT (x) = K (x) , ∀x ∈ XT ,

and, for all t = t0, . . . , T − 1:

Vt (x) = minu∈Ut

E(Ct (x , u,wt+1) + Vt+1 (ft (x , u,wt+1))), ∀x ∈ Xt

Reference textbooks: Bertsekas (2000); Puterman (1994); Quadrat/Viot (1999)




Most of the time, the DP equation cannot be solved analytically

Numerical methods have to be used

Curse of dimensionality

The complexity associated with the resolution of the DP equations by discretizationgrows exponentially w.r.t. the dimension of the state space




Most of the time, the DP equation cannot be solved analytically

Numerical methods have to be used

Curse of dimensionality

The complexity associated with the resolution of the DP equations by discretizationgrows exponentially w.r.t. the dimension of the state space

Some attempts of breaking the curse are through functional approximations:

Approximate Dynamic Programming Bellman/Dreyfus (1959); Gal (1989);de Farias/Van Roy (2003); Longstaff/Schwartz (2001); Bertsekas/Tsitsiklis (1996)...

Stochastic Dual Dynamic Programming Pereira/Pinto (1991); Philpott/Guan (2008);Shapiro (2010)



Summary

Stochastic optimization highlights risk attitudes tackling

Stochastic dynamic optimization emphasizes the handling ofonline information

Many open problems, because

many ways to represent risk (criterion, constraints)many information structurestremendous numerical obstacles to overcome


Two snapshots on ongoing research








Two snapshots on ongoing research Decomposition-coordination optimization methods under uncertainty









Decomposition-coordination: divide and conquer

Spatial decomposition

multiple players with their local informationlocal / regional / national /supranational

Temporal decomposition

A state is an information summaryTime coordination realized through Dynamic Programming, by value functionsHard nonanticipativity constraints

Scenario decomposition

Along each scenario, sub-problems are deterministic (powerful algorithms)Scenario coordination realized through Progressive Hedging, by updatingnonanticipativity multipliersSoft nonanticipativity constraints



Problems and methods: a very tentative tableFeel free to comment and criticize! ;-)

Decompose and coordinateSpace Time Scenarios Scenarios Trees

DP PH 2SSP

decentralized units OKstorage OKunit commitment OKshort term market OK

Dynamic Programming: DPProgressive Hedging: PHTwo and Multi-Stage Stochastic Programming : 2SSP

Use specific analytical features

Stochastic Programming: linearity, convexityStochastic Dual Dynamic Programming: linear dynamics, convex Bellmanfunction approximated by “cuts”



We have a nice decomposed problem but. . .Flower structure

We are almost in the case where

units could be driven independently

one from anotherUnit 1

Unit 2

Unit 3

Unit 4

Unit 5

Unit 6

Unit 7

Unit 8



We have a nice decomposed problem but. . .Flower structure

Unfortunately...Unit 1

Unit 2

Unit 3

Unit 4

Unit 5

Unit 6

Unit 7

Unit 8

Couplingconstraint



The associated optimization problem may be written as

min(u1,...,uN)

N∑

i=1

Ji (ui )

︸︷︷︸

costs minimization

underN∑

i=1

Θi(ui ) = D

︸︷︷︸

offer = demand

where

ui is the decision of each unit i

Ji (ui ) is the cost of making decision ui for unit i

Θi(ui ) is the production induced by making decision ui for unit i



Proper prices allow decentralization of the optimum

min(u1,...,uN)

N∑

i=1

Ji (ui ) underN∑

i=1

Θi(ui ) = D

The simplest decomposition/coordination scheme consists in

buying the production of each unit at a price λ(k)

and letting each unit minimizing its costs

minui

Ji (ui ) + λ(k)︸︷︷︸

price

Θi(ui )

then, modifying the price depending on the coupling constraint

λ(k+1) = λ(k) + ρ( N∑

i=1

Θi(ui ) − D)

(like in the “tatonnement de Walras” in economics)



Extension to dynamics and stochasticity

If we explicitely take into account time and uncertainties, a classical approchconsists in writing the new problem

minui,t

t∈0,T−1

i∈1,N

E

( N∑

i=1

( T−1∑

t=0

Li ,t

(xi ,t,ui ,t,wi ,t

)+ Ki

(xi ,T

)))

under the constraints

N∑

i=1

Θi ,t

(xi ,t,ui ,t ,wi ,t

)− dt = 0, t ∈ J0, T−1K

xi ,t+1 = fi ,t(xi ,t,ui ,t ,wi ,t

), i ∈ J1, NK, t ∈ J0, T−1K



We need to specify information constraints

The optimization problem is not well posed, because we have not specifiedupon what depends the control ui ,t of each unit i at each time t!

In the causal and perfect memory case, we say that the control ui ,t dependsof all past noises up to time t

ui ,t = φi ,t

(w1,0, . . . ,wN,0,d0 . . . . . .w1,t , . . . ,wN,t,dt

)

When the control ui ,t is looked after under the form

ui ,t = φi ,t

(wi ,0,d0 . . . . . .wi ,t ,dt

)

this expresses that each unit is managed with local information,which is a way to handle decentralized information



Looking after decentralizing prices models

Going on with the previous scheme, each unit i solves

minui,0,...,ui,T−1

E

„ T−1X

t=0

“

Li,t

`

xi,t , ui,t ,wi,t

´

+ λ(k)i,t Θi,t

`

xi,t , ui,t ,wi,t

´

”

+ Ki

`

xi,T

´

«

under the constraints

xi ,t+1 = fi ,t(xi ,t,ui ,t,wi ,t

), t ∈ J0, T−1K

In all rigor, the optimal controls u⋆i ,t of this problem depend upon xi ,t and

upon all past prices (λ(k)i ,0 , . . . , λ

(k)i ,t )

Research axis: find an approximate dynamical model for the prices, driven by

proper information (for instance, replace λ(k)i ,t by E

(

λ(k)i ,t | yt

)

, where the

information variable yt is a short time memory process)



Dual Approximate Dynamic Programming

λk

Samples/scenarios of

dual variable

at iteration k




λk


dual variable

at iteration k

Subproblem iSubproblem 1 Subproblem N

We solve subproblems

using E(λk |y)

by Dynamic Programming




λk


dual variable

at iteration k



using E(λk |y)


Policy Φ1“

x1, y”

Policy Φi“

xi , y”

Policy ΦN“

xN , y”

We obtain policies




λk


dual variable

at iteration k



using E(λk |y)


Policy Φ1“

x1, y”

Policy Φi“

xi , y”

Policy ΦN“

xN , y”

We obtain policies

We simulate scenarios of strategies and prices and

update prices using a gradient step

λk+1t = λk

t + ρ ×PN

i=1 g it

“

xi,kt , u

i,kt , wt

”

We update prices

using a gradient step




λkAt iteration k + 1



using E(λk |y)


Policy Φ1“

x1, y”

Policy Φi“

xi , y”

Policy ΦN“

xN , y”

We obtain policies

We simulate scenarios of strategies and prices and

update prices using a gradient step

λk+1t = λk

t + ρ ×PN

i=1 g it

“

xi,kt , u

i,kt , wt

”

We update prices

using a gradient step



Extension to interconnected dams



Contribution to dynamic tariffs

Spatial decomposition of a dynamic stochastic optimization problem

Lagrange multipliers attached to spatial coupling constraints are stochasticprocesses (prices)

By projecting these prices, one expects to identify approximate dynamicmodels

Such prices dynamic models are interpreted as dynamic tariffs


Two snapshots on ongoing research Risk constraints in optimization









Dam management under “tourism” constraint

Maximizing the revenuefrom turbinated water

under a tourism constraintof having enough waterin July and August



Stock trajectories



A single dam nonlinear dynamical model in decision-hazard


S(t + 1)︸︷︷︸

future volume

= minS♯, S(t)︸︷︷︸

volume

− q(t)︸︷︷︸

turbined

+ a(t)︸︷︷︸

inflow volume



decision-hazard:a(t) is not available at the beginning of period [t, t + 1[


decided at the beginning of period [t, t + 1[supposed to depend on S(t) but not on a(t)chosen such that 0 ≤ q(t) ≤ S(t)



The traditional economic problem is

maximizing the expected payoff

Suppose that

a probability P is given on the set Ω = RT−t0 of water inflows scenarios

`

a(t0), . . . , a(T − 1)´

turbined water q(t) is sold at price p(t), related to the price at which energycan be sold at time t

at the horizon, the final volume S(T ) has a value given by UtilFin`

S(T )´

,the “final value of water”

The traditional (risk-neutral) economic problem is to maximize theintertemporal payoff (without discounting if the horizon is short)

max E

T−1∑

t=t0

price︷︸︸︷

p(t)turbined

q(t)︸︷︷︸

+

final volume utility︷︸︸︷

UtilFin(S(T )

)



Dam management under “tourism” constraint

Traditional cost minimization/payoff maximization

max E

T−1∑

t=t0

turbined water payoff︷︸︸︷

p(t)q(t) +


UtilFin(S(T )

)

For “tourism” reasons:

volume S(t) ≥ S , ∀t ∈ July, August

In what sense should we consider this inequality which involves therandom variables S(t) for t ∈ July, August ?



Robust / almost sure / probability constraint

Robust constraints: for all the scenarios in a subset Ω ⊂ Ω

S(t) ≥ S , ∀t ∈ July, August

Almost sure constraints

Probability

water inflow scenarios along whichthe volumes S(t) are above the

threshold S for periods t in summer

= 1

Probability constraints, with “confidence” level p ∈ [0, 1]

Probability

water inflow scenarios along whichthe volumes S(t) are above thethreshold S for periods t in summer

≥ p

and also by penalization (to be discussed later), or in the mean, etc.



Environmental/tourism issue leads to dam management

under probability constraint

The traditional economic problem is max E [P(T )]where the payoff/utility criterion is

P(T ) =

T−1∑

t=t0

turbined water payoff︷︸︸︷

p(t)q(t) +


UtilFin(S(T )

)

and a failure tolerance is accepted

Probability

water inflow scenarios along whichthe volumes S(t) ≥ S

for periods t in July and August

≥ 99%



Stock trajectories



An improvement compared to standard procedures



However, though the mean is optimal,

the payoff effectively realized can be far from it



We propose a stochastic viability formulation

to treat symmetrically andto guarantee both environmental and economic objectives

Given two thresholds to be guaranteed

a volume S (measured in cubic hectometers hm3)a payoff P (measured in numeraire $)

we look after policies achieving the maximal viability probability

Π(S, P) = max Proba

water inflow scenarios along which

the volumesS(t) ≥ S

for all time t ∈ July, August

and the final payoff P(T ) ≥ P

The maximal viability probability Π(S, P) is the maximal probability toguarantee to be above the thresholds S and P



A stochastic viability formulationState, control and dynamic

The state is the couple x(t) =(S(t), P(t)

)volume/payoff

The control u(t) = q(t) is the turbined water

The dynamics is

S(t + 1)︸︷︷︸

future volume

= minS♯, S(t)︸︷︷︸

volume

− q(t)︸︷︷︸

turbined

+ a(t)︸︷︷︸

inflow volume

,

t = t0, . . . , T − 1

P(t + 1)︸︷︷︸

future payoff

= P(t)︸︷︷︸

payoff

+ p(t)q(t)︸︷︷︸

turbined water payoff

, t = t0, . . . , T − 2

P(T ) = P(T − 1) + UtilFin(S(T )

)

︸︷︷︸

final volume utility



A stochastic viability formulationState and control constraints

The control constraints are

u(t) ∈ B(t, x(t)

)⇐⇒ 0 ≤ q(t) ≤ S(t)

The state constraints are

x(t) ∈ A(t) ⇐⇒

S(t) ≥ S , ∀t ∈ July, August

P(T ) ≥ P



Dynamic programming equation

Abstract version

V (T , x) = 1A(T )(x)

V (t, x) = 1A(t)(x) maxu∈B(t,x)

Ew(t)

h

V“

t + 1, Dyn`

t, x , u, w(t)´

”i

Specific version

V (T ,S, P) = 1P≥P

V (T − 1,S, P) = max0≤q≤S

Ea(t)

h

V“

t + 1, S − q + a(t), P + UtilFin`

S´

”i

V (t,S, P) = max0≤q≤S

Ea(t)

h

V“

t + 1, S − q + a(t), P + pq”i

, t 6∈ July, August

V (t,S, P) = 1S≥S max0≤q≤S

Ea(t)

h

V“

t + 1,S − q + a(t), P + pq”i

,

t ∈ July, August



Maximal viability probability Π(S , P)

as a function of guaranteed thresholds S and P

For example, the probabilityto guarantee

a final payoff aboveP = 1 Meuros

and a volume aboveS = 40 hm3 in Julyand August

is about 90%



Iso-values for the maximal viability probability

as a function of guaranteed thresholds S and P



Contribution to quantitative sustainable management

Conceptual framework forquantitative sustainablemanagement

Managing ecological and economicconflicting objectives

Displaying tradeoffs betweenecology and economy sustainabilitythresholds and risk


Conclusion: a need for training









Trends are favorable to statistics and optimization

More telecom technology→ more data

More data, more unpredicability→ more statistics

More unpredicability → more storage→ more dynamic optimization

More unpredicability→ more stochastic dynamic optimization



Opportunities for higher education

Masters

Mathematiques, Informatique et Applications

Economie du Developpement Durable, de l’Environnement et de l’Energie

Renewable Energy Science and Technology Master ParisTech

and others ;-)

Research centers

CERMICS (mathematics and scientific computing)Electricite de France Research & Development Departement OSIRISOptimization, simulation, risks and statistics for the energy marketsand others ;-)

Consulting positions for Master and PhD graduates: ARTELYS, SETEC, etc.



Pedagogical material

CEA / EDF / INRIA Summer School on Stochastic Optimization slideshttp://www-hpc.cea.fr/SummerSchools2012-SO.htm

Advanced Course on Sustainable Optimization slideshttp://cermics.enpc.fr/~delara/ENSEIGNEMENT/

Sustainable_Optimization/Sustainable_Optimization.html

Scicoslab practical computer works http://cermics.enpc.fr/~scilab

More on the Optimization and Systems web pagehttp://cermics.enpc.fr/equipes/optimisation.html

More on my web pagehttp://cermics.enpc.fr/~delara/



Some related initiatives

Gaspard Monge Program for Optimization and operations researchhttp://www.fondation-hadamard.fr/pgmo

From January 7 to April 5, 2013, Institut Henri Poincare (IHP) thematicquarter Mathematics of Bio-Economics as a contribution to Mathematics ofPlanet Earth - MPE2013http://www.ihp.fr/en/ceb/mabies

From January 7 to March 1, 2013, Master Course on Stochastic Optimization(MMMEF)http://cermics.enpc.fr/equipes/optimisation.html

From April 8 to April 12, 2013, CIRM School on Stochastic Control for theManagement of Renewable Energies as a contribution to Mathematics ofPlanet Earth - MPE2013http://cermics.enpc.fr/~delara/ENSEIGNEMENT/

CIRM2013/Optim_ENR_CIRM2013/

More on my web page http://cermics.enpc.fr/~delara/



References

Bacaud, L., Lemarechal, C., Renaud, A., and Sagastizabal, C. A. (2001). Bundle methods in stochasticoptimal power management: A disaggregated approach using preconditioner. Computational Optimizationand Applications, 20(3):227–244.

Barty, K. (2004). Contributions a la discretisation des contraintes de mesurabilite pour les problemes

d’optimisation stochastique. These de doctorat, Ecole Nationale des Ponts et Chaussees.Barty, K., Carpentier, P., and Girardeau, P. (2010). Decomposition of large-scale stochastic optimal control

problems. RAIRO Operations Research, 44(3):167–183.Bellman, R. (1957). Dynamic Programming. Princeton University Press, New Jersey.Bellman, R. and Dreyfus, S. E. (1959). Functional approximations and dynamic programming. Math tables

and other aides to computation, 13:247–251.Bertsekas, D. P. (2000). Dynamic Programming and Optimal Control. Athena Scientific, 2 edition.Bertsekas, D. P. and Tsitsiklis, J. N. (1996). Neuro-Dynamic Programming. Athena Scientific.Birge, J. R. and Louveaux, F. V. (1997). Introduction to Stochastic Programming. Springer Series in

Operations Research. Springer Verlag, New York.Carpentier, P., Cohen, G., and Culioli, J.-C. (1995). Stochastic optimal control and decomposition-coordination

methods. In Durier, R. and Michelot, C., editors, Recent Developments in Optimization. Lecture Notes inEconomics and Mathematical Systems, 429, pages 72–103. Springer Verlag, Berlin.

de Farias, D. P. and Van Roy, B. (2003). The Linear Programming Approach to Approximate DynamicProgramming. Oper. Res., 51(6):850–856.

Dupacova, J., Growe-Kuska, N., and Romisch, W. (2003). Scenario reduction in stochastic programming. Anapproach using probability metrics. Mathematical Programming, 95(3):493–511.

Gal, S. (1989). The Parameter Iteration Method in Dynamic Programming Management Science,35(6):675–684.

Girardeau, P. (2010). A comparison of sample-based Stochastic Optimal Control methods. StochasticProgramming E-Print Series, http://www.speps.org. submitted to Optimization and Engineering.

Heitsch, H. and Romisch, W. (2003). Scenario Reduction Algorithms in Stochastic Programming.Computational Optimization and Applications, 24:187–206.

Longstaff, F. A. and Schwartz, E. S. (2001). Valuing american options by simulation: A simple least squaresapproach. Review of Financial Studies, 14(1):113–147.Michel DE LARA (Cermics, France) Horizon Maths 2012, Paris, 19 Dec. 2012 December 19, 2012 112 / 112

http://www.speps.org

Documents

Smart Power Systems: the Optimization Challenge · 2013-09-17 · Smart Power Systems: the Optimization Challenge Michel DE LARA Cermics, ´Ecole des Ponts ParisTech, France and Pierre