Thèse de doctoratMathématiques Appliquées

pour obtenir le grade deDocteur de l'École Polytechnique

LNG portfolio optimization

approach by stochastic programming techniques

présentée parZhihao CEN

sous la direction deJ.Frédéric BONNANS

soutenue publiquement le 22 nov. 2011

Jury

Président: Emmanuel GOBET CMAP, École PolytechniqueRapporteurs: René HENRION Weierstrass Institute, Berlin

Gilles PAGÈS LPMA, UPMCExaminateurs: Pierre BONAMI LIF, Aix Marseille Université

Thibault CHRISTEL TotalMichel DE LARA CERMICS, ENPC

Directeur de thèse J.Frédéric BONNANS INRIA & CMAP, École Polytechnique

Centre de Mathématiques Appliquées UMR 7641INRIA Saclay - Île de France

Acknowledgement

This thesis is the fruit of three years of research I spent between the COMMANDSteam of the Center of Applied Mathematics (CMAP) at Ecole Polytechnique, and theQuantitative Analysis team of Total Gas and Power. During this time, I have had thepleasure to work with several people whose various contributions where invaluable tothe accomplishment of my research and to the production of this manuscript. It is mypleasure to convey my sincere gratitude to them all.

Foremost, I thank my thesis supervisor Frédéric Bonnans, for his constant support,encouragement, enthusiasm, and outstanding guidance in all aspect of the work: fromthe research axis, to the details of the algorithm. I could never have hoped for better su-pervision. I immensely appreciate the excellent example he has set for me as a researcher,mathematician and professor.

My special thanks also goes to Thibault Christel, my advisor at Total. I appreciatehis guidance in my rst stage of my thesis, and his numerous remarks regarding the nalarticles and manuscript.

It is an honor for me to have René Henrion and Gilles Pagès as my rapporteurs.Many thanks is send to all members of the thesis jury: Pierre Bonami, Michel De Lara,Emmanuel Gobet.

I gratefully acknowledge Wallis, Nasséra, Sandra, Alexandra, the assistants at CMAP,for providing a pleasant environment for research and work. Many thanks to Sylvain,our IT assistant, for his constant support.

My time at CMAP, Ecole Polytechnique was made enjoyable in large part due to themany colleagues: Khalil, Emilie, the two Camille(s), Soledad, Francisco, Florent, Clé-ment, Zixian, Xavier, Laurent, Xiaolu, Chao, Khalid, Maxime, Olivier, Michaël, Ankit.A special thanks is given to Sylvie for her club of Belgium chocolate.

My heartfelt gratitude goes to the Quantitative Analysis team at Total Gas andPower for taking me in as an intern three years ago and for the time I spent with duringmy rst stage as Ph.D. student. Special thanks are given to J-C. Prevel for trusting meto work on this research opportunity, and to A. Boisson for providing me with a greatresearch environment.

I also want to mention some other colleagues for their useful discussion and for theirexcellent support: O. Soldatos, C. Zhang, G. Legrand, D. Besombes, J. Chabaud.

I also give thanks to Total SA. and the Association National Recherche Technologie(ANRT) for providing funding for this research.

i

I would like to express my gratitude to all my professors of my second year masterOJMC: P. Combette, S. Sorin, S. Gaubert, J-B. Lasserre as well as those of my thirdyear of study in ENSTA: P. Carpentier, J-P. Laumond, who gave me the solid knowledgeand deep understanding in optimization, control theory and their various applications,and encouraged me to continue my study as a Ph.D. student.

Last but not least, I would like to thank my family for all their love and encourage-ment. For my parents and my grand parents who raised me with a deep love and supportme in all my pursuits. And my thanks go to all my close friends in China to give metheir support during this period.

Thank you.

ii

Abstract

The work presented in this Ph.D dissertation is motivated by the problem of manage-ment of a eet of cargos transporting liqueed natural gas (LNG) initially proposed byTotal. The holder of the portfolio has to meet its commitments towards its counter-parts while trying to generate prots through arbitrating dierent commodities market.Thus, the management of portfolio can be modeled as a stochastic, dynamic and integeroptimization problem.This Ph.D dissertation is organised as follows:Chapter I We rst present the LNG portfolio management problem and give the

mathematical model. Then we summarize the main results of this work.

Chapter II We introduce a numerical method for solving continuous relaxation prob-lem. We propose an algorithm based on the combination of the vecto-rial quantization method as discretization method and the dual dynamicprogramming approach. We show the convergence of numerical schemaand give the error analysis on the discretization by quantization. Somenumerical tests on real energy market problem are performed.

Chapter III We also study the risk averse optimization by using conditional value atrisk (CVaR) as criterion. We show that the algorithm proposed in chap-ter II is also adapted to such formulation. Furthermore, we propose thetechnique of changes in probability measure in stochastic programmingin order to improve rare scenario simulation. Same numerical test as inChapter II is performed in order to make comparison.

Chapter IV We study the sensitivity of the portfolio with respect to several param-eters in the market price model. We proposed a numerical method tocompute sensitivity value based on Danskin's theorem. The convergenceof sensitivity value of discretized problem to the one of original problemis proved. Comparison between result obtained by algorithm in chapterII and other classical methods are provided.

Chapter V We study the stochastic integer programming problem. The integralitycutting plane method is applied to approximate the integer problem.We show that it is impossible to converge to the integer solution becauseof the non convexity and discontinuity of the Bellman value function.We apply a heuristic method and propose a small improvement. Somenumerical tests are also provided.

iii

Keywords: stochastic programming, vectorial quantization method, dual dynamic pro-gramming, sensitivity analysis, integer programming, Fenchel cut, portfolio manage-ment.

iv

Contents

1 Introduction 1

1.1 Context and motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1.1 Contract example . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.1.2 LNG portfolio optimization problem . . . . . . . . . . . . . . . . . 4

1.2 Summary of results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.2.1 Resolution of stochastic programming . . . . . . . . . . . . . . . . 61.2.2 Apply SDDP in risk aversion optimization . . . . . . . . . . . . . . 91.2.3 Sensitivity analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 111.2.4 Heuristic method for stochastic integer program . . . . . . . . . . . 12

1.3 Organization of thesis dissertation . . . . . . . . . . . . . . . . . . . . . . 13

2 Resolution of continuous relaxation problem 17

2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172.2 Problem formulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

2.2.1 Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192.2.2 Dynamic programming principle . . . . . . . . . . . . . . . . . . . 202.2.3 Feasibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202.2.4 Stochastic dual dynamic programming (SDDP) formulation . . . . 21

2.3 Vectorial Quantization Tree . . . . . . . . . . . . . . . . . . . . . . . . . . 222.3.1 Optimal quantization . . . . . . . . . . . . . . . . . . . . . . . . . . 232.3.2 Quantization (recombining) tree . . . . . . . . . . . . . . . . . . . 242.3.3 Dual dynamic programming formulation on quantization tree . . . 262.3.4 Aggregated formulation . . . . . . . . . . . . . . . . . . . . . . . . 28

2.4 Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292.4.1 Forward pass . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292.4.2 Backward pass . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292.4.3 Convergence result and error analysis on quantization . . . . . . . 32

2.5 Numerical tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332.5.1 Price model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332.5.2 Swing option . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342.5.3 Dynamic portfolio optimization . . . . . . . . . . . . . . . . . . . . 36

2.6 Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

v

2.6.1 Discussion on the approximation method as rst approach in re-mark 2.3.4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

2.6.2 Proofs of error analysis theorems . . . . . . . . . . . . . . . . . . . 42

3 Applying SDDP algorithm on risk averse optimization 49

3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 493.2 Measure of risk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 503.3 SDDP algorithm on conditional value at risk . . . . . . . . . . . . . . . . . 513.4 Application of changes in probability measure . . . . . . . . . . . . . . . . 54

3.4.1 Changes in probability measure in dynamic programming principle 553.4.2 Changes in probability measure in SDDP algorithm . . . . . . . . . 563.4.3 Choose drift φ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

3.5 Numerical tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 613.5.1 Result of static conditional value at risk without changes in prob-

ability measure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 623.5.2 Result of static conditional value at risk with changes in probability

measure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

4 Sensitivity analysis with respect to market price 71

4.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 714.2 A review of quantization discretization and stochastic dual dynamic pro-

gramming approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 734.2.1 Discretization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 734.2.2 Stochastic dual dynamic programming algorithm . . . . . . . . . . 74

4.3 Price model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 764.4 Sensitivity analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

4.4.1 Danskin's theorem and its applications . . . . . . . . . . . . . . . . 774.4.2 Convergence of sensitivity estimate . . . . . . . . . . . . . . . . . . 80

4.5 Algorithm and numerical tests . . . . . . . . . . . . . . . . . . . . . . . . . 854.5.1 Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 854.5.2 Comparison of methods . . . . . . . . . . . . . . . . . . . . . . . . 854.5.3 Swing option . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 874.5.4 Small commodity portfolio case study . . . . . . . . . . . . . . . . 89

4.6 Appendix Implicit scheme of nite dierence for 1 dimension PDE . . . 91

5 Study of multistage SMIP 97

5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 975.2 Dynamic programming principle . . . . . . . . . . . . . . . . . . . . . . . 985.3 Feasible set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1005.4 First heuristic method in forward pass . . . . . . . . . . . . . . . . . . . . 1025.5 Integer cutting plan technique . . . . . . . . . . . . . . . . . . . . . . . . . 105

5.5.1 Case when ut ∈ convUint,adt (xt) . . . . . . . . . . . . . . . . . . . . 106

5.5.2 Case when ut /∈ Uint,adt (xt) . . . . . . . . . . . . . . . . . . . . . . . 107

vi

5.5.3 Case when ut ∈ Utint,ad(xt) \ convU

int,adt (xt) . . . . . . . . . . . . . 108

5.6 Numerical test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1105.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112

6 Conclusion and perspective 115

6.1 Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1156.2 Perspectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116

A Approximation by quantization 119

B Cutting plane in integer programming 125

B.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125B.2 Binary variable by lifting and projecting methods . . . . . . . . . . . . . . 126

B.2.1 Balas, Ceria and Cornuéjols . . . . . . . . . . . . . . . . . . . . . . 126B.2.2 Sherali and Adams . . . . . . . . . . . . . . . . . . . . . . . . . . . 126B.2.3 Lovász and Schrijver . . . . . . . . . . . . . . . . . . . . . . . . . . 127

B.3 General integer variable . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127B.3.1 Gomory cut technique . . . . . . . . . . . . . . . . . . . . . . . . . 128B.3.2 Mixed integer rounding inequality . . . . . . . . . . . . . . . . . . 128B.3.3 Disjunctive cut . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128B.3.4 Fenchel cut technique . . . . . . . . . . . . . . . . . . . . . . . . . 129

vii

viii

Chapter 1

Introduction

The work presented in the Ph.D thesis dissertation deals with numerical methods forportfolio optimization. This study is motivated by practical challenges encountered byTotal, in liqueed natural gas (LNG) trading activity.

From a mathematical point of view, an LNG portfolio optimization is a multistagestochastic integer program. It is among the most challenging problems in mathematicalprogramming because it combines two generally dicult classes of problems: stochasticprograming and discrete optimization. Despite the fact that the mathematical formu-lation of LNG portfolio optimization has some particular structure, it is still far frombeing totally resolved. Moreover, approximation and heuristic method generally do notprovide good solutions and they are not ecient in terms of algorithm complexity.

In the thesis, we focus on the stochastic programming aspect. In other words, westudy the mathematical properties and the numerical methods associated to the continu-ous relaxation of the stochastic integer problem. It separates into three parts: computingthe optimal value and the optimal strategy in risk neutral framework; computing the op-timal value and the optimal strategy of risk averse optimization; sensitivity analysis ofoptimal value with respect to some parameters of the random process model. At theend, dicult points in integer programming have been discussed and some numericalmethods have been applied in our problem when combining of stochastic programmingtechniques.

This chapter is a general introduction of the thesis dissertation. It is organized asfollows: in section 1.1, we present in detail the LNG portfolio optimization problemand introduce the associated mathematical formulation. In section 1.2, we give a briefsummary of the results of the thesis. Finally, in section 1.3, we present the structure ofthe thesis dissertation.

1.1 Context and motivation

Let us rst introduce the characterises of a LNG trading portfolio. It is a shippingportfolio which is composed of supply and delivery contracts, and Total has to meet itscommitments toward its counterparts while trying to generate prots through arbitrating

1

1. Introduction

dierent commodity markets. The gure 1.1 gives an illustration of a simplied LNG

Figure 1.1: A ctive supply and demand portfolio, as well as the possible routes. N:producing country; •: consuming country.

portfolio. This example is the main numerical example used throughout this thesisdissertation.

At each time stage before expiration of the contract, the portfolio holder observes thecommodity spot price and then determines the optimal cargo number to allocate on eachroute. For economical reasons, cargos are always supposed to be fully charged. Then, theportfolio holder receives the dierence between the price between supplying market anddestination market times LNG quantity transported on route minus the shipping costs.The decision on the quantity to be transported is subject to volumetric constraints suchas local constraints and global constraints at each port, as well as limitation of someroutes.

1.1.1 Contract example

We give a brief example of supply purchase agreement (SPA). Contract is similar on thedestination side.

Notation Before describing one contract, we introduce some notations.

• t ∈ 0, 1, . . . , T is the index of time step;

• s ∈ S is the index of cargo size;

• cps is the capacity of cargo of type s;

• lp ∈ LP is the index of supplying port;

• ap ∈ AP is the index of destination port;

• ulp→apt,s is the number of cargo of type s send from lp to ap at time t;

2 Mon 5th Dec, 2011 10:04

1.1. Context and motivation

• ξt is the commodity index prices on dierent market at time t;

• clpt (ξt) is the running cost unit at time t of supplying port lp, which is a functionof commodity index prices ξt. Similarly we have cap

t (ξt) at destination side.

Monthly constraint One monthly constraint (or generally called local constraint)means the constraints on total LNG quantity leaving the port lp at time step t:

U lpt ≤

∑s

∑ap

cps · ulp→apt,s ≤ U

lpt , (1.1)

where U lpt (resp. U

lpt ) is the lower bound (resp. the upper bound) of quantity leaving the

port lp at time step t.Some contracts could also stipulate on the cargo number to be delivered:

U lpt ≤

∑s

∑ap

ulp→apt,s ≤ U

lpt , (1.2)

then U lpt (resp. U

lpt ) is the lower bound (resp. the upper bound) of the number of cargos

leaving the port lp at time step t.Mathematically, we can write the constraint in following form:

ut ∈ Ut ∩ Zn (1.3)

where Ut is a polyhedron set. We say that ut is a control variable.

Annual constraint One annual constraint (or generally called global constraint) meansthe constraints related to the total LNG quantity leaving the port lp in all horizon[0, T − 1].

U lp ≤T−1∑t=0

∑s

∑ap

cps · ulp→apt,s ≤ U

lp(1.4)

where U lp(resp. Ulp) is the lower bound (resp. the upper bound) of quantity leaving the

port lp in horizon [0, T − 1].Mathematically, we can write the constraint in following form:

T−1∑t=0

Atut ∈ XT (1.5)

where XT is a polyhedron set. We can then naturally introduce the state variable xt =∑t−1s=0 Asus. The annual constraints (1.5) are equivalent to

xt+1 = xt + Atut;x0 = 0, xT ∈ XT .

(1.6)

Mon 5th Dec, 2011 10:04 3

1. Introduction

Except the annual constraints, some other constraints such as seasonal constraint,can also be written in state constraint (1.6). Let us consider a seasonal constraint:

U lpT0,T1

≤T1∑

t=T0

∑s

∑ap

cps · ulp→apt,s ≤ U

lpT0,T1

(1.7)

We can still write the constraint as a state constraint (1.6) by just setting At = 0, t /∈[T0, T1].

Price formula The nal element of a contract is the price formula. It is a functionwhich takes commodity index at dierent market ξt as variable and denes the unit costprice clp

t at supplying port lp at time step t. Then the total cost at port lp is

T−1∑t=0

∑s

∑ap

clpt · cps · ulp→ap

t,s (1.8)

Mathematically, we can writeT−1∑t=0

ct(ξt)ut (1.9)

the total cost in period [0, T − 1].

1.1.2 LNG portfolio optimization problem

Thus, the LNG portfolio optimization problem maximizes the revenue subject to alllocal and global constraints. Mathematically, we can formulate the problem as follows.First of all, we have a discrete time Markov process (ξt), 0 ≤ t ≤ T in the probabilityspace L2(Ω, (Ft),P; Rd), where Ft denotes the canonic ltration associated with (ξt) :Ft := σ (ξs, 0 ≤ s ≤ t). The problem under consideration has the following expression:

inf E

[T−1∑t=0

ct(ξt) · ut + g(ξT , xT )

]subject to ut ∈ Ut ∩ Zn,Ft −measurable,

xt+1 = xt + Atut,

x0 = 0, xT ∈ XT almost surely;

(1.10)

where ut ∈ Zn is the integer control variable, xt ∈ Rm is the state variable, At ∈ Rm×n isthe technique matrix, ct(·) : Rd → Rn is running cost unit, and nally g(ξ, x) : Rd×Rm →R is a penalty function for nal state xT .

Remark 1.1.1. In fact, problem (1.10) has a particular structure. The most importantpoint in the formulation (1.10) is that random variable is only involved in criteria, insteadof right hand side of constraints in classical stochastic programming formulation. Thisconsists an important advantage making the feasible set of control variable independentof the random variable. In other words, the feasible set is deterministic.

4 Mon 5th Dec, 2011 10:04

1.2. Summary of results

Remark 1.1.2. This mathematical formulation (1.10) is not limited to LNG portfoliooptimization. Many energy/commodity market problems, or some problems in generalnancial market can be stated in this formulation.

• Gas supply contracts pricing (swing option). A swing option provides its holderthe right to exercise one and only one call or put on any one of a number ofspecied exercise dates (this latter aspect is Bermudan). Penalties are imposed onthe buyer if the net volume purchased exceeds or falls below specied upper andlower limits. It allows the purchaser to "swing" the price of the underlying asset.This option is primarily used in energy trading. 1

• Gas storage contract problem. A gas storage contract gives its purchaser the rightto use the gas storage. The contract holder could inject and withdraw gas ateach time step subject to some volumetric constraints and physical operationalconstraints. We can view the contract as a option with three actions: inject,withdraw or do nothing.

• General multi exercise option.

Most examples are one dimensional (control variable and state variable), see Bernhart[22] for recent development. The LNG portfolio optimization can be view as a highdimensional multi exercise option pricing problem.

1.2 Summary of results

We summarize in this section the main mathematical properties and numerical methodsstudied in the thesis. The thesis focus on three parts around the LNG portfolio optimiza-tion. In the rst part, we are interested in computing the optimal value and the optimalstrategy by some particular numerical method for both risk neutral and risk averse per-spective. In the second part, for the reason of trading activity, we are interested in thesensitivity information with respect to underlying commodity price model parameters.In the third part, we return to the stochastic integer problem and discuss the dicultpoints, and we nally give a heuristic method for the stochastic integer problem.

1This explication on swing option is taken from Wikipedia http://en.wikipedia.org/wiki/Option_

style.

Mon 5th Dec, 2011 10:04 5

1. Introduction

1.2.1 Resolution of stochastic programming

In this part of work, we focus on nding a numerical algorithm adapted to the continuousrelaxation problem of (1.10)

inf E

[T−1∑t=0

ct(ξt) · ut + g(ξT , xT )

]subject to ut ∈ Ut,Ft −measurable,

xt+1 = xt + Atut,

x0 = 0, xT ∈ XT almost surely;

(1.11)

Because high dimension of the control variable and state variable, classical methods innancial engineering such as PDE method (see for example Achdou and Pironneau [1])and regression method (see for example Bertsekas [9] and Tsitsikli [32, 33]) are not welladapted here. Thus, we turn to stochastic programming methods.

For surveys on stochastic programming, we refer to [10, 19, 30]. Unless the underlyingrandom space has nite support, our model is an optimization problem over innite-dimensional function spaces, which is dicult to solve. Analytical solutions are notavailable except for extremely simple and unrealistic cases. Numerical approximationmethods have been largely studied in the literature.

Discretization

Most approximation schemes are based on the discretization of the underlying randomspace by a scenario tree. A survey and evaluation of popular scenario generation tech-niques is provided in Kaut andWallace [20]. We choose the vectorial quantization methodintroduced by Pagès [5, 4] to discretize the random process. They establish in [3] an errorestimation of discretization by quantization on obstacle problem. The main idea of vecto-rial quantization is to approximate a random variable ξ taking value in an innite set (oruncountable set) by a random variable ξ taking value in a nite set Γ =

ξ1, ξ2, . . . , ξN

.

We say that the quantization is optimal if it minimizes the discretization error:

min‖ξ − ξ‖ : ξ ∈ Γ,#(Γ) ≤ N

(1.12)

The vectorial quantization tree is an approximation of discrete time random process bya nite state Markov chain such that at each time step, the nite random variable ξt

is an optimal quantization of the original random variable ξt. Pagès provides in [4] aheuristic method to build an optimal quantization tree, called the competitive learningvector quantization (CLVQ) algorithm.

In the thesis, we briey develop the approximation method by quantization. Let fbe a function. The original idea is to approximate the function value at point ξ by thevalue on its nearest quantized point ξ:

f(ξ)← f(ξ), ξ = projΓ(ξ). (1.13)

6 Mon 5th Dec, 2011 10:04

1.2. Summary of results

In other words, f is approximated by a piecewise constant function. We give anotheridea of approximation by using Delaunay triangulation build by the quantized points Γ.Then, the approximation now is:

f(ξ)← f(ξ) :=∑

i

λif(ξi) (1.14)

where ξiis the extreme point of the Delaunay triangle where nd projconv(Γ)(ξ) and λi is

the coecient associated to ξiof Delaunay triangle. The right hand side of approximation

(1.14) is a continuous, piecewise linear function.

Numerical method

The problem under study (1.11) can be naturally decomposed stage-wisely and respectthe dynamic programming principle. Let x[t] = (x0, x1, . . . , xt) (resp ξ[t] = (ξ0, ξ1, . . . , ξt))be the whole history of state variable (resp. of random variable) until time step t. Wedenote Q(t, x[t], ξ[t]) the Bellman value function (or cost-to-go function)

Q(t, x[t], ξ[t]) := essinf E

[T−1∑s=t

cs(ξs)us + g(ξT , xT )∣∣∣∣Ft

]subject to us ∈ Us, Fs −measurable,

xs+1 = xs + Asus,

xT ∈ XT almost surely.

(1.15)

By Markov property of (xt, ξt), we rst prove that Q is a function of (xt, ξt). Applyingthe dynamic programming principle, we can write

Q(t, xt, ξt) = essinf ct(ξt)ut +Q(t + 1, xt+1, ξt)subject to ut ∈ Ut, Ft −measurable,

xt+1 = xt + Atut,

(1.16)

where Q(t + 1, xt+1, ξt) is the conditional expectation of Q(t + 1, xt+1, ξt+1)

Q(t + 1, xt+1, ξt) := E[Q(t + 1, xt+1, ξt+1) | Ft

]. (1.17)

After discretizing by vectorial quantization method, we build a Markov chain (ξt)based on the tree and apply stochastic programming techniques to solve the continuousrelaxation problem (1.11). The method proposed is a combination of L-shape methodand stochastic dual dynamic programming (SDDP) algorithm.

• Stochastic programming dates back to the pioneering study by Dantzig [14]. Forthe continuous stochastic programming problem with recourse, well-known L-shapemethod is rst introduced by Van Slyke and Wets [31]. This approach is basedupon Benders' decomposition [8]. Now, L-shape method can be view as a classical

Mon 5th Dec, 2011 10:04 7

1. Introduction

method for some cases of stochastic programming problem. We can refer to Birgeand Louveaux [10] for more detail of the algorithm. The algorithm principle is basedon feasibility cut to build local feasible set of control variable ut and on optimalitycut to approximate the Bellman value Q(t, xt, ξt). In our problem, feasibility cutsare independent of random process as observed in remark 1.1.1, it can be read as

λxt xt + λu

t ut ≤ λ0t , (λx

t , λut , λ0

t ) ∈ It. (1.18)

Optimality cuts can be generally written as

Q(t, xt, ξt) ≥ λxt xt + λ0

t , (λxt , λ0

t ) ∈ O(ξt). (1.19)

Furthermore, we can also approximate conditional expectation Q(t, xt, ξt−1) byoptimality cuts

Q(t, xt, ξt−1) ≥ ϑ(t, xt, ξt−1,Ot) := supλxt xt + λ

0t : (λx

t , λ0t ) ∈ O(ξt−1). (1.20)

Thus, the dynamic programming formulation (1.16) can be read as

Q(t, xt, ξt) = essinf ct(ξt) · ut + ϑ(t + 1, xt+1, ξt,Ot)subject to xt+1 = xt + Atut,

ut ∈ Ut,Ft −measurable,

(feasibility cut) λxt xt + λu

t ut ≤ λ0t , (λx

t , λut , λ0

t ) ∈ It,

(optimality cut)

ϑ(t + 1, xt+1, ξt,Ot) =

∑ξi

t∈T (ξt)λiϑ(t + 1, xt+1, ξ

it,Oi

t)

ϑ(t + 1, xt+1, ξit,Oi

t) ≥ λxt xt + λ

0t , (λx

t , λ0t ) ∈ Oi

t

(1.21)

• SDDP algorithm has been rst introduced by Pereira and Pinto [23] to solve theproblem where randomness only appears on right-hand-side of constraint and isindependent from one stage to another. Philpott and Guan [17] have studiedthe convergence of the method. Shapiro [29] has recently analysed the statisticalproperties and rate of convergence of the method.

In our context, the random variable is in the criteria instead of the right handside of the state dynamic constraint. This dierence makes the Bellman functionQ(t, xt, ξt) only partial convex, which means that it is only convex with respect toxt and is non-convex with respect to ξt. Therefore, it is impossible to approximateglobally Q(t, xt, ξt) by lower bound linear functions, and it is the key argument todiscretize the random process. We have to build the optimality cuts with respectto xt to approximate Q(t, ·, ξt) from below. After discretization, we are able tostore the optimality cuts at the quantized points and approximate the optimalitycuts on non-quantized points by technique presented in previous section.

The algorithm then follows from SDDP procedure. In the forward pass, we simulatenumerically Mf scenarios of random process (ξm

t ),m = 1, . . . ,Mf , and compute

8 Mon 5th Dec, 2011 10:04

1.2. Summary of results

the optimal control (umt ) associated to each random process. Then, the forward

value v is the statistical estimate of the value of these Monte Carlo samples, which isstatistically an upper bound of the optimal value. In backward pass, we update theoptimality cuts at each quantized point. The backward value v is the optimal valueassociate to rst stage, which is a lower bound of optimal value of the discretizedproblem. The algorithm stops when the upper bound value v and the lower boundv are close enough. Some discussion on stopping condition of SDDP algorithm isalso provided in Shapiro [29].

• We prove the convergence result of the algorithm. The principle proof followsthe niteness of the optimality cuts and feasibility cuts on discretized problem,which is the main argument of nite convergence of L-Shape method. Then, usingstatistical argument, we obtain the convergence when the forward simulation followsdiscrete distribution. The convergence on the case where the forward samplinguses continuous distribution follows the theorem in Römisch [18] on the stabilityof optimal value with respect to small perturbation on the process distribution.

• We provide two numerical tests: one is the normalized swing option, another is theLNG portfolio optimization. Both tests shows good performance of the algorithm,even for large size problem.

1.2.2 Apply SDDP in risk aversion optimization

The multi stage stochastic programs (1.10) we have considered until now is under riskneutral approach. We also deal with the risk averse approach by using semi-continuousinferior convex risk measure as criterion. We focus on conditional value at risk CVaR inour work. Such risk measure is widely used in the practice of risk management.

inf (1− β)E [v] + βCVaRα(v)

subject to v =T−1∑t=0

ct(ξt)ut + g(ξT , xT ),

ut ∈ Ut, Ft −measurable,

xt+1 = xt + Atut,

x0 = 0, xT ∈ XT ,

(1.22)

where 0 ≤ β ≤ 1 and 0 < α 1 are some coecient.Such risk averse optimization problem has been largely studied in literature, in theory:

see Ruszczy«ski and Shapiro [26], Dentcheva and Ruszczy«ski [15, 16], and in numericalmethod: see Ruszczy«ski [25], Ruszczy«ski and Shapiro [27], Philpott and de Matos [24].

The techniques of changes in probability measure (importance sampling) is very pop-ular in variance reduction. Recently, Bardou et al. [7, 6] use Robbins-Monro algorithmto compute the best drift in importance sampling and apply it in CVaR hedging problem.

Mon 5th Dec, 2011 10:04 9

1. Introduction

• Following Rockafellar and Uryasev [34], CVaR can be reformulated as the optimalvalue of a stochastic optimization problem. Combined with (1.22), we obtain asimilar problem as (1.11) but with two additional one dimensional state variables.

inf βz + E[(1− β)v +

β

α(v − z)+

]subject to v = yT + g(ξT , xT ),

ut ∈ Ut, Ft −measurable,

xt+1 = xt + Atut,

x0 = 0, xT ∈ XT ,

yt+1 = yt + ct(ξt)ut, y0 = 0.

(1.23)

Therefore, our SDDP algorithm presented in previous section is still adapted tothis problem.

• In order to increase the extreme scenarios in Monte Carlo simulation, we apply thechanges in probability measure techniques into stochastic programming algorithm,relying on Girsanov theorem and theorem of changes in probability measure.

inf βz + EQφ

[(1− β)v +

1

ZφT

β

α(v − z)+

]subject to v = yT +

1

ZφT

g(ξT , xT ),

ut ∈ Ut, Ft −measurable,

xt+1 = xt + Atut,

x0 = 0, xT ∈ XT ,

yt+1 = yt +1

Zφt

ct(ξt)ut, y0 = 0,

(1.24)

where dQφ = ZφT dP and

Zφt = exp

(t∑

s=1

φs(Ws −Ws−1)−12

t∑s=0

φ2s

)(1.25)

The objective of the change is to reduce the variance of the Monte Carlo simulation.Because of some feature of our algorithm, modication of drift (φt) during SDDPprocedure increases considerably the complexity of the algorithm. Therefore, wehave to choose (φt) a priori, which is dicult. We have discuss several methods tocompute (φt).

• Numerical tests have been done on swing option pricing problems, and it shows goodperformance of this algorithm without using technique of changes in probability

10 Mon 5th Dec, 2011 10:04

1.2. Summary of results

measure. Compared with risk neutral results, the results of risk averse problemgive exactly what we expect: reduction of the extreme loss scenarios such that theoption value of most scenarios do not exceed the VaRα.

However numerical test using technique of changes in probability measure showsthat they fail to reduce the variance of Monte Carlo simulation. We only arrive toincrease the scenario number in the tail of the loss distribution.

1.2.3 Sensitivity analysis

The study on sensitivity analysis responds to the need of trading perspective. The valuesof sensitivities (called the greeks) allow traders to replicate the price variations originatingfrom the contracts they have to manage on a regular basis.

We rst assume that the random process, taking part in the modeling of futurecommodity spot prices, follows the celebrated Black model. The price model takes asparameters the market volatilities σ, correlation ρ as well as the forward curve F0.

ξit = (F t

0)i exp

(t−1∑s=0

σiW is −

12(σi)2T

)i = 1, . . . , d

where W it is a standard normal distribution N (0, 1), with correlation corr(W i

s ,Wjs ) = ρij .

We consider the continuous relaxation problem (1.11) where there is no nonlinearterm g(xT , ξT ) in criteria:

inf E

[T−1∑t=0

ct(ξt) · ut

].

Let us denote by v(F0, σ) the optimal value and by U(F0, σ) the set of optimal solutions.Then, the sensitivity analysis we are interested in is the derivative of the optimal valuefunction with respect to F0 and σ

DF0,σv(F0, σ). (1.26)

The sensitivity analysis under stochastic optimization (or stochastic control) has notbeen much studied. Most of the articles in the literature focus on the procedure on howto solve the problem and hence how to get the contract's price. In our study, we propose amethod to estimate sensitivities based on Danskin's theorem, which is well known amongoptimization practitioners.

• We prove that v(F0, σ) is Fréchet dierentiable at almost every point (F0, σ). Andon the point where v(F0, σ) is Fréchet dierentiable, we have

v′(F0, σ; dF0, 0) =E

[T−1∑t=0

DF0ctdF0 · u∗t

]

v′(F0, σ; 0, dσ) =E

[T−1∑t=0

Dσctdσ · u∗t

] (1.27)

Mon 5th Dec, 2011 10:04 11

1. Introduction

where u∗ ∈ U(F0, σ) is one optimal solution.

• Since U(F0, σ) is not available and we can only obtain an approximated optimalsolution by our algorithm, we study the sensitivity values using optimal solution onthe discretized problem. We provide two versions of discretized problem to modelboth forward and backward pass in SDDP algorithm. We prove the convergenceresult of sensitivity values of both discretized problem to the sensitivity values ofthe continuous problem.

• During numerical tests, we study the same two examples as in the rst part: swingoption and LNG portfolio optimization. Since swing option is a low dimensionproblem, comparisons between results obtained using SDDP method and otherclassical methods are provided and give evidence of good accuracy of the estimateof marginal prices.

1.2.4 Heuristic method for stochastic integer program

Finally, we study the multi stage stochastic integer program. First, we introduce aheuristic method to obtain a sub-optimal integer solution with the help of optimalitycuts computed by continuous relaxation resolution.

Q(t, xt, ξt) = essinf ct(ξt) · ut + ϑ(t + 1, xt+1, ξt)subject to xs+1 = xs + Asus,

us ∈ Us ∩ Zn,Fs −measurable, s ≥ t,

xT ∈ XT ,

(optimality cut) ϑ(t + 1, xt+1, ξt) ≥ λxt xt + λ

0t , (λx

t , λ0t ) ∈ O(ξt).

(1.28)

This sub optimal strategy gives an upper bound estimate of optimal value of integerproblem. The lower bound is still given by the lower bound of the continuous relaxationproblem. Then, the objective is to reduce the gap between the upper bound and thelower bound.

The main tool we study is the integrality cutting plane technique. In literature, manyvarious integrality cuts techniques have been proposed to solve the integer problem, seeCornuejols [13] for a survey. However, even in the deterministic case, it is very hard toobtain an optimal solution only by using the integrality cuts technique. It has been shownthat combination of integrality cuts technique with branch and bound technique (calledbranch and cut method or cut and branch method) will greatly improve the convergencespeed. However, applying branch and bound technique in the stochastic case will increaseconsiderably the complexity of the algorithm and make problem numerically intractable.Some studies using branch and bound style algorithm in stochastic programming arelimited to two stages problem, see [28, 2] for survey. As to multi stage case, there arejust a few publications. Even heuristic methods are rarely proposed and they are limitedto some particular problem [21].

12 Mon 5th Dec, 2011 10:04

1.3. Organization of thesis dissertation

In the thesis, we discuss the diculties of multi stage integer problem by a verysimple example, and show that dual programming type algorithm is not well adaptedto integer problem because of non-convexity and discontinuity of the Bellman valuefunction Q(t, xt, ξt) with respect to xt. Thus, our SDDP algorithm is not adapted tomultistage integer programming. Finally, by analysing in detail where the continuousoptimal solution ut is located, we propose a slight improvement, which will reduce thegap in some cases.

1.3 Organization of thesis dissertation

This manuscript is composed of three parts.

• Part I (chapter 2, 3) focuses on numerical method to solve the continuous relax-ation of problem (1.10). Chapter 2 is related to a paper entitled Energy contracts

management by stochastic programming techniques, written in collaboration withF. Bonnans and T. Christel, published as INRIA research report RR-7289 [11] andwill appear in special issue of Annal of operation research on stochastic program-

ming. We add a chapter 3 devoted to the analysis of the same type algorithm torisk aversion optimization problem.

• Part II (chapter 4) deals with the sensitivity analysis with respect to parameters inspot price model, based on Danskin's theorem. It is based on paper entitled Sensi-

tivity analysis of energy contracts by stochastic programming techniques written incollaboration with F.Bonnans and T.Christel, published as INRIA research report

RR-7574 [12] and will appear in the book Numerical Methods in Finance, editedby R. Carmona, P. Del Moral, P. Hu, N. Oudjane, in series of Springer Proceeding

in Mathematics.

• In part III (chapter 5), we study the multi-stage stochastic integer program.

• The appendix consists of two parts. First, we present some advancement of approx-imation method by quantization. Then, another chapter concentrates on variousintegrality cutting plane methods for integer solutions.

Bibliography

[1] Y. Achdou and O. Pironneau. Computational methods for option pricing, volume 30of Frontiers in Applied Mathematics. Society for Industrial and Applied Mathematics(SIAM), Philadelphia, PA, 2005.

[2] S. Ahmed. Two-stage stochastic integer programming: A brief introduction. InJames J. Cochran, Louis A. Cox, Pinar Keskinocak, Jerey P. Kharoufeh, andJ. Cole Smith, editors, Wiley Encyclopedia of Operations Research and Management

Science. John Wiley & Sons, Inc., 2010.

Mon 5th Dec, 2011 10:04 13

1. Introduction

[3] V. Bally and G. Pagès. Error analysis of the quantization algorithm for obstacleproblems. Stochastic Processes & Their Applications, 106(1):140, 2003.

[4] V. Bally and G. Pagès. A quantization algorithm for solving discrete time multi-dimensional optimal stopping problems. Bernoulli, 9(6):10031049, 2003.

[5] V. Bally, G. Pagès, and J. Printems. A stochastic quantization method for nonlinearproblems. Monte Carlo Methods and Appl., 7(1):2134, 2001.

[6] O. Bardou, N. Frikha, and G. Pagès. CVaR hedging using quantization basedstochastic approximation algorithm.

[7] O. Bardou, N. Frikha, and G. Pagès. Recursive computation of value-at-risk andconditional value-at-risk using mc and qmc. In Pierre L' Ecuyer and Art B. Owen,editors,Monte Carlo and Quasi-Monte Carlo Methods 2008, pages 193208. SpringerBerlin Heidelberg, 2009.

[8] J.F. Benders. Partitioning procedures for solving mixed-variables programmingproblems. Numer. Math., 4:238252, 1962.

[9] D.P. Bertsekas and J.N. Tsitsiklis. Neuro-Dynamic Programming. Athena Scientic,1996.

[10] J.R. Birge and F. Louveaux. Introduction to Stochastic Programming. Springer,New York, 2000.

[11] J.F. Bonnans, Z. Cen, and T. Christel. Energy contracts management by stochasticprogramming techniques. Inria research report 7289, INRIA, May, 2010. will appearin the special issue of Annual of operation research on stochastic programming.

[12] Z. Cen, J.F. Bonnans, and T. Christel. Sensitivity analysis of energy contractsmanagement problem by stochastic programming techniques. Rapport de rechercheRR-7574, INRIA, March 2011. to appear in Numerical Methods in Finance, seriesof Springer Proceeding in Mathematics, editor "R. Carmona, P. Del Moral, P. Hu,N. Oudjane, 2011.

[13] G. Cornuéjols. Valid inequalities for mixed integer linear programs. Mathematical

Programming B, 112:2008, 2006.

[14] G.B. Dantzig. Linear programming under uncertainty. Management Sci., 1:197206,1955.

[15] D. Dentcheva and A. Ruszczy«ski. Duality between coherent risk measures andstochastic dominance constraints in risk-averse optimization. Pac. J. Optim.,4(3):433446, 2008.

[16] D. Dentcheva and A. Ruszczy«ski. Robust stochastic dominance and its applicationto risk-averse optimization. Math. Program., 123(1, Ser. B):85100, 2010.

14 Mon 5th Dec, 2011 10:04

BIBLIOGRAPHY

[17] Z. Guan and A.B. Philpott. On the convergence of stochastic dual dynamic pro-gramming and related methods. Operation Research Letters, 36:450455, 2008.

[18] H. Heitsch, W. Römisch, and C. Strugarek. Stability of multistage stochastic pro-grams. SIAM J. on Optimization, 17:511525, 2006.

[19] P. Kall and S.W. Wallace. Stochastic Programming. John Wiley & Sons, Chichester,1994.

[20] M. Kaut and S.W. Wallace. Evaluation of scenario-generation methods for stochasticprogramming. Pacic Journal of Optimization, 3(2):257 271, 2007.

[21] A. Løkketangen and D.L. Woodru. Progressive hedging and tabu search appliedto mixed integer (0,1) multistage stochastic programming. Journal of Heuristics,2:111128, 1996. 10.1007/BF00247208.

[22] B. Marie. Modélisation et méthodes d'évaluation de contracts gaziers: approche par

contrôle stochastique. PhD thesis, Université Paris Diderot, 2011.

[23] M.V.F. Pereira and L.M.V.G. Pinto. Multi-stage stochastic optimization applied toenergy planning. Mathematical Programming, 52:359375, 1991.

[24] A.B. Philpott and V. de Matos. Dynamic sampling algo-rithms for multi-stage stochastic programs with risk aversion.http://www.epoc.org.nz/papers/PhilpottdeMatosRevision.pdf.

[25] A. Ruszczy«ski. Risk-averse dynamic programming for Markov decision processes.Math. Program., 125(2, Ser. B):235261, 2010.

[26] A. Ruszczy«ski and A. Shapiro. Conditional risk mappings. Math. Oper. Res.,31(3):544561, 2006.

[27] A. Ruszczy«ski and A. Shapiro. Optimization of convex risk functions. Math. Oper.

Res., 31(3):433452, 2006.

[28] S. Sen. Algorithms for stochastic mixed-integer programming models. In K. Aardal,G.L. Nemhauser, and R. Weismantel, editors, Handbook of Discrete Optimization,pages 515558. North-Holland Publishing Co., 2005.

[29] A. Shapiro. Analysis of stochastic dual dynamic programming method. European

Journal of Operational Research, 209:6372, 2011.

[30] A. Shapiro, D. Dentcheva, and A. Ruszczy«ski. Lectures on Stochastic Programming

: Modelling and Theory. SIAM, Philadelphia, 2009.

[31] R.M. Van Slyke and R. Wets. L-shaped linear programs with applications to optimalcontrol and stochastic programming. SIAM J. Appl. Math., 17:638663, 1969.

Mon 5th Dec, 2011 10:04 15

1. Introduction

[32] J.N. Tsitsiklis. An analysis of temporal-dierence learning with function approxi-mation. IEEE transactions on automatic control, 42(5):674690, 1997.

[33] J.N. Tsitsiklis and B. Van Roy. Optimal stopping of Markov processes : Hilbert spacetheory, approximation algorithms, and an application to pricing high-dimensionalnancial derivatives. IEEE transactions on automatic control, 44(10):18401851,1999.

[34] S. Uryasev and R.T. Rockafellar. Conditional value-at-risk: optimization approach.In Stochastic optimization: algorithms and applications (Gainesville, FL, 2000),volume 54 of Appl. Optim., pages 411435. Kluwer Acad. Publ., Dordrecht, 2001.

16 Mon 5th Dec, 2011 10:04