IN DEGREE PROJECT MATHEMATICS,SECOND CYCLE, 30 CREDITS

, STOCKHOLM SWEDEN 2016

Dynamic Programming Heuristics for the Optimization of Hydropower Planning

JORRIT PETERS

KTH ROYAL INSTITUTE OF TECHNOLOGYSCHOOL OF ENGINEERING SCIENCES

Dynamic Programming Heuristics for the Optimization of Hydropower Planning

J O R R I T P E T E R S

Master’s Thesis in Optimization and Systems Theory (30 ECTS credits) Master Programme in Applied and Computational Mathematics

(120 credits) Royal Institute of Technology year 2016

Supervisor at Powel: Ulf Brännlund Supervisor at KTH: Per Enqvist

Examiner: Per Enqvist TRITA-MAT-E 2016:30 ISRN-KTH/MAT/E--16/302--SE Royal Institute of Technology School of Engineering Sciences KTH SCI SE-100 44 Stockholm, Sweden URL: www.kth.se/sci

http://www.kth.se/sci

Abstract

In short-term planning of a hydropower system one optimizes thechoice of release of water and which generators to use. This problemis prone to Bellman’s curse of dimensionality, since the amount of statesincreases exponentially with the amount of hydropower stations in the sys-tem. In this thesis, two different dynamic programming heuristics to thisproblem are derived, described and compared with a heuristic developedby Powel AS. A difficulty that extends the problem, is the addition of apower reserve on each time period. A third dynamic programming heuris-tic is developed to deal with a constraint on the power reserve and theparallel with knapsack problem is described. The new dynamic program-ming approaches perform similar to the heuristic by Powel AS. The thirddynamic programming heuristic that deals with the reserve constraints,provides a better way of dealing with and fulfilling the constraint thancurrently in place by Powel AS.

1

Dynamiska programmeringsheuristiker för optimering avvattenkraftsystem

Abstract

För kortsiktig planering av vattenkraftsystem optimerar man mängdenvatten som släpps p̊a och valet av generatorer att använda. Det härproblemet drabbas av Bellman’s dimensionalitetens förbannelse d̊a an-talet tillst̊and ökar exponentiellt med antalet vattenkraftverk i systemet.I detta examensarbete härleds tv̊a dynamiska programmeringsheuristikersom jämförs med en heuristik utvecklad av Powel AS. En aspekt somförsv̊arar uppgiften är tillägget av en effektreserv för varje tidsperiod. Entredje dynamisk programmeringsheuristik utvecklas för att hantera re-striktioner p̊a effektreserven och parallellen till knapsack problemet beskrivs.De nya dynamisk programmeringstillvägag̊angsätten presterar likvärdigtmed Powel AS heuristik. Den tredje dynamiska programmeringsheuris-tiken som hanterar reservbegränsingar erbjuder en bättre lösning för atthantera och uppfylla bivillkoren i jämförelse med de som i nuläget användsav Powel AS.

2

Acknowledgements

From February to July 2016, I worked at Powel on my master thesis project formy master programme Applied Mathematics at the Royal Institute of Technol-ogy (KTH). During the research I had help of many people whom I would liketo thank.Firstly, I would like to thank my two supervisors, Per Enqvist at KTH and UlfBrännlund at Powel, for their very useful help during the thesis. Ulf, for ourdiscussion about the mathematics behind hydropower systems and finding newways to solve it. Per, for helping me keep the academic focus and supportingin the writing process of the thesis.Secondly, I would like to thank my colleagues at Powel for the enjoyable workingenvironment. I felt welcome at Powel and enjoyed talking with you and learningabout the company Powel.Thirdly, I would like to thank Per, Charlotte and Ulf for proofreading, Sarafor translating the abstract to Swedish and Timo for giving feedback on thepresentation.Last but not least, I would like to thank my family, friends and girlfriend fortheir support and help during my years as a student.

Stockholm, July 2016,Jorrit Peters

3

Contents

1 Introduction 6

2 Problem description 102.1 Mathematical model . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.1.1 Water value . . . . . . . . . . . . . . . . . . . . . . . . . . 132.2 Reserve . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142.3 Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

2.3.1 Stage I: Relaxed LP . . . . . . . . . . . . . . . . . . . . . 152.3.2 Stage II: Dynamic program . . . . . . . . . . . . . . . . . 152.3.3 Stage III: Refinement using LP . . . . . . . . . . . . . . . 16

2.4 Research Question . . . . . . . . . . . . . . . . . . . . . . . . . . 162.5 Test case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

3 Dynamic Programming problem 193.1 Initial approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

3.1.1 Time periods . . . . . . . . . . . . . . . . . . . . . . . . . 203.1.2 Discretization . . . . . . . . . . . . . . . . . . . . . . . . . 20

3.2 Simplification of the dynamic programming recursion . . . . . . . 213.2.1 Max inside sum sign . . . . . . . . . . . . . . . . . . . . . 223.2.2 Keep max outside sum sign . . . . . . . . . . . . . . . . . 22

4 Alpha 25

5 Heuristic Gamma 275.1 Correction for flow time . . . . . . . . . . . . . . . . . . . . . . . 275.2 Simplified example . . . . . . . . . . . . . . . . . . . . . . . . . . 285.3 Current form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315.4 Disproportional terms . . . . . . . . . . . . . . . . . . . . . . . . 325.5 General Form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335.6 Release set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

5.6.1 Current reservoir . . . . . . . . . . . . . . . . . . . . . . . 355.6.2 Downstream reservoirs . . . . . . . . . . . . . . . . . . . . 355.6.3 Most upstream reservoirs . . . . . . . . . . . . . . . . . . 36

5.7 Complexity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

6 Heuristic Delta 376.1 General form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

7 Epsilon 397.1 Reserve . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397.2 Dynamic programming equations . . . . . . . . . . . . . . . . . . 397.3 Parallel with Knapsack problem . . . . . . . . . . . . . . . . . . . 407.4 Incoming water assumption . . . . . . . . . . . . . . . . . . . . . 417.5 Complexity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

4

8 Results 438.1 Alpha, Gamma and Delta without reserve constraints . . . . . . 43

8.1.1 Sensitivity of discretization of release . . . . . . . . . . . . 438.1.2 Sensitivity of discretization of content level . . . . . . . . 458.1.3 Results on different datasets . . . . . . . . . . . . . . . . . 45

8.2 Alpha and Epsilon with reserve constraints . . . . . . . . . . . . 478.2.1 Sensitivity analysis . . . . . . . . . . . . . . . . . . . . . . 478.2.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

9 Conclusions 509.1 Datasets without reserve constraint . . . . . . . . . . . . . . . . . 509.2 Datasets with reserve constraint . . . . . . . . . . . . . . . . . . 51

10 Discussion 5210.1 3 stage Heuristics evaluation . . . . . . . . . . . . . . . . . . . . . 52

10.1.1 Stage I . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5210.1.2 Stage II . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5210.1.3 Stage III . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

A Datasets 56

B Additional results 57B.1 Alpha with long runtime . . . . . . . . . . . . . . . . . . . . . . . 57B.2 Deviation at the end . . . . . . . . . . . . . . . . . . . . . . . . . 58B.3 Random incoming data . . . . . . . . . . . . . . . . . . . . . . . 60

5

1 Introduction

Hydropower plants form an important part of the energy production in manyparts of the world. As can be seen in Figure 1, China produces the mosthydropower and Sweden is, despite its size, the 7th country in the world in hy-dropower production [7]. In Sweden, hydropower plants generate a very largepart of the total energy production [8] and in China, hydropower systems gen-erate 20 % of the total energy [13]. In Figure 2, one can see how hydropowerenergy is an important part of the energy production in Sweden. One of thefields of research around hydropower, is the planning of the water release in asystem of several hydropower stations. Within the subfield of planning of hy-dropower systems, a substantial part of the research comes from China. Theplanning of hydropower can be divided into long-term planning with a timehorizon of several years, seasonal-planning with a time horizon of a year, andshort-term planning with a time horizon from a few days to a few weeks. In thisthesis we investigate and describe algorithms to optimize the short-term plan-ning of hydropower plants. Optimization of large-scale hydropower systems hasbeen a subject which is often researched. The rise in computer power givesthe possibility to, more easily, optimize entire system at once. In this thesisthe profit generated by a hydropower system, while having time-varying energyprices, is optimized by determining the optimal policy. A policy consists of thereleases and which generators should be turned on for all stations for all timeperiods. A more detailed problem description is found in Section 2. A largehydropower system is often called a multi-reservoir system in the literature sincesuch a system contains several reservoirs where water can be stored and used.

Figure 1: Top hydropower production countries according to World EnergyCouncil [7]. Mtoe is an energy measurement meaning million tonne of oil equiv-alent

6

Figure 2: Energy production in Sweden over the last 65 years [8]

Variations of this problem have been solved in the literature and below onefinds an overview of the past literature on this subject. The type of problem issuited for a dynamic programming approach, an algorithm developed by Bell-man [4]. Dynamic programming is a type of problem solving with the followingcharacteristics [12]:

• The problem can be divided into stages.

• Each stage as a number of states.

• Decisions transform the current state to a state in the next stage.

• It finds the optimal decisions to the overall problem for all states at allstages.

• The optimal decision is independent of the past decisions and states, de-pends only on the current state.

• The procedure begins by finding optimal decision in last stage and workbackwards.

• Recursive relationship that identifies the optimal decision for a stage, giventhe optimal decision for the next stage is available.

Dividing the problem in multiple stages, decreases the complexity of the prob-lem. Yet if the number of states is very large, the problem is still very complexand requires a lot of computation power. This is the main disadvantage of dy-namic programming and is called Bellman’s ”Course of Dimensionality” andforms the largest challenge in this thesis. The amount of calculations increasesexponentially with the amount of reservoirs. Heidari (1971) [11] describes how

7

to apply discrete differential dynamic programming to the multi-reservoir prob-lem. Here one defines an initial feasible solution and decreases the search areato an area around this initial solution. Now the size of the problem is decreasedand can be solved. Around the new solution is again a search area defined andthe process is repeated until convergence is reached. This fails to work when theamount of reservoir is large since then the ”Curse of Dimensionality” comes intoplay again. Archibald (2006) develops an heuristics for multi-reservoir systems,which decomposes the systems such that each subgroup is small enough to nothave the ”Curse of Dimensionality”. The heuristic does not provide an optimalsolution, yet a good approximation.Ansarey et al (2014) [2] reports that ”solving multi-input multi-state energymanagement problems by dynamic programming is theoretically possible, butpractically very difficult due to the high computational cost”. Perez (2009) [16]mentions that adding an additional state largely increases the computationalcost. However, in a problem where we have to determine the release and thegenerators, we are bound to a multi-input multi-state problem. Musardo [6]reports several hours of computing time to solve an multi-state optimal controlproblem by dynamic programming. The application of the solution demands itto be solvable within several minutes. Therefore, heuristics have to be found todecrease the computation time while hopefully keeping an acceptable solutionnear optimal. Li et al (2013) [14] describes a method of optimizing large-scalehydropower systems using dynamic programming and two techniques to de-crease the curse of dimensionality. Firstly, they use discrete differential dynamicprogramming, as stated by Heidari [11]. Secondly, they decrease the memoryneeded by dynamically allocating and release memory using C++. As can bededuced from the literature, it is hard to solve a large scale dynamic program-ming problem. In this thesis several heuristics are described, which tackle thisproblem and decrease the curse of dimensionality.In all the papers above the goal was to find the optimal release from a reser-voir. However, in this thesis one also needs to determine for each station foreach time period which generators to use to let the water flow through. Thisincreases the amount of states polynomially. Kim (2001) [15] solves the short-term multiperiod optimal planning problem of power systems which generatespower by using steam. Here one not only has to determine the steam flow, butalso which boilers to use. Hence, similar to the problem of which generators touse. A 3-stage heuristics using both general optimization techniques and dy-namic programming is proposed to solve the problem. This 3-stage heuristics isvery similar to the 3-stage heuristics currently used at Powel AS. Comparisonand evaluation of the two heuristics is described in Section 10.1. Kim (2001)first generates alternatives, afterwards solves for the boilers using dynamic pro-gramming and then refines the solution fixing the boiler setup. This comparisonis placed in the discussion since it requires a comprehensive understanding ofthe model as described in this thesis.In this thesis a problem is solved of a large-scale dynamic programming problemof a hydropower system by developing an heuristic to overcome the curse of di-mensionality. The combination of a large-scale dynamic programming problem,where also the turning on/off of generators is considered, has not been describedin the literature before.When optimizing a multi-reservoir system with a finite time-horizon, one needsto make sure there is water left in the reservoirs at the last time unit. This is

8

important since after the last time unit the multi-reservoir system still has tobe used to generate power. A system with very low water levels is not desirable.In order to solve this problem, a value has to be given to the water. A wa-tervalue needed for a short-term model can be generated by long-term models.Cervellera (2006) [5] and Halliburton (2004) [10] describe how to use stochasticdynamic programming to optimize a long-term model and thus generating avalue of water.

9

2 Problem description

For a given river with hydroplants, we need to maximize the revenue by de-termining the optimal release, spill, and generator combination for each timeperiod, for each hydroplant. This has to be done while fulfilling multiple con-straints such as water level in reservoirs. This can be formulated as a largeMixed Integer Problem (MIP) as done in Section 2.1.We can adjust the release, spill, and generator combination. The release is howmuch water is released from a reservoir and goes through a generator to producepower. Spill is water that is spilled from the reservoir without generating power.Water released and water spilled can take a different path and even end up ina different reservoir. The last variable we can adjust is the generator combina-tion. A station has a number of generators and we need to decide when to turnon and when to turn off each generators. In this problem, each combinationof on and off generators is called a generator combination. Hence, if a stationhas 3 generators, it has 8 generator combinations since 23 = 8. Note here thatthe generators are often not equivalent so we cannot treat certain combinationas similar situation. Usually, it is only possible to release no water if all thegenerators are turned off. The highest release is possible when all releases areturned on. Hence, it is important to have the correct generator combination,yet turning on and off involves costs.We want to maximize the revenue of the produced power. Since the energy pricefluctuates over time, this is not equal to maximizing the power production. Theenergy price largely fluctuates between day and night and also between days.The cost incurred are turn on and turn off costs of the generators, and possiblepenalty costs for having the reservoir content near the maximum/minimum.Many constraints are imposed on this problem. Most constraints are maximumand minimum bounds on the variables such as; content level, flow, increase inflow, decrease in flow, spill, increase in spill, decrease in spill.In order to make it easier to understand, below one can find an example ofhow a part of the river can look in Figure 3 and one can find a close up of astation in Figure 4. In Figure 3, one can see the spill and release can go thesame path but not necessarily. The release and spill of reservoir 1 both flow toreservoir 2, while the release of reservoir 0 flows to reservoir 2, yet the spill ofreservoir 0 flows to reservoir 3. The spill is e.g., flowing using a small side river.Furthermore, two reservoir can release into the same reservoir, hence a fork con-struction. An example of this is that both reservoir 1 and reservoir 2 can flowinto reservoir 2. In Figure 4, one can see three generators, hence 8 generatorcombinations. It also shows that the power output is dependent on the planthead; the difference in level of the top reservoir and the lower reservoir. Oftenthe power generation is a function of both the reservoir content of the stationand the reservoir content of the station below. However, if the next reservoir isnot directly after the station, the content level of the reservoir below does notaffect the power generation.

10

Figure 3: Part of a river. Source: Powel AS[3]

Figure 4: Model of a hydropower station. Source: Powel AS[3]

11

2.1 Mathematical model

Here we formulate the MIP of the problem. Firstly we state the variables andfunction:

gi,t = combination of turbines at station i at time,

ri,t = water release/flow at station i at time t,

si,t = water spill at station i at time t,

G = Contains all gi,t,

R = Contains all ri,t,

S = Contains all si,t,

pt = Energy price at time t,

ci,t = Reservoir level at station i at time t,

ct := c0:n,t all content levels at time t,

W (c0, cT ) = Loss in water value from c0 till cT ,

t(ci,t, ri,t) = tactical soft cost for content level and release,

inci,t = Incoming water at station i at time t, estimated from relaxed LP,

s(gi,t−1, gi,t) = Transition costs from one generator combination to another generator combination,

P (ri,t, gi,t, ci,t, ci+1,t) = Power generated, dependent on release, generator combination,

content level and content level of reservoir below.

After this we state objective function which is

maxR,S,G

∑i,t

pt · P (ri,t, gi,t, ci,t, ci+1,t)−∑i,t

s(gi,t−1, gi,t)−∑i,t

t(ci,t, ri,t)−W (c0, cT ).

(1)

In the first term, we loop over all station and time periods and sum the powergenerated times the price at the time period. In the second term, we deductthe transition costs of the generator combinations. In the third term, we deductthe tactical soft costs for the water level and release. In the fourth term, wecalculate and deduct the loss in water value between the first time period andthe last time period. We want to maximize the objective function while fulfillingthe constraint. The water balance equation is given by

ci,t = ci,t−1 + inci,t − ri,t − si,t ∀i, t. (2)

Hence, the new content is the old content plus the incoming water (both fromthe station above and from nature) minus the release and spill.The constraints for maximum and minimum limits per reservoir per time are:

ci,tMIN ≤ ci,t ≤ ci,tMAX , ∀i, t, (3)ri,tMIN ≤ ri,t ≤ ri,tMAX , ∀i, t, (4)si,tMIN ≤ si,t ≤ si,tMAX , ∀i, t, (5)pi,tMIN ≤ pi,t ≤ pi,tMAX , ∀i, t. (6)

12

The constraint for maximum and minimum limits per time is

sptMIN ≤∑i

pi,t ≤ sptMAX , ∀t. (7)

The constraints for maximum change per reservoir per time are:

ci,t − ci,t−1 ≤ ∆ci,t,MAXUP , ∀i, t, (8)ci,t−1 − ci,t ≤ ∆ci,t,MAXDOWN , ∀i, t, (9)ri,t − ri,t−1 ≤ ∆ri,t,MAXUP , ∀i, t, (10)ri,t−1 − ri,t ≤ ∆ri,t,MAXDOWN , ∀i, t. (11)

The constraints for maximum change per time are:∑i

pi,t −∑i

pi,t−1 ≤ ∆sptMAXUP , ∀t, (12)∑i

pi,t−1 −∑i

pi,t ≤ ∆sptMAXDOWN , ∀t. (13)

and as last we have

ci,t ≥ 0, ∀i, t, (14)ri,t ≥ 0, ∀i, t, (15)gi,t ∈ G(i), ∀i, t. (16)

where G(i) contains all the generator combinations for station i.

2.1.1 Water value

In the above description W (c0, cT ) is described as the loss in water value fromc0 (initial content levels) to cT (content levels at end of time horizon). If thisterm is not included in the optimization problem, it is be optimal to use all thewater in the river for power production. That leads to a river with very lowwater levels at the end of the time horizon, but afterwards the river also needsto be used to generate power. This is solved by giving the remaining water inthe river a value, so called watervalue. The water value estimates the futureincome, the water in the river could generate. The water value is a function ofall the reservoir levels. The water value is the marginal change in the expectedfuture revenue at a change in water supply. The exact value depends on watersupply, future inflows, power prices, efficiencies, water rights and much more.It is the derivative of the expected future revenue function, as can be seen inFigure 5. It shows a watervalue function of one station and the relation w.r.t.the expected future income function. This function is an approximation sincethe watervalue of a reservoir is also depend on how much water is available inall other reservoirs in the river. A better approximation uses the content levelof the reservoir itself and the content level of the reservoir directly downstream.If the content of the reservoir below is almost empty, the water is scarcer thanwhen the content of the reservoir below is almost full. When water is scarcer,it implies that the marginal increase in future expected income of additionalwater in the content is higher. In Figure 6, one can see an example of the watervalue function using two content levels as input. In the problem setting here

13

Figure 5: Water value of one station. Source: Powel AS[3]

Figure 6: Expected future income of two stations. Source: Powel AS[3]

we use the water value as input in the model and do not have to determine itself. A long-term stochastic model outputs the watervalue at the time neededand this is used to calculate W (c0, cT ); the loss in water value from c0 till cT .

2.2 Reserve

Reserves are traded on the reserve market. When one is obligated to havereserve, it means that the power production must be able to change for a certaintime period in order to ensure stability of the power net. If, for example,the frequency on the power goes down, more power should be inserted in thenet. This happens by activating a reserve. A power company can commit tosupply a certain amount of reserve for a certain time period and if necessarythe company has to supply the power. Note that these commitment are pertime period and not per station. Hence, the commitment can be fulfilled byany combination of hydropower stations. There are three main types of reserve:primary, secondary, and tertiary. Furthermore, one can have reserve upwards

14

(producing more power) and reserve downwards (producing less power). Thesereserve can be seen as constraints in the model. However these constraintsare not per station, they are only per time period. A commitment to a reserveinfluences the optimal planning and should be taken into account when designingand choosing the heuristic.In this thesis, the only constraint added is a constraint that takes care of theprimary reserves upwards. Hence the possibility of having to produce additionalpower in a time period. We define rsti as the reserve contribution upwards bystation i and time t and rstup as the needed reserves for time period t. Now weadd ∑

i

rsti ≥ rstup, ∀t, (17)

to our constraints. The amount of a reserve a station can supply at a time,is a function of the generator combination, the current power production, andthe maximum power production. If it is possible to produce more power (hencethe necessary generators are turned on), before reaching the maximum powerproduction we have a reserve.

2.3 Algorithm

The amount of variables is simply too large to solve the MIP, as stated in Section2.1, in reasonable time. For a river with 20 hydroplants and if on average eachhydroplant has 4 different generator combinations variables, a release variableand a spill variable for a two-weeks schedule with timeperiod of one hour (hence336 timeperiods) the amount of variables is 40320. However, 26880 variables areinteger variables considering the generator combinations. In order to solve theproblem within a reasonable amount of time a 3 stage heuristic is developed.

2.3.1 Stage I: Relaxed LP

The reason why it takes too long to solve the MIP, is partly due to the integervariables of the generator combinations. Relaxing this constraint makes evenlarge river systems solvable in less than a minute by IBM’s optimizer CPLEX.In the example stated above, the 26880 integer variables of the generator com-binations become continuous variables. This makes the problem a lot easier tosolve. In this stage one relaxes the integer requirement on the generator com-binations and one gets a release per station per time period. Note that sincewe have relaxed in the generator combinations, we do not take into account thestart and stop costs in this stage. The release is optimal, if one could relax theinteger requirements in reality. However, a generator can not be on for a quarterand off for three-quarters. The release determined is seen as a guideline of thearea where the optimal release should be in.

2.3.2 Stage II: Dynamic program

This stage uses the release determined in stage I as a guideline of the searchneighborhood of the optimal release. The releases are discretized around therelease found in stage I. The black line in Figure 7 is the release acquired fromstage I and the dotted blue lines contain the area where the release is allowed to

15

fluctuate in stage II. This deviation from the release found in stage I is insertedin the model by the user. Note how the deviation is constant for all timeperiods except the last. The red line is the minimum and maximum level of thereservoir and the yellow line represents the content levels for which penalty costsare applied when crossed. The problem is solved by dynamic programming andthe optimal generator combinations are stored. As state variable, we have thecontent level of each reservoir and the generator combination. At each reservoirand each period we need to decide on a release and a new generator combination.In Section 3 the dynamic programming approach is explained in detail. Thisstage is very critical to the success of the algorithm and this is where the focusof this thesis lies.

Figure 7: Reservoir level in stage II. Source: Powel AS[3]

2.3.3 Stage III: Refinement using LP

The optimal generator combinations in the MIP in this stage are fixed to thevalues found in stage II. Now it takes a short time to solve the problem andfind the corresponding releases to this pattern of turbine combinations by usingCPLEX.

2.4 Research Question

What is the best heuristic to solve Stage II?The current method by Powel AS, named Alpha, is compared with two methodsderived and programmed, named Gamma and Delta. Furthermore, the additionof a reserve constraint is researched and a heuristic named Epsilon is derivedand compared to the current Alpha. In order to determine which heuristic isbest, we compare the optimality from each heuristic on several test cases andperform a sensitivity analysis on the parameters inputted by the user. Theheuristics are compared and evaluated on the following three criteria:

16

• Optimality: For each dataset CPLEX solves the problem completely inorder to have an optimal value. For each solution given by the heuristicsthe gap from the optimal value found by only using CPLEX is stated.

• Robustness: Here we measure if the heuristic gives similar performancesfor different datasets.

• Speed.

2.5 Test case

The heuristics is tested on the data provided by Vattenfall of the Luleälven.Luleälven is the most important river in hydropower energy in Sweden[1]. Italso includes two forks, where two reservoirs flow into one reservoir. This forkis an extra difficulty for the heuristics. In Figure 8, one can find a schematicoverview of the river. Each red square represents a hydropower station andeach blue rectangle represents a reservoir. As input data a dataset with normalvalues is provided by Powel AS. The variables in this dataset are changed inorder to test several datasets on each heuristic. The following adjustments aremade to the datasets:

• Low, medium and high initial water levels in order to simulate the differentseasons.

• Low, medium and high natural inflow in order to simulate drought andheavy rainfall.

• Low,medium and high energy prices.

• Constant natural inflow and fluctuating natural inflow.

17

Figure 8: Schematic overview of Luleälven

18

3 Dynamic Programming problem

3.1 Initial approach

The dynamic programming approach of Stage II is here introduced. The dy-namic programming solves the entire problem, yet only the generator combi-nations are taken and used in stage III. Intuitively, the dynamic programmingrecursion goes backwards in time to calculate the value of each state and thenforwards in time to choose the optimal path. Hence, stepping one time unit eachrecursion. The state needs to define the entire system at a given time periodand the decision variable a step to the next time period. This leads toState variables:

• The (discretized) content in each reservoir,

• generator combination in each reservoir at start of time period.

Decision variable:

• The (discretized) release in each reservoir,

• new generator combination in each reservoir.

Figure 9: Visualization of the initial approach DP

In Figure 9, one can see this initial approach visualized. On the vertical axisthe reservoirs are shown, hence here are four reservoirs. On the horizontal axisthe time axis is displayed, hence here are four time periods. The most left top

19

circle symbolizes the first reservoir in the first time period. The blue rectangleseach represent a state, they contain for a time period all the reservoirs. Theblack arrows represent the step from one state to another, hence symbolizesthe decision. In each combination of generators and content in each reservoir,we need to decide the optimal release and new generator combination. Thecorresponding dynamic programming recursion is:

Vt(G,C) = maxḠ

maxR

f(G, Ḡ, C,Ct+1(C,R)) + Vt+1(Ḡ, Ct+1(C,R)), (18)

VT (G,C) = VG,C , (19)

where G are the turbine combinations of all station, and C the discretizedcontent levels of all stations. The function f contains all the costs and profits,such as power revenue and turn on/off costs, related to being in the given stateand take the given decision. Vt+1 is the value function of the new state reachedafter the decision is taken. The problem is that the amount of states andamount of decisions is too large to solve this DP in reasonable time. The testcase has 18 reservoirs. When we discretize the content of each reservoir into 10unique contents and assume each reservoir has 3 generators, hence 8 generatorcombinations, then we have (10 · 8)18 = 1.8014399 · 1034 states. If we discretizethe release in 8 unique releases, we have (8 · 8)18 = 3.2451855 · 1032 differentdecision to check each step in recursion. These numbers are too high to providea solution within reasonable time. This is largely due to the fact that there are18 reservoirs, as the amount of states increases exponentially with the amount ofreservoirs. In this thesis we explain several heuristics, which decide the optimalrelease and generator combination for each reservoir separately instead of all atonce. Before deriving these heuristics, some important details are explained.

3.1.1 Time periods

In order to completely formulate and understand the problem, it is importantto understand which decision is taken at which time and which variables itaffects. In Figure 10, one can see how the decision of a time period influencesthe content of that same time period. The decision of time 1 is taken by lookingat the content at the beginning of time period 1, hence the content when timeperiod 0 ends. The water balance equation can be written as

Content 0 - Release(Decision 1) + Incoming 1 = Content 1. (20)

3.1.2 Discretization

In discrete dynamic programming the amount of states a system can be in, isfinite [12]. This is the reason the content level of a reservoir is discretized inseveral possible content levels instead of taken as a continuous variable. In casethe content is discretized in 8 different levels, then for each level the value (incombination with each generator combination) is calculated. Also the decisionvariable release is not taken as a continuous variable, yet also discretized. It ispossible in setting of dynamic programming to take this as a continuous vari-able. However, discretization of the release decreases the possibilities and hencedecreases the computation time needed. It is possible to use an optimization al-gorithm to determine the best release since the value function is convex. Now it

20

Figure 10: Zoom-in on the time periods

is possible that a certain release does not exactly lead to a state with one of thediscretized content levels. Instead, it can lead to a content level in between twodifferent discretized content levels. In this case the value is interpolated. Thismakes the solution a lot more flexible than when only the discretized contentlevels could be reached when deciding on a certain release.

3.2 Simplification of the dynamic programming recursion

The dynamic programming recursion as described in Section 3.1, has too manystates and possible decision to be used in practice. Therefore, a simplificationneeds to be found which keeps a good solution, yet decreases the needed com-putation power. The idea is that we define multiple dynamic programmingrecursions and combine them all into one larger system which represents theentire river. We start with the last time unit, where the water values are given.Then we go one time unit back and try to determine the value per stationinstead of for all stations at once. This makes the problem easier to handle.Before this can be done, the needed equations have to be derived. The basisare the dynamic programming recursion as stated in Equation 18 and 19. If weestimate the value function into separate parts for each station, we acquire

Vt(G,C) ≈n∑i

V ti (gi, ci, ci+1), (21)

21

where i represents a station number. Now we can transform Equation 18 and19 into

n∑i

V ti (gi, ci, ci+1) = maxḠ

maxR

n∑i

fi(gi, ḡi, ci(R), ci+1(R)) +

n∑i

V t+1i (ḡi, ct+1i (R), c

t+1i+1(R)),

(22)n∑i

V Ti (G,C) = known. (23)

From this point there are two ways to continue the problem, we can eithertake the max inside the sum sign or keep the max outside the sum sign andapproximate another way.

3.2.1 Max inside sum sign

Here we assume independence between stations, which is clearly not the case inreality, yet it makes the calculation a lot easier. When taking away the contentlevel of the reservoir below as a state variable (such as done in heuristic Alpha),we can transforms Equation 22 into

n∑i

V ti (gi, ci) =

n∑i

(maxḡi

maxri

fi(gi, ḡi, ci, ci+1(ci, ri)) + Vt+1i (ḡi, c

t+1i+1(ci, ri))

),

(24)n∑i

V iT (G,C) = known. (25)

Writing this as n different equation we get for all i = 1, 2, .., n:

V ti (gi, ci) = maxḡi

maxri

fi(gi, ḡi, ci, ci+1(ri)) + Vt+1i (ḡi, c

t+1i+1(ri)), (26)

V Ti (G,C) = known. (27)

Hence we only look at one station and the future value V it+1 also only consistsof the future income of that station. This method leads to model Alpha asexplained in Section 4.

3.2.2 Keep max outside sum sign

Here we form a recursion, which itself forms one step of the DP recursion, asstated in Equation 22. This derivation leads to method Gamma and Delta, asexplained in Section 5 and Section 6, respectively.

Vt(G,C) 'n∑i

V ti (gi, ci, ci−1) = maxḠ

maxR

n∑i

fi(gi, ḡi, ci, ci−1, ri) +

n∑i

V t+1i (ḡi, ct+1i , c

t+1i−1)

(28)

At the most bottom station, i.e. n = 1,

V t1 (g1, c1, c0) = maxḡ1

maxr1

f1(g1, ḡ1, c1, c0, r1) + Vt+11 (ḡ1, c

t+11 (r1), c

t+10 (r1)).

(29)

22

Note that c0 is the content level of the reservoir below. However, the mostbottom/downstream station does not have a reservoir below and hence this isthe water level of the sea and consequently has no influence on the planning.Hence, we state c0 = c

t0(r) = 0 for all time periods and all releases. One station

higher, i.e. n = 2 we have

V t1 (g1, c1, c0) + Vt2 (g2, c2, c1) = max

ḡ1maxr1

maxḡ2

maxr2

(f1(g1, ḡ1, c1, c0, r1) + V

t+11 (ḡ1, c

t+11 (r1, r2), c

t+10 (r1))

+ f2(g2, ḡ2, c2, c1, r2) + Vt+12 (ḡ2, c

t+12 (r2), c

t+11 (r2))

).

(30)

Here one should note that in V t+11 we state ct+11 (r1, r2). Here r2 is the incoming

water at the next time period in reservoir 1. If we continue in similar fashion,the problem becomes too large. Therefore we make a simplification by definingSti . For n = 1 we have

St1(g1, c1, c0) = Vt1 (g1, c1, c0) = max

ḡ1maxr1

f1(g1, ḡ1, c1, c0, r1) + Vt+11 (ḡ1, c

t+11 (r1), c

t+10 (r1)).

(31)

Hence,

V t1 (g1, c1, c0) = St1(g1, c1, c0). (32)

For n = 2, it becomes a bit harder. We want St2 to be the value taking intoaccount the current station and the station below, hence St2 ' V t2 + V t1 . Bycombining Equation 30 and Equation 31 we acquire

V t1 (g1, c1, c0) + Vt2 (g2, c2, c1) = max

ḡ2maxr2

(St1(g1, c1, c0)

+ f2(g2, ḡ2, c2, c1, r2) + Vt+12 (ḡ2, c

t+12 (r2), c

t+11 (r2))

).

(33)

Yet the problem is that St2(g2, c2, c1) does not know all the information insertedin V t1 (g1, c1, c0)+V

t2 (g2, c2, c1). The difference is that g1 and c0 is not known i.e.,

the starting generator combination of station 1 of time period t and the contenttwo station lower. For both variables a good approximation of the value hasto be found in order to get a good estimate of St2(g2, c2, c1). The effect of thevalue of c0 on a decision taken two station upstreams is disregarded and simplythe value acquired in stage I is inserted. However, g1 has a larger influence.The approximation is to take the best g1 possible. One should also note that inEquation 30, we state ct+11 (r1, r2) where r2 is the incoming water. Due to thesimplification we do not know the exact incoming water from above. The valueof the release of the station above is not passed as a parameter. We tackle thisproblem by taking the value of incoming acquired in stage I. However, we doknow c1 and by using this fact we state

St2(g2, c2, c1) = maxḡ2

maxr2

(maxg1

St1(g1, ct1(r2), c

t0) + f2(g2, ḡ2, c2, c1, r2) + V

t+12 (ḡ2, c

t+12 (r2), c

t+11 (r2))

).

(34)

23

Note that here we write maxg1 , the reason for this is that we do not knowin which generator combination station 1 starts in the current time period.We solve this by taking the best generator combination. This is a necessarysimplification since the state has to contain the generator combination. Whenwe expand the St1(g1, c1(r2), c0) term in the above Equation 34, using Equation31, we acquire

St2(g2, c2, c1) = maxḡ2

maxr2

(maxg1

St1(g1, ct1(r2), c

t0) + f2(g2, ḡ2, c2, c1, r2) + V

t+12 (ḡ2, c

t+12 (r2), c

t+11 (r2))

)= max

ḡ2maxr2

(maxg1

(maxḡ1

maxr1

f1(g1, ḡ1, c1(r2), c0, r1) + Vt+11 (ḡ1, c

t+11 (r2, r1), c

t+10 (r1))

)

+ f2(g2, ḡ2, c2, c1, r2) + Vt+12 (ḡ2, c

t+12 (r), c

t+11 (r2))

). (35)

Here one can see the similarities between St2(g2, c2, c1) (Equation 34) and Vt1 (g1, c1, c0)+

V t2 (g2, c2, c1) (Equation 30 ). Hence we just determined a ḡ2 and r2 for this state.To find V t2 (g2, c2, c1) that following equations are used:

St2(g2, c2, c1) ' V t2 (g2, c2, c1) + V t1 (g1, c1, c0), (36)V t2 (g2, c2, c1) ' St2(g2, c2, c1)− V t1 (g1, c1, c0) = St2(g2, c2, c1)−max

g1St1(g1, c

t+11 (r2), c0).

(37)

Note here no new optimization problem is solved, we use the values as calculatedabove and by subtraction we separate it into a part that only contains the futurevalue of the current reservoir. Hence for each state we find the optimal generatorcombination and release and we store Sti , the cumulative value function includingthe lower stations, and V ti the value function of the station itself.For station i and time t we have

Sti (gi, ci, ci−1) = maxḡi

maxri

(maxgi−1

Sti−1(gi−1, cti−1(ri), c

ti−2) + fi(gi, ḡi, ci, ci−1, ri)

+ V t+1i (ḡi, ct+1i (ri), c

t+1i−1(ri))

).

(38)Hence we just determined a ḡi and ri for the state gi, ci, ci−1. For this state wefind V ti (gi, ci, ci−1) by

Sti (gi, ci, ci−1) ' V ti (gi, ci, ci−1) + V ti−1(gi−1, ci−1, ci−2), (39)V ti (gi, ci, ci−1) ' Sti (gi, ci, ci−1)− V ti−1(gi−1, ci−1, ci−2)

= Sti (gi, ci, ci−1)−maxgi−1

Sti−1(gi−1, cti−1(ri), c

ti−2). (40)

Now we can also state the forward recursion. The only difference is we donot need to take maxgi−1 since we already passed the state and gi−1 is known.Hence,

Sti (gi, ci, ci−1) = maxḡi

maxri

(Sti−1(gi−1, ci−1(ri), ci−2) + fi(gi, ḡi, ci, ci−1, ri)

+ V t+1i (ḡi, ci(ri), ci−1(ri))

). (41)

24

4 Alpha

Method Alpha is developed by Powel. The other methods we benchmark withrespect to method Alpha. Method Alpha performs backwards and forwards oneach station separately, starting with the most upstream station. The back-wards recursion is done on a single station. The backwards recursion gives theoptimal value function for each time period for each state. After the backwardsrecursion we perform a forwards recursion on the station. This forward recur-sion starts at t = 0 and determines for each time period the optimal release andgenerator combination. This decision takes us to a new state in the next timeperiod. After calculating the dynamic programming recursion and deciding onthe release pattern and generator combination of the most upstream station, thestation below is calculated similarly. A large advantage of this method is thatthe incoming in the reservoirs is already known since the DP for all the abovestations is already calculated backwards and forwards. We already have deter-mined the optimal release and hence the incoming of the reservoir below whenperforming the backwards recursion on the reservoir below. Below we state thebackwards and forwards equation and sketch the forward and backwards order.As stated in Equation 26 and Equation 27, the recursion for station i and timet is written as

V ti (gi, ci) = maxḡi

maxri

fi(gi, ḡi, ci, ci+1(ri)) + Vt+1i (ḡi, c

t+1i+1(ci, ri)), (42)

V iT (G,C) = known. (43)

Note how it only depends on the value of the same reservoir one time periodahead and does not consider the reservoir below. In order to visualize the

Figure 11: Backwards pattern ofmethod Alpha

Figure 12: Forwards pattern ofmethod Alpha

heuristic Alpha in similar fashion as Figure 9, both the backwards and forwardsrecursion are displayed in Figure 11 and Figure 12. The backwards recursiongives the optimal value function and the forwards recursion gives the optimalreleases and generator combinations. Note that each node/circle represents acombination of a reservoir and a time. The left top node/circle represents themost upstream station at the first time period. Heuristic Alpha solves onedynamic program per station, hence the arrows connect only their own stationin different time periods. Firstly, a backwards recursion is performed, startingat the last time period and working backwards. This is shown in Figure 11 as thearrows point towards the left. Secondly, a forward recursion is performed wherethe optimal configuration of a station is determined, see Figure 12. For all the

25

other heuristics we show similar visualization in order to clarify the difference.The method has some disadvantages:

• Possibly infeasible due to not knowing the content level of reservoir below.An example is when a station release a lot of water and the reservoir belowoverflows,

• approximation of water value due to calculation per station separately. InSection 2.1.1 it is discussed why this is an approximation,

• approximation of power production to calculation per station separately.In Figure 4 one can see how the power production can be dependent onthe reservoir content below as well,

• constraints on all stations at once, such as reserve, are not possible toinclude.

The methods Gamma and Delta hope to resolve all or part of these disadvan-tages while also producing a good result within a reasonable time.

There is a possibility of enforcing a reserve constraint on the problem, asexplained in detail in Section 2.2. This constraint is currently dealt by giving apositive rewards in the function fi to a station where some reserve is possible.The hope is that all the reserves for a time period then add up to fulfilling theconstraint. Often this does not work and CPLEX is forced to solve the problemcompletely. This large disadvantage is overcome by heuristic Epsilon.

26

5 Heuristic Gamma

The Gamma Method is a continuation of the formulas derived in Section 3.2.2.As state variables for each station for each time period we have :

• The content level of the current reservoir,

• the content level of a reservoir below (downstream),

• the start combination of the generators.

As decision variables for each station for each time period we have:

• the release of the current reservoir,

• the new generator combination.

The backwards recursion equation for station i and time t is

Sti (gi, ci, ci−1) = maxḡi

maxri

(maxgi−1

Sti−1(gi−1, cti−1(ri), c

ti−2) + fi(gi, ḡi, ci, ci−1, ri)

+ V t+1i (ḡi, ct+1i (ri), c

t+1i−1(ri))

). (44)

Hence, we just determined a ḡi and ri for the state gi, ci, ci−1. For this state wefind V ti (gi, ci, ci−1) by

V ti (gi, ci, ci−1) = Sti (gi, ci, ci−1)−max

gi−1Sti−1(gi−1, c

t+1i−1(ri), c

t+1i−2)

= maxḡi

maxri

(fi(gi, ḡi, ci, ci−1, ri) + V

t+1i (ḡi, c

t+1i (ri), c

t+1i−1(ri)).

)(45)

Note here that we use the decision made in Equation 44 and do not maximizeagain when calculating the V term. Terms from Equation 44 are simply sub-tracted from each other. These two equations are used to find S and V . Thedifference between S and V is very important. The intuition behind V is that itonly looks at the current reservoir and the future profits from this reservoir. Scontains a V term of the same reservoir in the next time period and another Sterm of the reservoir one station downstream. Hence S looks at the effect of allthe reservoirs downstream and ahead of time. In the Figure 13 the red circlesrepresent the effect of the S term and the blue circles represent the effect of theV term. As we choose a higher release the value from the V term decreases dueto a lower content level, the value of the S term increases due to a higher contentlevel. One can also note that the S term seems to have influence on more circles(hence station and time combinations) than the V term. In the last part of thissection a term is added to bring S and V into proportion.In Equation 44 there are two parts that increase as we evaluated a higher release,namely f and S, and there is one part that decreases, namely V .

5.1 Correction for flow time

One should note that the above equations assume that the flow time betweenthe reservoirs is 0. However, in reality the flow times between reservoirs is notalways 0. If for example the flow time is 5 time periods then the Sti−1 term

27

Figure 13: Difference of V and S visualized

is constant for all combinations of gi, ri. This is because the release from thestation has no influence on the content of the station below in the same timeperiod, if it takes 5 time periods for the released water to reach the next reser-voir. It has an influence on the content of the station below 5 time periods later.This implies often the Sti−1 term is neglected in the optimization. This is a hugeloss of information and in order to tackle this problem, we change the t in Sti−1to St+di−1 where d is the flow to next reservoir. Hence S

t+di−1 is always affected by

the combinations of gi, ri.As state variables we now have the generator combination, content and contentof the reservoir below. In case the flow time is not 0, it is not logical to take thecontent of the reservoir below at the same time period. This content is of noinfluence to the decision to be made. Hence, we take the content of the reservoirbelow at the time period added by the time to flow.Hence in method Gamma we follow the water downstream, if the water takes 5time periods to reach the next station then we take that as our next part in therecursion.

5.2 Simplified example

A simplified situation is sketched and the recursion equations and updatingequations for a single node are stated. In Figure 14 some nodes have been givennames. In this example we assume we know all the nodes with subscript 1 andA2 and we want like to determine the backwards and forwards equation whichdetermine the release and generator combination of B2. The arrows representthe flow of water, hence A2 flows to B2 which flows to C2 and so on. Henceafter determining B2 the next node to determine in the forward recursion isC2. The diagonal arrows between reservoir A and B, and between reservoir Band C symbolize that it takes one time period for the water to flow to the nextreservoir. The vertical arrow from reservoir C to D symbolizes the flow timefrom C to D is 0.

28

Figure 14: Forward Recursion of Simplified Example visualized

The backwards recursion is given by

S1B(g1B , c

1B , C

1C) = max

g2B

maxr2B

f1B(g1B , g

2B , C

1B , C

1C , r

2B)

+ V 2B(g2B , C

2B , C

2C)

+ maxg1C

S1C(g1C , C

1C + ∆r

2B , C

1D). (46)

The reason why we state C1C + ∆r2B , instead of just C

1C , is derived further down

in this section in Equation 53. Note that the content below of cCB1 is C1C , hencewe follow the flow of water. f1B is a function which includes all the costs andrevenue given this state and decision. V 2B is the cost of being in the state withthe given state values, it gives a value to new content and generator combinationof reservoir B. S1C gives value to the state we end up in one reservoir downstreamin the correct time period. Since we do not know the generator combination ofC1 in the backwards recursion, we take the best possible generator combinationby maximizing over the possibilities. This works since it is mostly importanthow S1C changes when we loop over the decision that can be made. Hence, aslong as the difference between the different evaluations of S1C are of the correctsize, the optimal decision is still taken.In order to find V 1B we have the equation, as derived in Section 3.2.2,

V 1B(g1B , c

1B , C

1C) = S

1B(g

1B , c

1B , C

1C)−max

g1C

S1C(g1C , C

1C + ∆r

2B , C

1D). (47)

29

Below ˜inc and r̃ are used to indicate this value of inc and r are taken from theLP solution from stage I. We have the following update equation for the V -part:

C2B = C1B + ˜inc

2

B − r2B , (48)

C2C = C1C + inc

2C − r̃2C . (49)

Note that if we, for simplification, assume there is not additional incoming waterother than the release from a previous reservoir, we can state inc2C = r

2B . This

implies

C2C = C1C + r

2B − r̃2C . (50)

For the S-part we first define

∆r2B = r2B − r̃2B . (51)

Hence, ∆r2B is how much additional release we have with respect to the LPsolution. In the backwards case, we only know the LP solution, hence thevalues of S and V are calculated with the incoming from the LP considered.Since the incoming is not a state variable and the incoming in the DP is oftennot the same as the incoming of the LP, we add the additional incoming alreadyto the content level in the S1C term. Note the C

1C + ∆r

2B in Equation 47. Below

is explained in detail why this is allowed and preferred. When we only knowthe LP solution we can state

C2C = C1C +

˜inc2C − r̃2C= C1C + r̃

2B − r̃2C . (52)

However, we decide on an optimal r2B , a better estimate than r̃2B . Yet inside

S1C(g1C , C

1C + ∆r

2B , C

1D) this is not known and cannot be passed as an argument

since it is not a state variable. By replacing C1C by C1C + ∆r

2B we acquire

C2C = C1C + ∆r

2B +

˜inc2C − r̃2C= C1C + ∆r

2B + r̃

2B − r̃2C

= C1C + r2B − r̃2C . (53)

This has two advantages. Firstly C2C is better estimated hence S1C(g

1C , C

1C +

∆r2B , C1D) gives a better value function. Secondly, the decision of the release r

2B

has an influence on S1C . This is very important, otherwise S1C is constant for

each choice of g2B and r2B and thus irrelevant to the maximization problem.

In the forward recursion we already passed the node C1, hence the generatorcombination is decided and known. In the forward recursion the optimal releaseand generator combination are decided and is stated here as

S1B(g1B , c

1B , C

1C) = max

g2B

maxr2B

f1B(g1B , g

2B , C

1B , C

1C , r

2B)

+ V 2B(g2B , C

2B , C

2C)

+ S1C(g1C , C

1C + ∆r

2B , C

1D). (54)

30

Figure 15: Backwards Recursion visualized

5.3 Current form

After this simplified example in Section 5.2, we can state the general backwardsand forwards recursion for station i and time t. Furthermore, we have given das flow time till next reservoir and d2 as time to flow to the reservoir after that.Hence,

Sti (gti , c

ti, c

t+di−1) = max

ḡt+1i

maxrt+1i

f ti (gti , ḡ

t+1i , c

ti, c

t+di−1, r

t+1i )

+ V t+1i (ḡt+1i , c

t+1i , c

t+d+1i−1 )

+ maxgt+di−1

St+di−1 (gt+di−1 , c

t+di−1 + ∆r

t+1i , c

t+d2i−2 ). (55)

Hence, here are ḡt+1i and rt+1i determined for the state g

ti , c

ti, c

t+di−1. For this state

one finds V ti (gti , c

ti, c

t+di−1) by

V ti (gti , c

ti, c

t+di−1) = S

ti (g

ti , c

ti, c

t+di−1)−max

gt+di−1

St+di−1 (gt+di−1 , c

t+di−1 + ∆r

t+1i , c

t+d2i−2 ). (56)

We can do this since the flow always goes to the future. Hence St+di−1 is alwaysalready calculated if we implement the backwards recursion properly. In figure15 one can see in which order the backwards recursion is performed. We startwith the node on the most bottom station to the right and follow the blackarrows upwards while calculating the backwards. Note that this way the neededinformation for the above stated equation are always available. In Figure 16 theorder of the forwards recursion is given. The black arrows show which S term isevaluated and the yellow arrows show which V term is evaluated. Note here thenumbers on the nodes show the order of calculation and this follows the black

31

Figure 16: Foward Recursion visualized

arrows. Here the order follows the water downstream and forward in time. Thisway all the information needed for the forward equation is known.

5.4 Disproportional terms

As stated above in the beginning of Section 5 and illustrated in Figure 13, theterm S is disproportional to the term V . For the example as stated in Figure14, the S1B term is written down while each S term is expanded. We acquire

S1B(g1B , c

1B , C

1C) = max

g2B

maxr2B

f1B(g1B , g

2B , C

1B , C

1C , r

2B) + V

2B(g

2B , C

2B , C

2C)

+ S1C(g1C , C

1C + ∆r

2B , C

1D)

= maxg2B

maxr2B

f1B(g1B , g

2B , C

1B , C

1C , r

2B) + V

2B(g

2B , C

2B , C

2C)

+ maxg2C

maxr2C

f1C(g1C , g

2C , C

1C , C

1D, r

2C) + V

2C(g

2C , C

2C , C

2E)

+ S1D(g1D, C

1D + ∆r

2C , C

1E)

= maxg2B

maxr2B

f1B(g1B , g

2B , C

1B , C

1C , r

2B) + V

2B(g

2B , C

2B , C

2C)

+ maxg2C

maxr2C

f1C(g1C , g

2C , C

1C , C

1D, r

2C) + V

2C(g

2C , C

2C , C

2D)

+ maxg2D

maxr2D

f1D(g1D, g

2D, C

1D, C

1E , r

2D) + V

2D(g

2D, C

2D, C

2E). (57)

Hence, after expanding the S terms, we are left only with f and S. All the V ’s,except the first one, are expanded from the S term, hence a large influence. Ifone writes out the current equations, one notices that the S term is a lot largerthan the V term. The S term includes a (lower) V term and the next S termwhich itself includes the same again. Hence, a change of reservoir level in the Sterm actually means a change of reservoir level in many terms. Since the S termis a lot bigger than the V term, the heuristic does not make the optimal choice.

32

Figure 17: Multiplier visualizedwith two downstream stations

Figure 18: Multiplier visualizedwith three downstream stations

In this case, it is much more profitable, than it is in reality, to release waterdownstream (towards the S term), than let it stay in the reservoir (towards theV term). To solve this problem, we make the two terms proportional. This canbe done by adding a discount factor to the S term. By writing a discount factorof 12 for all, but the two most downstream stations, the effect is similar. Hence,

Sti (gti , c

ti, c

t+di−1) = max

ḡt+1i

maxrt+1i

f ti (gti , ḡ

t+1i , c

ti, c

t+di−1, r

t+1i )

+ V t+1i (ḡt+1i , c

t+1i , c

t+d+1i−1 )

+ maxgt+di−1

1

2St+di−1 (g

t+di−1 , c

t+di−1 + ∆r

t+1i , c

t+d2i−2 ). (58)

The effect can be easily seen in Figure 17 and Figure 18. Before each row ofthe S term a multiplier is placed. One can see that the multipliers always addup to one and hence we have that the effect is proportional. If this multiplieris not added in the formula, the following happens when the algorithm is run.The release of water from a station has a very large positive effect on the S termand a small negative effect on the V term. Since this effect is disproportional,it causes always a larger release of water is chosen than optimal.We use 12 as multiplier since it adds to one and is easy to implement in therecursion. One could also use a different multipliers, such as 1n , where n isthe number of reservoirs downstream. However, such a system of multipliers isharder to implement in the recursion.

5.5 General Form

As one can see in Equation 58, the term maxḡt+1idoes not influence St+di−1 and

maxgt+di−1does not influence V t+1i . Hence one can reformulate Equation 58 to

Equation 59 and decrease the computational power needed for the three maxi-mize operators. The change of nested loops of maxḡt+1i

and maxgt+di−1to sequential

loops decreases the computational time. In order to summarize here all the nec-essary recursions are stated. The general form for all, but the two most bottom

33

station of the backwards recursion is

Sti (gti , c

ti, c

t+di−1) = max

rt+1i

(maxḡt+1i

(f ti (g

ti , ḡ

t+1i , c

ti, c

t+di−1, r

t+1i )

+ V t+1i (ḡt+1i , c

t+1i , c

t+d+1i−1 )

)+ max

gt+di−1

(12St+di−1 (g

t+di−1 , c

t+di−1 + ∆r

t+1i , c

t+d2i−2 )

)). (59)

In the general form for the two most bottom station we have taken away the 12in order to ensure the multipliers always sum up to one. Hence, the backwardsrecursion is

Sti (gti , c

ti, c

t+di−1) = max

rt+1i

(maxḡt+1i

(f ti (g

ti , ḡ

t+1i , c

ti, c

t+di−1, r

t+1i )

+ V t+1i (ḡt+1i , c

t+1i , c

t+d+1i−1 )

)+ max

gt+di−1

(St+di−1 (g

t+di−1 , c

t+di−1 + ∆r

t+1i , c

t+d2i−2 )

)). (60)

The general form for all, but the two most bottom station of the forwardsrecursion is

Sti (gti , c

ti, c

t+di−1) = max

rt+1i

(maxḡt+1i

(f ti (g

ti , ḡ

t+1i , c

ti, c

t+di−1, r

t+1i )

+ V t+1i (ḡt+1i , c

t+1i , c

t+d+1i−1 )

)+

1

2St+di−1 (g

t+di−1 , c

t+di−1 + ∆r

t+1i , c

t+d2i−2 )

). (61)

Similarly to the backwards, we have taken away the 12 in order to ensure themultipliers always sum up to one. Hence, the forwards recursion is

Sti (gti , c

ti, c

t+di−1) = max

rt+1i

(maxḡt+1i

(f ti (g

ti , ḡ

t+1i , c

ti, c

t+di−1, r

t+1i )

+ V t+1i (ḡt+1i , c

t+1i , c

t+d+1i−1 )

)+ St+di−1 (g

t+di−1 , c

t+di−1 + ∆r

t+1i , c

t+d2i−2 )

). (62)

5.6 Release set

An important part of the heuristic is to determine the set of possible releasesthe heuristic can choose from in each station at each time period. It is boundedby the following aspects:

• current reservoir should not overflow,

• current reservoir should not underflow,

34

• bounded by maximum release and minimum release,

• bounded by maximum increase and decrease of the release,

• Any reservoir downstream should not be forced to overflow or underflow.

The last item is more complicated, yet very important.

5.6.1 Current reservoir

The current reservoir should not overflow, underflow and the release is boundedby a maximum and minimum set by the generators. We acquire

Max Release ≤ Max Content− Current Content + incoming, (63)Max Release ≤ Max release of generator combination. (64)

In heuristic Gamma we follow the water downstream hence it is always knownhow much water is incoming from above.

5.6.2 Downstream reservoirs

It is more complicated to not let any of the downstream reservoirs overflow orunderflow. If a reservoir releases a large amount of water, the reservoir belowneeds to process this water. There are three possibilities: the reservoir belowhas enough place left, the reservoir below is forced to spill or flow the water tothe next reservoir, the maximum release of the reservoir below is smaller thanthe amount of water it needs to process. The last option leads to overflowingand hence an infeasible solution. Note it could happen in any of the downstreamreservoirs, not just the reservoir directly below. Let assume we have reservoirA which flows into a downstream reservoir B, which flows into a reservoir C.Now we can state the maximum release of A as how much water we can add toreservoir B without overflowing + the maximum release from B. Since reservoirB could fully open up the generators in order to handle the water that comesfrom A. However, this maximum release from B is in the same way dependent onthe maximum release of C. In the heuristic Gamma, we follow water completelydownstream before going to the next time period which gives us the opportu-nity to calculate the maximum releases properly. We start by looking at themost downstream station since the maximum release there does not depend onanother reservoir. Here we calculate the maximum allowed incoming by thefollowing formula:

Max Incoming = Max Content− Current Content + Max Release. (65)

The maximum incoming in, e.g. reservoir C, caps the max release from thereservoir upstream, reservoir B. Hence, for each time period at the highestreservoir, we calculate all the caps for the max releases of all stations followingthe flow down starting in that timeperiod. In this way we can never force anydownstream reservoir to overflow. Similar method is applied in order to neverlet any downstream reservoir underflow.

35

5.6.3 Most upstream reservoirs

In the most upstream reservoirs, the incoming water from above is consideredknown. An upstream reservoir is a reservoir where no other station flows into,only nature influences the inflow of the reservoir. In the short term model aprediction of nature is taken and considered known. In the rare case that theminimum flow from the reservoir is more than the incoming water by nature forseveral time periods in sequence, a problem of underflowing can occur. In thisperiod the water level is always forced to decrease. Since all the variables areknown, we can calculate when these periods are and take proper precautions.The precaution is to have enough water in the reservoir in order to survive thisperiod of forced decreasing water level. Note that this can only be done with themost upstream reservoirs. For lower reservoirs the algorithm has the possibilityof releasing more water upstream such that the reservoir is not critically low.

5.7 Complexity

Formula for the amount of states:

Amount of states = (2 · Content discretization ·Amount of generator combinations).(66)

Formula for the amount of decisions:

Amount of decisions = (Release discretization ·Amount of generator combinations).(67)

As one can see there is a linear relation between the variables and the amountof states/decisions. Hence a twice as accurate release discretization makes thedynamic programming part twice as slow.

36

6 Heuristic Delta

Heuristic Delta is rather similar to heuristic Gamma. In Gamma there is aproblem that the S and V term were disproportional with respect to each other.In Gamma the solution was to decrease the effect of the S term. Here we lookat the V term and change it into ∆S in order to bring it into proportion. Belowthe formula is stated and the terms and intuition is explained. For station i andtime t we have

Sti (gi, ci, ci−1) = maxḡi

maxri

(maxgi−1

∆Sti−1(gi−1, cti−1(ri), c

ti−2) + fi(gi, ḡi, ci, ci−1, ri)

+ St+di (ḡi, ct+di (ri), c

t+di−1(ri))

), (68)

where

∆S(gi, ci, ci−1) = S(gi, ci(ri), ci−1)− S(gi, ci, ci−1). (69)

where ci(r) is the content level adjusted by the best choice of release and ci thecontent level of the LP solution, hence with the LP release. ∆S is the extraadded value by this choice of release and generators w.r.t. the solution chosenwith the LP release. Hence, it can be seen as the marginal effect of the newlychosen release w.r.t the release as found in stage I. This does not change thechoice of the release and generator combination since S(gi, ci, ci−1) is constantw.r.t the choices. However, it prevents the S term from growing rapidly sinceit is always the addition of two other S terms. S is the value of the entire riverdownstream and ahead of time from the current state. Now both terms S and∆S are in equal proportion.

In order to visually show the proportions are equal, one can look at Figure19. The green color represents the influence of the S term, the red color representthe influence of the ∆S term and the yellow color the influence of both terms. Itis logical that the yellow circles have influence from both since water followingthe arrows can reach the yellow colors via red or green. However, due to thedifference taken into the ∆S term the yellow nodes are not counted twice andhence everything is in proportion. The choice of the releases and updating ofthe water balancing equation are the same as in Heuristic Gamma.

Figure 19: Difference of V and S visualized

37

6.1 General form

In order to summarize all the necessary recursions are stated below. The generalform of the backwards recursion is

Sti (gi, ci, ci−1) = maxri

(maxḡi

(fi(gi, ḡi, ci, ci−1, ri) + S

t+di (ḡi, c

t+di (ri), c

t+di−1(ri))

)+ max

gi−1∆Sti−1(gi−1, c

ti−1(ri), c

ti−2)

), (70)

and the general form of the forward recursion is:

Sti (gi, ci, ci−1) = maxḡi

maxri

(∆Sti−1(gi−1, c

ti−1(ri), c

ti−2) + fi(gi, ḡi, ci, ci−1, ri)

+ St+di (ḡi, ct+di (ri), c

t+di−1(ri))

). (71)

38

7 Epsilon

In heuristic Epsilon the focus lies on the handling of a reserve constraint thatconstraints on a certain time period over all stations instead of on a separatestation. This makes it harder since in heuristic Gamma and Delta there is nopossibility to enforce such a constraint. In order to fulfill the constraint anadditional state has to be added, which keeps track of the reserve and passesthis information to the next station downstream in the same time period.

7.1 Reserve

The amount of a reserve a station can supply at a time is a function of thegenerator combination, the current power production and the maximum powerproduction. If it is possible to produce more power (hence the necessary gener-ators are turned on) before reaching the maximum power production, we havea reserve. This yields

Reserve = Max power production given generator combination− Current power production.(72)

The additional constraint has the form∑i

rsti ≥ rstup, ∀t ∈ T, (73)

where RSti is the amount station i contributes to the reserve requirement attime t. Note the maximum reserve is set for a time period t and can differ pertime period.

7.2 Dynamic programming equations

The backwards recursion of heuristic epsilon is stated as

Sti (gt−1i , c

t−1i , rs

ti) = max

gti

maxrti

f ti (gt−1i , c

t−1i , rs

ti, g

ti , r

ti) + V

t+1i (g

ti , c

ti(r

ti), 0)

+ maxgt−1i−1

Sti−1(gt−1i−1 , c̃

t−1i−1, rs

ti−1(g

ti , r

ti)). (74)

and the V term is updated similarly as in heuristic Gamma. Two terms foundin Equation 74 are simply subtracted in order to acquire V . Note that, just asin heuristic Gamma, a new optimization problem is not solved here.

V ti (gt−1i , c

t−1i , rs

ti) = S

ti (g

t−1i , c

t−1i , rs

ti)−max

gt−1i−1

Sti−1(gt−1i−1 , c̃

t−1i−1, rs

ti−1(g

ti , r

ti))

= maxgti

maxrti

f ti (gt−1i , c

t−1i , rs

ti, g

ti , r

ti) + V

t+1i (g

ti , c

ti(r

ti), 0).

(75)

The most bottom station is making the reserve constraint feasible. If the amountof not fulfilled reserve is higher than the amount the most bottom station cancontribute to the reserves, the constraint does not hold. Hence, all states for

39

the most bottom station where it has a too high not fulfilled reserve variableshould have the cost value infinity. For the most bottom station i = 0, we have

St0(gt−10 , c

t−10 , rs

t0) = max

gt0

maxrt0

f t0(gt−10 , c

t−10 , rs

t0, g

t0, r

t0) + V

t+10 (g

t0, c

t0(r

t0), 0),

(76)

where f t0 has value −∞ if rst0 is higher than the max reserve of the station. Bythe nature of dynamic programming, this implies for the next station upstream alot of states with high reserves are also infinity and hopefully the most upstreamstation has finite states for the amount of reserve required. If so the forwardrecursion simply walks down stream while avoiding the infinity states and henceending up at the most bottom station with the reserve constraint fulfilled. TheV term is calculated in similar fashion as above., namely

V t0 (gt−10 , c

t−10 , rs

t0) = S

t0(g

t−10 , c

t−10 , rs

t0). (77)

Note that in Equation 74 there is one state-variable unknown in V t+1i (gti , c

ti(r

ti), rs

t+1i ),

namely , rst+1i , the reserve constraint in the next time period. In Sti−1(g

t−1i−1 , c̃

t−1i−1, rs

ti−1(g

ti , r

ti)

there are even two state-variables unknown, namely the generator combinationof the reservoir below, gt−1i−1 , and the content of the reservoir below, c

t−1i−1. These

three variables cannot be deduced from the state and decisions and hence haveto be estimated. Here we choose to use the best estimate for the content of thereservoir below, hence the stage I solution. For both the reserve and the gen-erator combination we choose to insert the best variable available. In the caseof the generator combination this means taking the maximum of the possiblegenerator combinations, similar to heuristic Gamma and Delta. For the reserveconstraint this means taking 0 reserve as state for the next time period. Notethat this is only used to determine the value of choice of the release and gener-ator combination for the current time period and current reservoir. During theforward recursion, the start generator combination of the reservoir below andthe start content of the reservoir below is already determined (hence gt−1i−1 and

ct−1i−1 are known). The forward recursion is therefore written as

Sti (gt−1i , c

t−1i , rs

ti) = max

gti

maxrti

(f ti (g

t−1i , c

t−1i , rs

ti, g

ti , r

ti) + V

t+1i (g

ti , c

ti(r

ti), 0)

+ Sti−1(gt−1i−1 , c

t−1i−1, rs

ti−1(g

ti , r

ti))

). (78)

One should note that in Equation 74, the Sti−1 is dependent on the chosengenerator combination gti , unlike as in heuristic Gamma and Delta. In Gammaand Delta this made it possible to avoid the nesting of two loops, yet that is notpossible here. This increases the needed computation time since the backwardsrecursion in heuristic Epsilon has three nested loops for each possible state,instead of two nested loops in heuristic Gamma and Delta.

7.3 Parallel with Knapsack problem

The problem of where to allocate the reserves is formulated and solved as theknapsack problem. Below a short explanation of the knapsack problem is given

40

and the parallel with the above formulation is explained. A formulation of theknapsack problem [9]:

maximize z =

n∑i=1

bi · xi, (79)

subject to

n∑i=1

wi · xi < W, (80)

and xi ≥ 0 and integer, (81)

where bi is the value of item xi and wi is the weight of item xi. We want to opti-mize the value of all the item that fits in our knapsack with a weight restriction.In dynamic programming we define a function V (w) which has the optimal valuefor a backpack of size w. The corresponding dynamic programming recursion isstated as:

V (0) = 0, (82)

V (w) = maxwi

(bi + V (w − wi)), (83)

such that w − wi ≥ 0. (84)

Hence, for each item it is checked which one has the most added value (bi) plusoptimal value for left over weight (V (w − wi)).The parallel between this and the Equation 74 is as follows. In the knapsackproblem we check for all the items which one is best to put in the knapsack at thecurrent moment. In heuristic Epsilon we check for all possibilities of release andgenerator combination which choice is best at the current time period and sta-tion and hence how much reserve to add. In the knapsack problem we have bi asadded value from the current choice of item, the equivalent in heuristic Epsilonis f ti (g

t−1i , c

t−1i , rs

ti, g

ti , r

ti) + V

t+1i (g

ti , c

ti(r

ti), 0). In the knapsack recursion we

have V (w−wi) as the optimal value of the state we end up in when taking itemwi. In heuristic Epsilon the equivalent is maxgt−1i−1

Sti−1(gt−1i−1 , c

t−1i−1, rs

ti−1(g

ti , r

ti)).

Note here that rsti−1(gti , r

ti) is a function of the choice of release and generator

combination and determines in to which state we go.

7.4 Incoming water assumption

During the backward recursion, the values of the states are evaluated whileassuming the values such as incoming water from the stage I solution. It islikely that the incoming water changes in the forward recursion since it is directlyrelated with the release we determine. This implies the values of the states canbe a bit off and might lead to a suboptimal solution. This is usually not a largeproblem, yet in some situation it is problematic. Imagine a station where itis optimal to contribute 200 MW to the reserve constraint in a certain state.However, if now in the forward recursion the station upstream release a lot ofwater and hence the incoming water of this station increases, there is a problem.It is likely that in order to deal with the large income of water the station isforced to release more water than optimal otherwise, while being in the samestate. This in its turn has to consequence that it cannot contribute 200 MWto the reserve constraint but maybe only 150 MW at its maximum. This in its

41

turn can make it infeasible to fulfill the reserve constraint for this time period.There are quite a lot conditions that need to be fulfilled when such a scenariooccurs, however when the reserve constraints requires to have a large amountof reserve this happens. The solution is to ignore this and continue to stageIII. In stage III it is likely to fulfill the reserve constraint with the generatorcombination fixed by heuristic Epsilon.However, even when such a extraordinary situation does not occur there stillis the problem that the value of the states are slightly off. This can lead to asuboptimal solution and is a disadvantage of method Epsilon.

7.5 Complexity

Formula for the amount of states:

Amount of states = (Reserve discretization · Content discretization ·Amount of generator combinations).(85)

Formula for the amount of decisions:

Amount of decisions = (Release discretization ·Amount of generator combinations).(86)

As one can see there is a linear relation between the variables and the amountof states/decisions. Hence, a twice as accurate reserve discretization makes thedynamic programming part twice as slow.

42

8 Results

In order to compare the heuristics, they are tested on several different datasets.As stated in section 2.4 the datasets are compared and evaluated on:

• optimality: Gap between global optimum and value found by heuristic,

• robustness,

• speed.

The heuristic usually ends in a local optimum near the global optimum and israther sensitive to the inputed variables by the user. These variables are thediscretization of the content, discretization of the release and deviation fromthe stage I solution. In order to find good values to use and compare all theheuristics, a sensitivity analysis is done using different sets of input variablesand the normal datasets of values of the river. First heuristic Alpha , Gammaand Delta are compared on datasets where there are no reserve constraints.Separately heuristic Alpha is be compared with heuristic Epsilon on a datasetwith reserve constraints. All the tables shows how much percentage the heuristicis worse than the solution supplied by CPLEX. Between the brackets the tablesshow the run time. Note that this percentage can be negative if a better solution,than supplied by CPLEX, is found.

8.1 Alpha, Gamma and Delta without reserve constraints

In both the sensitivity of the discretization of the release and of the contentsteps the time aspect is important. In order to compete with heuristic Alpha,the goal is to keep the time it takes for the heuristics to run on the testcaseLuleälven under 10 seconds.

8.1.1 Sensitivity of discretization of release

When determining how to discretize the release, one should take into accountthis only affects stage II and in stage II only the generator combination found isused in the final solution as determined in stage III. Hence, the release we deter-mine in stage II does not have to be very accurate, as long as the correspondinggenerator combination is good. As can be seen in Table 1, the solution is veryinsensitive to the amount of release steps. Note that heuristic Alpha takes adifferent approach on the release discretization, instead of always splitting it inthe same number of releases, one can specify in absolute terms how large therelease step should be. Since this is specified in the dataset, the release stepsconsidered here does not affect heuristic Alpha. The outcome might be differentin a different dataset, so another sensitivity analysis is performed on a datasetwith a price that replicate a day and night rhythm, as shown in Table 2. Fromthis data is deduced that discretization of the release in just 5 different levelsis sufficient. However, in case the optimality gap is large, a discretization inusing a larger amount of steps can be tested, yet the speed of stage II decreaseslinearly with the increase of amount of steps. This is logical since the amountof states increases linearly with the amount of release steps.

43

Table 1: Results with different release steps and Content steps = 5 on normaldataset

Amount of discretizationsin release

Alpha Gamma Delta

3 0.27% ( 4.9sec) 2.47% ( 5.4sec) 2.14% ( 6.2sec)4 0.27% ( 4.9sec) 2.83E-4% ( 7.2sec) 1.98E-3% ( 8.6sec)5 0.27% ( 4.9sec) 2.72E-4% ( 9.3sec) 1.98E-3% ( 11.1sec)6 0.27% ( 4.8sec) 0.30% ( 11.2sec) 1.99E-3% ( 13.8sec)7 0.27% ( 4.8sec) 0.26% ( 13.1sec) 1.99E-3% ( 16.1sec)8 0.27% ( 4.8sec) 0.03% ( 15.2sec) 1.99E-3% ( 18.6sec)9 0.27% ( 4.9sec) 2.91E-4% ( 17.2sec) 1.99E-3% ( 21.3sec)10 0.27% ( 4.8sec) 0.30% ( 19.4sec) 1.99E-3% ( 23.8sec)

Table 2: Results with different release steps and Content steps = 5 on datasetwith a day and night rhythm on the prices

Amount of discretizationsin release

Alpha Gamma Delta

3 0.04% ( 4.1sec) 5.85% ( 4.7sec) 1.08% ( 5.4sec)4 0.04% ( 4.0sec) 0.20% ( 6.4sec) 1.32% ( 7.7sec)5 0.04% ( 4.0sec) 0.22% ( 8.4sec) 0.38% ( 10.1sec)6 0.04% ( 4.0sec) 0.22% ( 10.2sec) 1.37% ( 12.3sec)7 0.04% ( 4.1sec) 0.25% ( 11.9sec) 2.26% ( 14.5sec)8 0.04% ( 3.9sec) 0.23% ( 13.8sec) 1.33% ( 17.0sec)9 0.04% ( 3.9sec) 0.24% ( 15.9sec) 1.42% ( 19.8sec)10 0.04% ( 4.1sec) 0.23% ( 17.4sec) 2.53% ( 21.3sec)

44

Table 3: Results with different content level discretizations and release steps =5 on normal dataset)

Amount of discretizationsin content

Alpha Gamma Delta

3 2.89E-4% ( 3.8sec) 1.25% ( 3.1sec) 10.74% ( 3.3sec)5 0.27% ( 4.9sec) 2.83E-4% ( 7.6sec) 1.98E-3% ( 9.0sec)7 2.92E-4% ( 6.2sec) 0.03% ( 17.1sec) 1.98E-3% ( 20.8sec)9 2.24E-4% ( 7.2sec) 0.56% ( 34.3sec) 1.99E-3% ( 42.4sec)11 2.92E-4% ( 8.2sec) 0.03% ( 60.7sec) 1.98E-3% ( 74.3sec)12 2.92E-4% ( 9.4sec) 0.55% ( 99.8sec) 1.98E-3% (124.7sec)

Table 4: Results with different content level discretizations and release steps =3 with a day and night rhythm on the prices

Amount of discretizationsin content

Alpha Gamma Delta

3 0.06% ( 3.2sec) 0.77% ( 2.7sec) 1.85% ( 3.1sec)5 0.04% ( 4.1sec) 0.20% ( 6.3sec) 1.32% ( 7.8sec)7 0.04% ( 4.9sec) 0.24% ( 14.7sec) 2.32% ( 18.6sec)9 0.05% ( 5.9sec) 0.25% ( 29.3sec) 1.38% ( 37.5sec)11 0.03% ( 6.8sec) 0.23% ( 53.0sec) 1.39% ( 67.4sec)

8.1.2 Sensitivity of discretization of content level

When determining how to discretize the content level one should keep in mindthat the value of the content level that do not belong exactly in the discretizationare approximated by interpolation. In Table 3 and Table 4, one can see fordifferent amount of discretization the result.

From this data is deduced that the discretization of the content levels in5 different contents is sufficient. At 5 different content levels the heuristics istakes less than 10 seconds, which is useful when comparing multiple datasets.One can see here that the number of content steps also has an influence onheuristic Alpha and for heuristic Alpha a higher amount of content steps ispossible without a long runtime. In order to compare the heuristics, there ischosen to have the same content steps in each heuristic. However, there is ananalysis done where the amount of content steps of Alpha is increased such thatis gets the same runtime as Gamma and Delta. The results can be found inAppendix B.1. The speed of stage II decreases linearly with the increase ofamount of steps. This is logical since the amount of states increases linearlywith the amount of content levels.

8.1.3 Results on different datasets

Using the values determined above, the heuristics are tested on 20 differentdatasets. The datasets in Table 5 have been given short descriptive nameswhich are explained in detail in Appendix A. Note that the optimality gapcan be negative if the heuristics gives a better solution than CPLEX reachesafter the convergence criteria of CPLEX are reached