A Multilevel Coordinate Search Algorithm for Well ... · of most important tasks in solving well placement and control optimization problems. In this work, we introduce and apply

A Multilevel Coordinate Search Algorithm for Well

Placement, Control and Joint Optimization

Qihong Fenga, Ronald D. Haynesb,∗, Xiang Wanga,∗∗

aSchool of Petroleum Engineering, China University of Petroleum (Huadong), Qingdao,Shandong, China 266580

bDepartment of Mathematics & Statistics, Memorial University of Newfoundland, St.John’s, NL, Canada A1C 5S7

Abstract

Determining optimal well placements and controls are two important tasksin oil field development. These problems are computationally expensive,nonconvex, and contain multiple optima. The practical solution of theseproblems require efficient and robust algorithms. In this paper, the multi-level coordinate search (MCS) algorithm is applied for well placement andcontrol optimization problems. MCS is a derivative-free algorithm that com-bines global search and local search. Both synthetic and real oil fields areconsidered, and the performance of MCS is compared to the generalizedpattern search (GPS), the particle swarm optimization (PSO), and the co-variance matrix adaptive evolution strategy (CMA-ES) algorithms. Resultsshow that the MCS algorithm is strongly competitive, and outperforms forthe joint optimization problem and with a limited computational budget.The effect of parameter settings are compared for the test examples. For thejoint optimization problem we compare the performance of the simultaneousand sequential procedures.

Keywords: Well Placement, Well Control, Joint Optimization, MultilevelCoordinate Search, Derivative-free optimization, Reservoir simulation-basedoptimization

∗Corresponding author∗∗Principal corresponding authorEmail addresses: [email protected] (Ronald D. Haynes), [email protected]

(Xiang Wang)

Preprint submitted to Computers & Chemical Engineering April 4, 2019

arX

iv:1

510.

0351

7v1

[m

ath.

OC

] 1

3 O

ct 2

015

1. Introduction

Determining the optimal well locations and controls in an oil field is a chal-lenging task. The decision is hard to make since the reservoir performance isaffected by geological, engineering, economical and other parametersTavallaliet al. (2013); Knudsen and Foss (2013); Shakhsi-Niaei et al. (2014). Opti-mization algorithms provide a systematic way to solve this problem. Byusing optimization algorithms, a quality solution can be achieved automat-ically and hence reduce the risk in decision-making. Well placement andcontrol optimization generally are computationally expensive and noncon-vex, and not every optimization algorithm is appropriate for these problems.Therefore, finding and applying algorithms that are efficient and robust is oneof most important tasks in solving well placement and control optimizationproblems.

In this work, we introduce and apply the multilevel coordinate search(MCS) algorithm for the problems of optimizing well placement, well con-trol, and joint placement with control. MCS Huyer and Neumaier (1999) is aglobal optimization algorithm and is designed to handle the complex topogra-phy and multimodality of the multidimensional nonlinear objective functionswithout requiring excessive computing resources. Rios Rios and Sahinidis(2013) completed a systematic comparison using a test set of 502 problems.MCS outperforms the other 21 derivative-free algorithm implementations(see Table 1). Though MCS has shown its superiority in benchmark andreal world problems Huyer and Neumaier (1999); Rios and Sahinidis (2013);Lambot et al. (2002), to the best of our knowledge, it dose not appear tohave been applied for the optimization of oil field development. We compareMCS, generalized pattern search (GPS), particle swarm optimization (PSO),and covariance matrix adaptive evolution strategy (CMA-ES) in three typ-ical test cases from the field of optimal reservoir production development.Our results demonstrate that MCS is strongly competitive and outperformsthe other algorithms in most cases.

Oil field development optimization has two main sub-problems: wellplacement optimization, and well control optimization. These two prob-lems are often treated separately Oliveira and Reynolds (2014); Bouzarkounaet al. (2012); Wang et al. (2009); Brouwer and Jansen (2004). Recently, therehas been increasing focus on optimizing well placement and control jointlyForouzanfar et al. (2015); Humphries et al. (2013); Isebor et al. (2014a). Wellplacement problems aim to optimize the locations of injection and produc-

2

Table 1: Derivative-free solvers considered in Rios’ work.

Solver Version Language

ASA 26.30 CBOBYQA 2009 FortranCMA-ES 3.26beta MatlabDAKOTA/DIRECT 4.2 C++DAKOTA/EA 4.2 C++DAKOTA/PATTERN 4.2 C++DAKOTA/SOLIS-WETS 4.2 C++DFO 2.0 FortranFMINSEARCH 1.1.6.2 MatlabGLOBAL 1.0 MatlabHOPSPACK 2.0 C++IMFIL 1.01 MatlabMCS 2.0 MatlabNEWUOA 2004 FortranNOMAD 3.3 C++PSWARM 1.3 MatlabSID-PSM 1.1 MatlabSNOBFIT 2.1 MatlabTOMLAB/GLCCLUSTER 7.3 MatlabTOMLAB/LGO 7.3 MatlabTOMLAB/MULTIMIN 7.3 MatlabTOMLAB/OQNLP 7.3 Matlab

tion wells. The location of each vertical well is parametrized by its planecoordinates (x, y), which are usually integers in the reservoir simulator. Wellcontrol problems focus on optimizing production scheduling. The optimiza-tion variables are often the time-varying bottom hole pressures (BHPs) orthe flow rates for each well. The joint problem optimizes well placement andcontrol parameters simultaneously. Thus, the joint problems are more com-plex and challenging with an increase in the number and type of variablesIsebor et al. (2014a).

In the past, a number of algorithms have been devised and analysed forboth separate and joint problem of well placement and control optimiza-tion. These algorithms fall into two categories: gradient-based methods and

3

derivative-free methods. Applications of gradient-based methods to oil fieldproblems have been presented in many papers Volkov and Voskov (2014);Wang et al. (2009); Brouwer and Jansen (2004); Zandvliet et al. (2008);Sarma et al. (2006); Zhou et al. (2013). These methods take advantageof the gradient information to guide their search. The gradient of the ob-jective function can be obtained by using adjoint-based techniques Volkovand Voskov (2014); Zandvliet et al. (2008); Sarma et al. (2006), or may beapproximated by using numerical methods such as finite difference perturba-tion Wang et al. (2009); Zhou et al. (2013). These algorithms do have somedrawbacks for the well placement and control problem; these problems arenonconvex and generally contain multiple optima. For some problems, par-ticularly well placement, the optimization surface can be very rough, whichresults in discontinuous gradients Ciaurri et al. (2011).

For the joint well placement and control optimization problem, two pro-cedures are proposed and studied. The first one is a simultaneous procedure,which optimizes over all well locations and control parameters simultane-ously. The second one is a sequential procedure, that decouples the jointproblem into the well placement optimization subproblem and the well con-trol placement optimization subproblem. The simultaneous procedure en-sures that the best solution exists somewhere in the search space. But it maybe difficult to find the global optima because the search space may be verylarge and rough. The sequential procedure divides the optimization variablesinto two smaller groups and optimizes separately. For each subproblem, thesearch space is smaller than the simultaneous one, but it can not ensure thebest solution exists in the search space because the optimal location dependson how the well is operated and vice-versa. There are several papers Li et al.(2012); Bellout et al. (2012); Isebor et al. (2014b) which demonstrate that thesimultaneous procedure is superior to the sequential approach. In Humphrieset al. Humphries et al. (2013); Humphries and Haynes (2015), however, theyfound that a sequential procedure was competitive and even preferable tothe simultaneous approach in some cases. Based on this type of argument,we do further investigation of the effectiveness of these two procedures usinga test example.

Many black-box, derivative-free methods have been used in oil field prob-lemsMerlini Giuliani and Camponogara (2015). Typical algorithms includegenetic algorithms (GA) AlQahtani et al. (2014); Bouzarkouna et al. (2012),particle swarm optimization (PSO) Onwunalu and Durlofsky (2009, 2011),generalized pattern search (GPS) Asadollahi et al. (2014); Isebor (2009), co-

4

variance matrix adaptation strategy (CMA-ES) Bouzarkouna et al. (2012);Forouzanfar et al. (2015) and hybrid approaches Isebor et al. (2014a); Humphriesand Haynes (2015). These algorithms can be classified as either determinis-tic or stochastic, and provide global or local search. The stochastic/globalapproaches have also been hybridized with deterministic/local search tech-niques. These algorithms only require the value of objective function andinvolve no explicit gradient calculations. For an optimization problem, thederivative-free methods generally need more function evaluations to convergeas compared to the gradient-based ones. However, most of these algorithmsparallelize naturally and easily, which make their efficiency satisfactory Ciau-rri et al. (2011).

We are particularly interested in using the multilevel coordinate search(MCS) algorithm for the following reasons: 1) it combines a global searchwith a local search, which leads to a quicker convergence than many meth-ods that operate only at the global level. 2) it is an intermediate betweenheuristic methods that find the global optimum only with high probabilityand methods that guarantee to find a optimum with a required accuracy.3) it does not need analytic or numerical derivatives. 4) it is guaranteed toconverge if the objective is continuous in the neighbourhood of a global min-imizer, no additional smoothness properties are required. 5) the algorithmparameters in MCS have a clear meaning and are easy to choose. 6) it hasproved itself in benchmark tests and many real world problems Huyer andNeumaier (1999). Based on these features, we believe that MCS has a lotof potential to solve oil field optimization problems, which are nonconvex,nonlinear, containing many local optima and discontinuities.

In this paper, we apply MCS to optimization problems of varying com-plexity in terms of the number and type of optimization variables, the dimen-sion and size of the reservoir models, and the number of wells. We investigatethe effects of the algorithmic parameters (initialization list type, number oflevels, and the affect of local search) on the optimization results. We pro-pose a detailed comparison between MCS and three other popular algorithms(GPS, PSO, and CMA-ES).

This paper is organized as follows: Section 2 describes the formulationsof the well placement, the well control, and the joint optimization problems.Section 3 gives an overview of the optimization algorithm MCS. In Section4 we list the optimization algorithms and configurations considered and de-scribe our numerical experiments. Results and discussions are presented inSection 5. Finally, in Section 6 we provide some concluding remarks.

5

2. Problem Formulation

In this section, we formulate the optimization problem of interest – de-scribing the well placement optimization problem and the well control opti-mization problem separately, then we give the joint problem formulation.

The objective functions for these problems are often the net present value(NPV) or cumulative oil production. We use NPV as the objective functionfor all our work. NPV accounts for revenue from the oil and gas produced,and for the cost of handling water production and injection. NPV is definedas

NPV =Nt∑k=1

[∆tk

(1 + b)tkτ

(Np∑i=1

rgpqi,kgp +

Np∑i=1

ropqi,kop −

Np∑i=1

cwpqi,kwp −

Ni∑i=1

cwiqi,kwi

)],

(1)where qi,kgp , qi,kop , qi,kwp and qi,kwi are the flow rates of the gas, oil, water producedand water injected for well i at time step k, respectively; rgp and rop are thegas and oil revenue; cwp and cwi are the cost of water produced and injected.Nt is total number of time steps, tk is the time at the end of kth time step,∆tk is kth time step size, τ provides the appropriate normalization for tk,e.g., τ = 365 days, and b is the fractional discount rate.

Well placement optimization. In the well placement optimization problem,we seek to determine the optimal locations for a specified number of produc-tion and injection wells. The optimization problem studied here is definedas follows:

minx1∈Zn1

−NPV (x1) (2)

subject to u1 ≤ x1 ≤ v1, (3)

where x1 denotes the discrete well placement variables. Well locations arereal variables in actual fields but are usually treated as integers in reservoirsimulators. The well placement variables in our work are continuous but willbe rounded before we pass them to the simulator to evaluate the objectivefunction. All wells in our work are assumed to be vertical, hence the locationsof each well are given by plane coordinates (x, y). Thus the total number ofvariables n1 = 2(Np + Ni), where Np and Ni are the number of productionand injection wells, respectively.

6

For given well locations x1, the flow rates of the gas, oil, water produced,and water injected qgp, qop, qwp, qwi for each well over the development lifecycle are calculated by using a reservoir simulator. The value of NPV canthen be obtained by equation (1).

Though both linear or nonlinear constraints can be handled using penaltyfunctions, filter methods, or other techniques, we deal with only bound con-straints u1 and v1 in our work.

Well control optimization. The well control optimization problem aims todetermine the optimal time-varying well setting for each of the productionand injection wells. The optimization problem can be stated as follows:

minx2∈Rn2

−NPV (x2) (4)


where x2 denotes the well control variables. The time-varying well controlsare represented by piecewise functions in time with Nt intervals. The numberof variables for this problem is n2 = Nt(Ni +Np).

In this case, the well locations are fixed, and the flow rates of the gas, oil,water produced, and water injected qgp, qop, qwp, qwi are functions of the wellcontrols x2. Again bounds u2 and v2 are placed on the control variables.

Joint well placement and control optimization. The joint problem optimizesboth well locations and controls. The optimization problem is defined asfollows:

minx1∈Zn1 ,x2∈Rn2

−NPV (x1,x2) (6)


u2 ≤ x2 ≤ v2, (8)

where x1 and x2 denote the well location and control variables. The flowrates of the gas, oil, water produced, and water injected qgp, qop, qwp, qwi arefunctions of the well locations x1 and the well controls x2.

Two procedures are commonly used for joint well placement and controloptimization—a simultaneous procedure or a sequential procedure. The si-multaneous procedure optimizes well locations and controls simultaneously,hence the number of optimization variables, n1 + n2 = (2 + Nt)(Ni + Np),

7

is larger than the separate problems, which makes the optimization moredifficult.

In the sequential procedure, well placement is optimized first using somereasonable control scheme. The controls are then optimized for the wells withthe best locations found in the first stage. These two stages are repeated.The sequential procedure decouples the joint problem into two separate sub-problems, and the difficulty for each subproblem is decreased. The number ofoptimization variables for the well placement stage is n1 = 2(Np+Ni), for thewell control stage is n2 = Nt(Ni +Np). The joint problem is worth studyingbecause the optimal location of each well depends on how the well is operatedHumphries et al. (2013); Humphries and Haynes (2015) and vice-versa.

3. Multilevel Coordinate Search

MCS, first proposed by Huyer and Neumaier Huyer and Neumaier (1999),was inspired by DIRECT Jones et al. (1993) —a branching scheme whichsearches by recursively splitting hyperrectangles. Like DIRECT, MCS isa mathematical programming approach which provides a systematic searchwithin the bound constraints (the bounds can be infinite for MCS). MCSbuilds upon DIRECT by introducing a multilevel mechanism which allows abalanced global and local search. DIRECT has no local search capability.

Levels are assigned as an increasing function of the number of times a boxhas been split. The global search portion of the algorithm is accomplishedby splitting boxes that have not been searched often – those with a lowlevel. Within a level the boxes with the best function values are selected todo the local search. The local search builds a quadratic model, determines apromising search direction and performs a line search. This allows for quickerconvergence while the global part of the algorithm identifies a region nearthe global optimum.

MCS allows for a more irregular splitting than DIRECT, giving preferenceto regions with low function values. Convergence to a global minimum isguaranteed as the number of levels goes to infinity if the objective functionis continuous around the global optimizer.

Huyer and Neumaier Huyer and Neumaier (1999) reports that MCS workswell in problems where the global optimum can be constrained by finitebound constraints. Posık, Huyer, and Pal Posık et al. (2012) report verygood performance in the early search phase with a small budget of objectivefunction evaluations.

8

MCS provides numerous heuristic enhancements over DIRECT, the pseu-docode of the basic steps of MCS can be found in Alg. 1. A complete de-scription of the algorithm is quite complex and can be found in Huyer andNeumaier (1999). For the experiments in this paper, we used the implemen-tation of MCS from Neumaier (2008).

In the results presented in Section 5 we will test 5 configurations of MCS.Here we provide a little more detail of some configurable aspects of MCSwhich motivate the configurations tested.

During the initialization portion of the algorithm MCS accepts an ini-tialization list which is used to produce an initial set of boxes partitioningthe search space. MCS continues to process and split boxes until some boxesreach level smax. Hence smax controls the precision of the global search phasebefore any local search would be attempted. MCS also has the option toturn off the local search phase.

We provide a simple example to see how MCS works. We consider theobjective function f = x2

1(4 − 2.1x21 + x4

1/3) + x1x2 − 4x22(1 − x2

2), whichis a four-hump camel function with 2 unknowns. We use a lower boundof [−3,−2] and an upper bound of [3, 2]. The global minimizer for thisfunction is [0.0898,−0.7127] and the global minimum value is -1.0316. Wechoose the default parameter settings for MCS: the initialization list x1

i = ui,x2i = (ui + vi)/2, x3

i = vi, a maximum number of level smax = 5n + 10 = 20,and we turn the local search phase on. Fig. 1 shows a loop of MCS whensolving the problem.

Fig. 1(a) presents the boxes after the initialization procedure (lines 1–4 of Alg. 1). By using the default initialization list, MCS first splits theroot box along the x-coordinate at the midpoint, the two boundary points,and the points determined by the golden ratio between. Then MCS choosesone of the new boxes that has the highest estimated variability and splits italong the y-coordinate. The initialization list is defined by the user. Differentinitialization lists results in a different split of the boxes. One other commonlyused initialization list is x1

i = (5ui+vi)/6, x2i = (ui+vi)/2, x3

i = (ui+5vi)/6.By using this initialization list, the boxes after the initialization procedureare as shown in Fig. 19.

After the initialization procedure, the search space will be further splituntil one of the boxes reaches the maximum level smax. Fig. 1(b) shows theboxes after the splitting procedure. As mentioned above, smax decides thedepth to which MCS explores a region and hence controls the precision ofthe global search phase. If smax = 5n, then the boxes obtained as shown in

9

Algorithm 1: The MCS algorithm

Input : Evaluation function f , bound constraints u,vInput : Initialization list xji (j = 1, · · · , Li, i = 1, · · · , n), maximum

level smax, local search on-off state1 for i = 1 to n do

2 x← the best of {xj}Lij=1, where xj is x with xi changed to xji ;

3 Split the current box B along the ith coordinate at xji and between;4 B ← the one has best function value of the boxes containing x;

5 while there are boxes of level s < smax do6 for all non-empty levels s = 2 to smax − 1 do7 Choose the box B at the level s with the lowest function value;8 i← the coordinate used least often when producing box B;9 if s > 2n(i+ 1) then // Split by rank

10 Split the box B along the ith coordinate;

11 else if s ≤ 2n(i+ 1) then // Split by expected gain

12 Determine the most promising splitting coordinate i;13 Compute the minimal expected function value fexp at new

point;14 if fexp<fbest

then15 Split B along the ith coordinate;16 else17 Tag B as not promising and increasing its level by 1;

18 for Base points x of all the new boxes at level smax do19 Start a local search from x if improvement is expected;

Output: xbest,fbest

10

Fig. 19.Once a box reaches the maximum level, a local search starts from its base

point. Fig. 1(c) shows the points evaluated by the local search. After that,MCS will cycle back to the split procedure.

-3 -2 -1 0 1 2 3-2

-1

0

1

2

(a) Initialization

-3 -2 -1 0 1 2 3-2

-1

0

1

2

(b) Splitting

-3 -2 -1 0 1 2 3-2

-1

0

1

2

(c) Local search

Figure 1: The boxes and test points of MCS for the six-hump camel function. The dashedlines are the contour lines of the function. The global optima is known and is indicatedby ’+’. The test points are indicated by dots, the black dots are points tested duringinitialization procedure, while the blue and green dots are the points tested during thesplitting procedure and the local search procedure, respectively.

4. Methodology

In this section, we list the algorithms and the configurations tested, andgive a detailed description of the test examples used in this paper. Theexamples include a well placement optimization problem, a well control op-

11

-3 -2 -1 0 1 2 3-2

-1

0

1

2

Figure 2: The boxes after the initialization procedure of MCS with the initialization listx1i = (5ui + vi)/6, x2

i = (ui + vi)/2, x3i = (ui + 5vi)/6.

-3 -2 -1 0 1 2 3-2

-1

0

1

2

Figure 3: The boxes after the splitting procedure of MCS with smax = 5n.

timization problem and a joint optimization problem of well placement andcontrol.

4.1. Optimization algorithms and configurations considered

The main optimization algorithm considered in this paper is MCS asdescribed in Section 3. For comparison three algorithms – generalized pat-tern search (GPS), particle swarm optimization (PSO) and covariance matrixadaptation evolution strategy (CMA-ES) – are used. The MCS code we useis version 2.0 from Neumaier (2008), which is written in Matlab MATLAB(2012).

In order to analyse the sensitivity of the parameters in the MCS algo-rithm, we apply MCS with five different settings of the parameters to thethree examples. The five settings used are:

• MCS-1: MCS with its default settings from Huyer and Neumaier (1999).

12

A simple initialization list is used consisting of midpoints and boundarypoints, i.e. x1

i = ui, x2i = (ui + vi)/2, x

3i = vi. The number of levels

is chosen as smax = 5n + 10, where n is the dimension of the problem.The maximal number of visits in local search is 50, and the acceptablerelative accuracy for local search is γ = 0.01.

• MCS-2: MCS with the initialization list x1i = (5ui + vi)/6, x

2i =

(ui + vi)/2, x3i = (ui + 5vi)/6. Unlike the initialization list in MCS-1,

the points here are uniformly spaced but do not include the boundarypoints. The other settings are same as in MCS-1.

• MCS-3: MCS with an auto-generated initialization list. In MCS-3, wefirst perform a sequence of line searches along all coordinate directionsto generate the initialization list. The other settings are same as inMCS-1.

• MCS-4: MCS with a larger maximum number of levels, smax = 10n.This is chosen to improve its ability in the global search phase. Theother settings are same as in MCS-1.

• MCS-5: MCS without the local search phase. In MCS-5, we set themaximal number of visits in the local search to 0. The other settingsare same as in MCS-1.

Generalized pattern search (GPS) Audet and Dennis (2002); Torczon(1997); Yin and Cagan (2000) is a deterministic local search algorithm. Itdoes not require the gradients and hence, it can be used on problems thatare not continuous or differentiable. For the parameter settings, we use 2Npositive spanning set for all three examples, where N is the dimension of thesearch space.

Particle swarm optimization (PSO) Kennedy (2011); Vaz and Vicente(2007) is a population-based stochastic search method. PSO’s search mech-anism mimics the social behavior of biological organisms such as a flockof birds. PSO can search a very large space of candidate solutions, whichreduces the chance of getting trapped at an unsatisfactory local optimum.For the parameter settings, the population size is set to 50 and weightingparameters of ω = 0.5, and c1 = c2 = 1.25 are used.

The covariance matrix adaptation strategy (CMA-ES) Hansen and Kern(2004); Loshchilov (2013); Auger and Hansen (2005) is a population-based

13

stochastic optimization algorithm. Unlike a genetic algorithm (GA), PSO,and other classic population-based stochastic search algorithms, candidatessolutions of CMA-ES are sampled from a probability distribution which areupdated iteratively. For CMA-ES, we use the settings from Hansen et al.Hansen and Kern (2004) (See Table 2).

Table 2: Strategy parameter values used in CMA-ES.

Parameter Value

λ 4 + b3 ln(n)cµ bλ/2ccc

4n+4

cσµeff+2

n+µeff+3

dσ 1 + 2 max(

0,√

µeff−1n+1

− 1)

+ cσ

µcov µeff

ccov1

µcov

2(n+√

2)2 +(

1− 1µcov

)min

(1, 2µeff−1

(n+2)2+µeff

)In our study, we use the maximum number of function evaluations (sim-

ulation runs) as the optimization stopping criterion. PSO and CMA-ES arestochastic algorithms and the result of each trial is different. Thus, in orderto assess the overall performance of these algorithms, we run each of them10 times for every test example.

It is also worth mentioning that, GPS and CMA-ES need an initial pointto start the optimization processes. PSO can generate a sequence of initialpoints automatically, so it can be run without specifying an initial point.For MCS, the initial point is determined by the initialization list. In ourthree examples, we use a physically reasonable initial point in GPS, PSO,and CMA-ES.

4.2. Simultaneous and sequential procedures for the joint optimization prob-lem

For the joint well placement and control optimization problem, we con-sider both a simultaneous procedure and a sequential procedure. The simul-taneous procedure optimizes well locations and controls simultaneously. Tosolve the joint problem with a simultaneous approach we consider MCS (with5 different configurations), GPS, PSO, and CMA-ES.

14

The sequential procedure divides the optimization process into a wellplacement optimization stage and a well control optimization stage. Eachstage is an independent optimization problem and can be optimized usingsame or different algorithms. We label the approaches used for the sequentialprocedure in the form Algorithm1-Algorithm2, where Algorithm1 denotes thealgorithm used for the well placement optimization stage and Algorithm2denotes the algorithm used for the well control optimization stage. Manysuch combinations are possible.

The combinations we considered in this paper are divided into 3 groups.The first group includes MCS-MCS, GPS-GPS, PSO-PSO, and CMA-ES-CMA-ES, which use the same algorithm in both the well placement stageand the well control stage. The second group includes MCS-GPS, MCS-PSO, and MCS-CMA-ES, use MCS for well placement stage. The thirdgroup GPS-MCS, PSO-MCS, and CMA-ES-MCS uses MCS for well controlstage.

4.3. Example 1, well placement optimization

The first example uses the PUNQ-S3 model, which is a small reservoirmodel based on an actual North Sea reservoir Gao et al. (2006). The modeluses 19 × 28 × 5 grid blocks with ∆x = ∆y = 180m and 1761 active gridblocks. The simulation model involves a three-phase gas-oil-water flow. Thefield initially contains 6 production wells and no injection wells due to thestrong aquifer. Fig. 4 shows the depth of the top face and permeability field,together with the initial well locations of PUNQ-S3 model.

The reservoir simulation time is 20 years, the bottom hole pressure of eachwell is fixed at 200 bar. We seek to optimize the well locations of all 6 wells.Every well has (xi, yi) two variables which gives a total of 12 optimizationvariables. Only bound constraints are considered in this example. The xbounds are 1 ≤ x ≤ 19, and y bounds are 1 ≤ y ≤ 28 for all 6 wells.

The objective function we use is the net present value (NPV), the detaileddescription for the calculation of NPV is given in Section 2. The productionparameters (the flow rates of the gas, oil, water produced and water injected)are calculated by Eclipse GeoQuest (2014), a commercial reservoir simulationsoftware from Schlumberger Ltd. The parameters used to calculate NPV aregiven in Table 3.

15

(a) Tops (b) Permeability

Figure 4: Properties and initial well locations for the PUNQ-S3 model used in Example1.

4.4. Example 2, well control optimization

This example from Oliveira and Reynolds (2014) uses a single-layer reser-voir model with 51 × 51 uniform grid blocks with ∆x = ∆y = 10m and∆z = 5m. The model consists of four production wells and one injectionwell. The wells form a five-spot well pattern. We consider an oil-water twophase flow in this model. The permeability field and well placements areshown in Fig. 5. There are two high permeability zones and two low per-meability zones in the model. The permeabilities are 1000mD and 100mD,respectively. Detailed reservoir information is given in Table 4.

The reservoir lifetime is set to 720 days. With a fixed injection rateof 240m3/d for well INJ-01, we seek to optimize the liquid rates of fourproduction wells. Two variations of this optimization problem are considered.

• Case 1: each well is produced under a liquid rate throughout its lifetime.This gives 4 optimization variables in total.

• Case 2: the liquid rate for each well is updated every 90 days (8 control

16

Table 3: Economic parameters used for Example 1.

Parameter Value

Gas revenue USD 0.5/m3

Oil revenue USD 500.0/m3

Water-production cost USD 80.0/m3

Annual discount rate 0

Figure 5: Permeanbility field (mD) for the five-spot model used in Example 2.

periods). This gives 32 optimization variables in total.

The objective function we use for this example is NPV and the corre-sponding economic parameters are same as in Example 1 and are given inTable 3. Only bound constraints are considered and the detailed optimizationparameters are given in Table 5.

4.5. Example 3, joint well placement and control optimization

This example use a 2D reservoir model with the permeability and porosityfields taken from the third layer of the SPE10 benchmark model Humphrieset al. (2013). It consists of 60× 50 grid cells and the size of each grid cell is32 × 32 × 10m. We consider an oil-water two phase flow in this model andthe initial oil saturation is 0.8. Fig. 6 shows the permeability and porosityfields of the model.

17

Table 4: Reservoir parameters used in Example 2.

Parameter Value

Reservoir grid 51× 51× 1Grid size 10m× 10m× 5mPorosity 0.2

Net-to-gross ratio 0.2Initial oil saturation 0.8

Initial pressure 200 barOil viscosity 0.42 mPa · s

Water viscosity 1.7 mPa · s

Table 5: Optimization parameters used in Example 2.

Parameter Case 1 Case 2

Variables 4 32Initial rate, m3/d 20 40

Minimum rate, m3/d 0 0Maximum rate (PRO-01, PRO-03), m3/d 40 40Maximum rate (PRO-02, PRO-04), m3/d 80 80

The optimization problem is to place four wells in the reservoir, includ-ing two production (P1, P2) and two injection wells (I1, I2). All wells arecontrolled via BHP that is updated every two years. The production periodfor this example is 10 years. Thus, there are two location variables and fivecontrol variables per well and 28 variables in total. Only bound constraintsare considered in this example. The economic and optimization parametersare summarized in Table 6 and Table 7 respectively. Both the simultaneousprocedure and the sequential procedure are used for this example.

5. Results and discussion

5.1. Example 1

The results of Example 1 are shown in Table 8. In this table, the ultimateNPV after 600 simulation runs for each algorithm is given. Moreover, forPSO and CMA-ES, the maximum, minimum, mean, median, and standarddeviation of NPV are given. From the table we can see that GPS obtains

18

0 10 20 30 40

0

10

20

30

40

50

−1

0

1

2

3

4

(a) Permeability(log10)

0 10 20 30 40

0

10

20

30

40

50

0.0

0.1

0.2

0.3

0.4

(b) Porosity

Figure 6: Permeability and porosity fields of SPE10 model used in Example 3.

Table 6: Economic parameters used in Example 3.

Parameter Value

Oil revenue USD 503.2/m3

Water-production cost USD 75.5/m3

Water-injection cost USD 50.3/m3

Annual discount rate 0

the highest NPV value after the 600 simulation runs. Though the maximumNPV for PSO and CMA-ES are slightly higher than MCS, MCS generallyperforms better than PSO and CMA-ES when comparing the mean and themedian values of NPV. MCS-5, the variant without local search obtains thelowest NPV.

Plots of the NPV for the four algorithms versus the number of simulationruns are shown in Fig. 7. Note that for PSO and CMA-ES, 10 trials areperformed and the solid lines are the median NPV over all 10 trials. SinceGPS and MCS-1 (with default settings) are deterministic algorithms, onlyone trial is performed.

Form Fig. 7 we can see that GPS, PSO and CMA-ES obtain a higher NPVthan MCS at the beginning. This is because for GPS, PSO, and CMA-ES,the initial well locations use reasonable values given by industry–locationsused in actual production Gao et al. (2006). MCS, on the other hand, startsthe optimization without using the initial points. After no more than 200

19

Table 7: Optimization parameters used in Example 3.

Parameter P1 P2 I1 I2

Initial location (60,25) (1,25) (30,1) (30,50)Initial BHP, bar 175 175 362.5 362.5

Minimum BHP, bar 100 100 275 275Maximum BHP, bar 250 250 450 450

Table 8: Results for Example 1. Values shown are NPV in $× 109 USD.

(a) Deterministic algorithms (MCS, GPS)

Algorithm NPVMCS-1 2.10MCS-2 2.16MCS-3 2.21MCS-4 2.15MCS-5 1.87GPS 2.32

(b) Stochastic algorithms (PSO, CMA-ES)

Algorithm Trials Max Min Mean Median Std.PSO 10 2.18 2.00 2.07 2.04 0.07CMA-ES 10 2.22 1.91 2.03 2.00 0.12

simulation runs, MCS obtains a higher NPV than PSO and CMA-ES. InExample 1, MCS showed a good performance searching over the global searchspace and converges to a relatively high objective function value fast in spiteof the poorer initial solution.

Since PSO and CMA-ES are stochastic algorithms, the performance isdifferent for each trial. Fig. 8 shows the range of NPV amongst the trialsfor PSO and CMA-ES. In this figure, the areas between the maximum andminimum NPV are filled with orange for PSO and with purple for CMA-ES.It is clear that the NPV obtained by PSO and CMA-ES has a high variationfor this example. This suggests that when solving this problem by PSO orCMA-ES, a single trial has a high risk to obtain an unsatisfactory NPV.

The five different MCS configurations, each having different parametersettings, are also tested with this example. These algorithms are divided

20

0 100 200 300 400 500 600

Simulation runs

1.0

1.2

1.4

1.6

1.8

2.0

2.2

2.4

NP

V,×

10

9U

SD

GPSPSOCMA-ESMCS

Figure 7: Optimization performance for Example 1 using GPS, PSO, CMA-ES and MCS(MCS-1). The solid lines are median NPV over all 10 runs of PSO and CMA-ES.

0 100 200 300 400 500 600

Simulation runs

1.6

1.7

1.8

1.9

2.0

2.1

2.2

2.3

2.4

NP

V,×

10

9U

SD

PSO

(a) PSO

0 100 200 300 400 500 600

Simulation runs

1.6

1.7

1.8

1.9

2.0

2.1

2.2

2.3

2.4

NP

V,×

10

9U

SD

CMA-ES

(b) CMA-ES

Figure 8: The median value of NPV and its corresponding standard deviation for Example1 using PSO and CMA-ES. The variability amongst the 10 runs for each algorithm is alsoshown.

into 3 groups. The first group (MCS-1, MCS-2, and MCS-3) use differentinitialization lists. This allows us to check the impact of initialization list.The second group (MCS-1, MCS-4) use a different maximum number of levelssmax. The higher smax, the better the global search ability. The third group(MCS-1, MCS-5) is used to analyse the role of local search in MCS. Detailedresults for these three groups are shown in Fig. 9.

Fig. 9(a) shows the performance of MCS with different initializationlists. Initialization list III (MCS-3) ultimately achieves the highest NPV,followed by initialization list II (MCS-2), and then initialization list I (MCS-1). Comparing the two pre-set initialization lists (MCS-1 and MCS-2), MCS-2 converges faster than MCS-1. This indicates that the uniformly spaced

21

0 100 200 300 400 500 600

Simulation runs

1.0

1.2

1.4

1.6

1.8

2.0

2.2

2.4N

PV,×

10

9U

SD

MCS-1MCS-2MCS-3

(a) Initialization list

0 100 200 300 400 500 600

Simulation runs

1.0

1.2

1.4

1.6

1.8

2.0

2.2

2.4

NP

V,×

10

9U

SD

MCS-1MCS-4

(b) Levels

0 100 200 300 400 500 600

Simulation runs

1.0

1.2

1.4

1.6

1.8

2.0

2.2

2.4

NP

V,×

10

9U

SD

MCS-1MCS-5

(c) Local search

Figure 9: Performance of MCS with different configuration settings for Example 1. Threeimportant parameters in MCS: initialization list, number of levels, and local search.

initialization list, not containing any boundary points, is more suitable thanthe one with boundary points. To explore this, we normalize the search spaceto the [0, 1]-interval and map the initialization lists and the global optima tothe normalized search space in Fig. 10. It is clear that the optimal solutionis aligned better with initialization list II (MCS-2) which explains the betterperformance of MCS-2 for this problem. The optimal solution is not knowna priori in most cases, so although a suitable initialization list can improvethe performance of MCS, it is difficult to choose between MCS-1 and MCS-2a priori.

MCS-3 generates the initialization list by using a line search. This takesa few additional simulation runs before the splitting and local search steps,so it converges a little slower than MCS-1 and MCS-2 at the beginning.Ultimately, however, MCS-3 shows faster convergence than MCS-1 and MCS-

22

1 2 3 4 5 6 7 8 9 10 11 12

Optimization variables

0.0

0.2

0.4

0.6

0.8

1.0

Valu

es

viui+5vi

6

ui+vi2

5ui+vi6

ui

optima

Figure 10: Normalized boundary, initialization lists, and optimal solution for Example 1.

2, and finally obtains a higher NPV.Fig. 9(b) shows the performance of MCS with different specified max-

imum levels smax. MCS with a larger maximum level, namely smax = 10nultimately obtains a higher NPV than smax = 5n+ 10.

The performance of MCS with and without local search are shown inFig. 9(c). The convergence speed of MCS without local search is severelydecreased and the maximum NPV found is reduced by 11%.

5.2. Example 2

Example 2 includes 2 cases. The results are shown in Table 9 and Table10. Case 1 runs 400 simulations to optimize 4 variables and Case 2 runs 3200simulations to optimize 32 variables. From Table 9 we can see that for Case1, all algorithms GPS, PSO, CMA-ES and MCS (except for configurationMCS-5) are able to obtain a high NPV value at the end of the optimization,and the ultimate difference between the algorithms is small. The mean andmedian NPV found by PSO is slightly smaller than the other algorithms.For Case 2, similar conclusions can be obtained from Table 10. After 3200simulation runs, GPS obtains the highest NPV. CMA-ES and MCS (againexcept for configuration MCS-5) are in the middle, while PSO performs theworst.

Plots of the NPV of the four algorithms versus the number of simulationruns are shown in Fig. 11. As in Example 1, 10 trials are performed for PSOand CMA-ES, and the solid lines are the median NPV over all 10 trials ofthese two algorithms.

The initial control used with GPS, PSO, and CMA-ES is the middle valuebetween the lower and upper bounds. Coincidentally, MCS with default

23

Table 9: Results for Case 1 of Example 2. Values shown are NPV in $× 106 USD.





settings also uses the middle value as its start point. So for this example,the 4 algorithms have the same start point.

0 50 100 150 200

Simulation runs

2.5

3.0

3.5

4.0

4.5

5.0

5.5

NP

V,×

10

6U

SD

GPSPSOCMA-ESMCS

(a) Case 1

0 500 1000 1500 2000 2500 3000

Simulation runs

2.0

4.0

6.0

8.0

10.0

12.0

14.0

NP

V,×

10

6U

SD

GPSPSOCMA-ESMCS

(b) Case 2

Figure 11: Optimization performance for Example 2 using GPS, PSO, CMA-ES and MCS.The solid lines are median NPV over all 10 runs of PSO and CMA-ES.

From Fig. 11 we can see that, although the final NPV obtained by MCSis not the highest over all algorithms tested, MCS outperforms when thenumber of simulation runs is limited. When the number of simulation runsis limited to 15% of the final number of simulation runs (60 simulation runsfor Case 1 and 480 simulation runs for Case 2), the NPV obtained by each

24

Table 10: Results for Case 2 of Example 2. Values shown are NPV, ×106 USD.





algorithm is given in Table 11. Note that in this table we use the medianNPV of 10 trials for PSO and CMA-ES. We use the median instead of themean because it is less sensitive to outliers in the data Huber (1981). Whenthe total number of simulation runs is limited, MCS showed significant ad-vantages over PSO, GPS, and CMA-ES. Again MCS-5 provides poor results– showing the importance of the local search feature within MCS. This tableshows the potential of MCS with a low computational budget.

Table 11: Results of Example 2 with a limited simulation runs. Values shown are NPV,×106 USD obtained after 15% of maximum simulation runs.

(a) Case 1

Algorithm NPVMCS-1 5.28MCS-2 5.28MCS-3 5.26MCS-4 5.28MCS-5 3.88GPS 5.01PSO 4.79CMA-ES 4.92

(b) Case 2

Algorithm NPVMCS-1 11.53MCS-2 11.83MCS-3 11.96MCS-4 10.63MCS-5 9.80GPS 8.80PSO 9.67CMA-ES 11.52

25

0 50 100 150 200

Simulation runs

2.5

3.0

3.5

4.0

4.5

5.0

5.5N

PV,×

10

6U

SD

PSO

(a) Case 1, PSO

0 50 100 150 200

Simulation runs

2.5

3.0

3.5

4.0

4.5

5.0

5.5

NP

V,×

10

6U

SD

CMA-ES

(b) Case 1, CMA-ES

0 500 1000 1500 2000 2500 3000

Simulation runs

2.0

4.0

6.0

8.0

10.0

12.0

14.0

NP

V,×

10

6U

SD

PSO

(c) Case 2, PSO

0 500 1000 1500 2000 2500 3000

Simulation runs

2.0

4.0

6.0

8.0

10.0

12.0

14.0

NP

V,×

10

6U

SD

CMA-ES

(d) Case 2, CMA-ES

Figure 12: The median value of NPV and its corresponding standard deviation for twocases of Example 2 using PSO and CMA-ES. The variability amongst the 10 runs for eachalgorithm is also shown.

As we progress from Case 1 to Case 2, the number of optimization vari-ables increases from 4 to 32. The performance of GPS with a low number ofsimulation runs decreases. In Case 2, the maximum NPV found by GPS isless than the other 3 algorithms when the number of simulation runs is lim-ited to 1000. After 1000 simulation runs, GPS is able to find a higher NPVthan PSO. The early stage of the optimization process mainly reflects theglobal search phase, and the later stage of the optimization process includesthe effect of the local search phase. In Case 2, it is clear that PSO performsbetter than GPS at an early stage, but GPS outperforms later. Overall,MCS, which includes both a global search phase and a local search phase,showed a better convergence rate than GPS and PSO.

Fig. 12 shows the range of NPV between the trials for PSO and CMA-ES

26

for Example 2. In this figure, the areas between the maximum and minimumNPV are filled with orange for PSO and with purple for CMA-ES. From thisfigure we can see that for Case 1, the range of NPV is large initially and thenthe range decreases for both PSO and CMA-ES. CMA-ES has a small vari-ability near convergence. For Case 2, with a larger number of optimizationvariables than Case 1, the range of NPV does not decrease for PSO. Eachtrial falls into a local optima and has a difficult time to escape. The rangefor CMA-ES decreases to near zero. This indicates that for PSO and CMA-ES, a large computational budget can decrease the performance variabilityfor this example. Compared to CMA-ES, PSO more easily falls into a localoptima for problems with a large number of optimization variables.

As in Example 1, we tested 5 different MCS configurations and dividedthem into 3 groups to do further analysis. The results are shown in Fig. 13and Fig. 14.

0 50 100 150 200

Simulation runs

2.5

3.0

3.5

4.0

4.5

5.0

5.5

NP

V,×

10

6U

SD

MCS-1MCS-2MCS-3


0 50 100 150 200

Simulation runs

2.5

3.0

3.5

4.0

4.5

5.0

5.5

NP

V,×

10

6U

SD

MCS-1MCS-4

(b) Levels

0 50 100 150 200

Simulation runs

2.5

3.0

3.5

4.0

4.5

5.0

5.5

NP

V,×

10

6U

SD

MCS-1MCS-5

(c) Local search

Figure 13: Performance of MCS with different configuration settings for Case 1 of Example2.

27

0 500 1000 1500

Simulation runs

2.0

4.0

6.0

8.0

10.0

12.0

14.0N

PV,×

10

6U

SD

MCS-1MCS-2MCS-3


0 500 1000 1500

Simulation runs

2.0

4.0

6.0

8.0

10.0

12.0

14.0

NP

V,×

10

6U

SD

MCS-1MCS-4

(b) Levels

0 500 1000 1500

Simulation runs

2.0

4.0

6.0

8.0

10.0

12.0

14.0

NP

V,×

10

6U

SD

MCS-1MCS-5

(c) Local search

Figure 14: Performance of MCS with different configuration settings for Case 2 of Example2.

Fig. 13(a) and Fig. 14(a) compare the performance of MCS with differentinitialization lists for the two cases of Example 2. For Case 1, the convergencerate of MCS-2 is the fastest, followed by MCS-3 and MCS-1. MCS-2 andMCS-3 ultimately obtain the highest NPV.

For Case 2, MCS-1 and MCS-2 give a similar convergence rate at theearly stage, then MCS-1 falls behind MCS-2. MCS-3 shows a very slow rateof convergence at early stage of the optimization process, but it obtains thehighest NPV finally. MCS-3 generates the initialization list by using a linesearch. This takes a few additional simulation runs before the splitting andlocal search steps. Which explains the slow convergence initially.

The effect of the maximum number of levels is shown in Fig. 13(b) andFig. 14(b). For Case 1, using smax = 5n + 10 (MCS-1) performs similarlyto using smax = 10n (MCS-4). For Case 2, which has 32 variables, usingsmax = 5n + 10 (MCS-1) converges slightly faster than using smax = 10n

28

(MCS-4), and finally obtains a higher NPV. This indicates that a smallnumber of levels is enough for these cases.

Fig. 13(c) and Fig. 14(c) shows that local search plays an important rolein MCS, without it the convergence speed decreases significantly.

5.3. Example 3

5.3.1. Simultaneous procedure

The simultaneous procedure optimizes over all well locations and controlparameters simultaneously. For this problem, we optimize the locations andcontrol parameters of the 4 wells. Each well has 2 location variables and 5control variables, Thus there are 28 variables in total. Given this problem’scomplexity, we set the maximum number of simulation runs for this examplewhen using the simultaneous procedure, to be 10000. The maximum, mini-mum, mean, median, and standard deviation of NPV for each algorithm isgiven in Table 12. From the table we can see that MCS obtains the highestNPV value after 10000 simulation runs. Only the configuration MCS-2 per-forms unsatisfactorily amongst all five MCS algorithms. The average NPVfor PSO and CMA-ES are in the middle, while GPS performs the worst.

Table 12: Results of simultaneous procedure for Example 3. Values shown are NPV in$× 108 USD.





Plots of the NPV of the four algorithms versus the number of simulationruns are shown in Fig. 15. As with examples 1 and 2, the solid lines for PSOand CMA-ES are the median NPV over all 10 trials.

29

0 2000 4000 6000 8000 10000

Simulation runs

2.0

4.0

6.0

8.0

10.0

NP

V,×

10

8U

SD

GPSPSOCMA-ESMCS

Figure 15: Optimization performance of simultaneous procedure for Example 3 using GPS,PSO, CMA-ES and MCS. The solid lines are median NPV over all 10 runs of PSO andCMA-ES.

0 2000 4000 6000 8000 10000

Simulation runs

2.0

4.0

6.0

8.0

10.0

NP

V,×

10

8U

SD

PSO

(a) PSO

0 2000 4000 6000 8000 10000

Simulation runs

2.0

4.0

6.0

8.0

10.0N

PV,×

10

8U

SD

CMA-ES

(b) CMA-ES

Figure 16: The median value of NPV and its corresponding standard deviation for Example3 using PSO and CMA-ES. 10 runs are used for each algorithm.

From Fig. 15 we can see that for this example, MCS converges fastestcompared to the other 3 algorithms, followed by CMA-ES, PSO, and GPS inthat order. Unlike Example 1 and 2, the convergence speed of GPS is slowestamong all algorithms. The NPV of GPS has a jump at about 4000 simulationruns. It appears that at this point GPS jumps from a local optima.

Fig. 16 shows the range of NPV between the trials for PSO and CMA-ES.In this figure, the areas between the maximum and minimum NPV are filledwith orange for PSO and with purple for CMA-ES. It is clear that the NPVobtained by PSO and CMA-ES has a high variation for this example.

As with Examples 1 and 2, we use five MCS configurations, and the resultsare compared in 3 groups in Fig. 18. Fig.18(a) shows the performance of

30

MCS with different initialization lists. We can see that for this example, theinitialization list without boundary points (MCS-2) performs unsatisfactorilyboth in terms of the convergence rate and the final NPV compared withthe other two configurations. This is because the optimal solution for thisexample lies near the boundary, as shown in Fig. 17. Using line search togenerate the initialization list (MCS-3) ultimately obtains the highest NPVfor this example.

1 3 5 7 9 11 13 15 17 19 21 23 25 27

Optimization variables

0.0

0.2

0.4

0.6

0.8

1.0

Valu

es

viui+5vi

6

ui+vi2

5ui+vi6

ui

optima

Figure 17: Normalized boundary, initialization lists, and the optimal solution for Example3.

Fig. 18(b) shows the performance of MCS with different numbers ofmaximum levels. Using smax = 10n (MCS-4) and smax = 5n + 10 (MCS-1)performs similarly at an early stage, and using a higher number of levelsoutperforms after 6000 simulation runs.

The performance of MCS with and without local search are shown inFig. 18(c). MCS without local search (MCS-5) performs worse than thedefault algorithm (MCS-1) for a quite long time. But when the number ofsimulation runs reaches 6000, the NPV of MCS-5 jumps to a value higherthan for MCS-1. With enough computation budget and a large smax, MCSwithout the help of a local search can also obtain a high NPV.

5.3.2. Sequential procedure

The sequential procedure decouples the joint problem to two separatesubproblems. For the well placement optimization subproblem, we optimizethe locations of the 4 wells under the given control scheme. This gives 8optimization variables. For the well control optimization subproblem, weoptimize the well controls of the 4 wells with the given well locations. Eachwell has 5 control steps, this gives 20 optimization variables in total. The

31

0 2000 4000 6000 8000 10000

Simulation runs

2.0

4.0

6.0

8.0

10.0N

PV,×

10

8U

SD

MCS-1MCS-2MCS-3


0 2000 4000 6000 8000 10000

Simulation runs

2.0

4.0

6.0

8.0

10.0

NP

V,×

10

8U

SD

MCS-1MCS-4

(b) Levels

0 2000 4000 6000 8000 10000

Simulation runs

2.0

4.0

6.0

8.0

10.0

NP

V,×

10

8U

SD

MCS-1MCS-5

(c) Local search

Figure 18: Performance of MCS with different configuration settings for Example 3.

maximum number of simulation runs for each well placement optimizationstage is 60, while for each well control optimization stage the maximumnumber of simulation runs is set to 140. And the maximum number ofsimulation runs for the problem in total is 5000. Based on the results of theprevious section we use MCS with configuration 1 in the sequential procedure.

The maximum, minimum, mean, median, and standard deviation of NPVfor each algorithm combination is given in Table 13. Plots of the NPV ofthe approaches versus the number of simulation runs are shown in Fig. 19.From Table 13 and Fig. 19 we can see that MCS-MCS converges fastestcompared to the other combinations and obtained the highest NPV value atthe end of the optimization. GPS-MCS converges slowly at the early stage,but it ultimately obtained the second highest NPV. The combinations whichcontain stochastic algorithms, especially CMA-ES, performs unsatisfactorilyfor this example.

32

We also compare the optimal NPV obtained using the simultaneous andthe sequential procedures, the results are shown in the beanplots in Fig.20. A beanplot Kampstra (2008) promotes visual comparison of univariatedata between groups. In a beanplot, the individual observations are shownas small lines in a one-dimensional scatter plot. In addition, the estimateddensity of the distributions is visible and the mean (bold line) and median(marker ‘+’) are shown.

From Fig. 20 we can see that, with the simultaneous procedure, thefinal NPV values obtained by all algorithms are less than 109USD. Withthe sequential procedure, MCS-MCS, GPS-MCS, and MCS-PSO can obtainNPV values higher than 109USD. The optimal well placement and the finaloil saturation distribution for the simultaneous and sequential procedure areshown in Fig. 21. The corresponding optimal controls of each well are givenin Table 14. The optimal well locations obtained by the simultaneous proce-dure are significantly different from the locations obtained by the sequentialprocedure. From the final oil saturation distribution, we can see that, thelocations obtained by the sequential procedure gives a larger sweep area. Theoptimal controls obtained by the simultaneous procedure are similar.

In theory, for a joint well placement and control optimization problem,the simultaneous procedure can find the global optima but this is not guar-anteed for the sequential procedure since the optimal location of each welldepends on how the well is operated and vice-versa. The simultaneous proce-dure, with a larger number of optimization variables, makes the joint problemmore difficult. It requires a higher computational budget and has a higherrisk of falling into a local optima and achieving a suboptimal solution, espe-cially for a larger scale problem. The sequential procedure, decouples a hardjoint problem into two easier subproblems, and hopes to approach the globaloptima iteratively This may shows better performance than the simultane-ous procedure. In general, the sequential procedure is worth considering inpractice.

5.4. Summary

Our test results show that MCS is strongly competitive with existingalgorithms for well placement, well control, and joint problems. In all 3examples, MCS offers good convergence speed, especially when the numberof simulation runs is limited. Based on the results of the examples, forplacement and control optimization we suggest a MCS configuration whichuses a line search to generate the initialization list. The number of levels

33

Table 13: Results of the sequential procedure for Example 3. Values shown are NPV in$× 108 USD.

Algorithm Trials Max Min Mean Median Std.MCS-MCS 1 11.48 11.48 11.48 11.48 0GPS-GPS 1 8.65 8.65 8.65 8.65 0PSO-PSO 5 8.11 6.64 7.54 7.89 0.79CMA-ES-CMA-ES

5 6.58 5.75 6.04 5.79 0.47

MCS-GPS 1 8.53 8.53 8.53 8.53 0MCS-PSO 5 10.78 9.55 10.03 9.76 0.66MCS-CMA-ES

5 9.99 9.48 9.69 9.59 0.27

GPS-MCS 1 10.11 10.11 10.11 10.11 0PSO-MCS 5 9.01 6.63 7.84 7.90 1.19CMA-ES-MCS

5 8.98 5.80 7.42 7.47 1.59

smax = 5n+10 is enough for most problems but a higher smax should be usedfor some difficult problems. Local search is an important part of MCS, andis highly recommended.

6. Concluding Remarks

In this paper, we applied the multilevel coordinate search algorithm forthree typical oil field development optimization problems. The problemsinclude well placement optimization, well control optimization, and joint op-timization of well placement and control. The performance of MCS has beencompared with generalized pattern search, particle swarm optimization, andcovariance matrix adaptation evolution strategy through several case stud-ies including both synthetic and real reservoirs. The results presented heredemonstrate that the MCS algorithm is strongly competitive, and outper-forms the other mentioned algorithms in most cases, especially for the jointoptimization problem. MCS has significant advantages in solving optimiza-tion problems with a limited number of simulation runs.

For joint well placement and well control optimization problem, both thesimultaneous procedure and the sequential procedure were considered. In ourexample, the sequential procedure finds the best solution. Although the si-

34

0 1000 2000 3000 4000 5000

Simulation runs

2.0

4.0

6.0

8.0

10.0

12.0N

PV,×

10

8U

SD

GPS-GPSPSO-PSOCMAES-CMAESMCS-MCS

(a)

0 1000 2000 3000 4000 5000

Simulation runs

2.0

4.0

6.0

8.0

10.0

12.0

NP

V,×

10

8U

SD

MCS-GPSMCS-PSOMCS-CMAES

(b)

0 1000 2000 3000 4000 5000

Simulation runs

2.0

4.0

6.0

8.0

10.0

12.0

NP

V,×

10

8U

SD

GPS-MCSPSO-MCSCMAES-MCS

(c)

Figure 19: Optimization performance of the sequential procedure for Example 3 usingdifferent algorithm combinations.

multaneous procedure can theoretically obtain the global optima, the sequen-tial procedure is worth considering in practice. The sequential procedure de-couples a difficult joint problem to two easier separate subproblem, decreasesthe number of optimization variables, make the problem easier to solve anddecreases the risk of the algorithm falling into a local optima. Among allalgorithm combinations considered in this paper, MCS-MCS showed bestperformance both in terms of convergence speed and final NPV value in thesequential procedure.

MCS has shown its potential in our work, but more research is needed.Future work includes applying the MCS algorithm to realistic large-scale oilfield cases, to linearly and nonlinearly constrained problems, and exploringthe performance of MCS while using a Hook-Jeeves search instead of thedefault local search procedure.

35

MC

S-1

MC

S-2

MC

S-3

MC

S-4

MC

S-5

GP

S

PS

O

CM

AE

S

MC

S-M

CS

GP

S-G

PS

PS

O-P

SO

CM

A-E

S-C

MA

-ES

MC

S-G

PS

MC

S-P

SO

MC

S-C

MA

-ES

GP

S-M

CS

PS

O-M

CS

CM

A-E

S-M

CS

4

5

6

7

8

9

10

11

12

13N

PV,×

10

8U

SD

Figure 20: Beanplot of the final NPV values for the simultaneous and the sequentialprocedure for Example 3. The left side of the red dotted vertical line gives the resultsobtained by algorithms with the simultaneous procedure, and the right side gives theresults obtained by algorithm combinations with the sequential procedure. The individualhorizontal fine line show the NPV obtained by each trial. The horizontal blue bold lineand the red marker ‘+’ denote the mean and median of all trials, respectively.

Acknowledgments

The authors acknowledge funding from the Natural Sciences and Engi-neering Research Council of Canada (NSERC) Discovery Grant Program,the National Science and Technology Major Project of the Ministry of Sci-ence and Technology of China (2011ZX05011-002), and the program of ChinaScholarships Council (No. 201406450017).

References

References

AlQahtani, G. D., Alzahabi, A., Spinner, T., Soliman, M. Y., 2014. A compu-tational comparison between optimization techniques for wells placementproblem: Mathematical formulations, genetic algorithms and very fast sim-ulated annealing. Journal of Materials Science and Chemical Engineering02 (10), 59–73.

36

0 10 20 30 40 50

0

10

20

30

40

50

60

P1

P2

I1

I2

0.2

0.4

0.6

0.8

(a) Simultaneous procedure

0 10 20 30 40 50

0

10

20

30

40

50

60

P1

P2

I1

I2

0.2

0.4

0.6

0.8

(b) Sequential procedure

Figure 21: The optimal well placement and the final oil saturation distribution for thesimultaneous procedure and the sequential procedure for Example 3.

Asadollahi, M., Nvdal, G., Dadashpour, M., Kleppe, J., Feb. 2014. Produc-tion optimization using derivative free methods applied to Brugge fieldcase. Journal of Petroleum Science and Engineering 114, 22–37.

Audet, C., Dennis, J., Jan. 2002. Analysis of Generalized Pattern Searches.SIAM Journal on Optimization 13 (3), 889–903.

Auger, A., Hansen, N., Sep. 2005. A restart CMA evolution strategy withincreasing population size. In: The 2005 IEEE Congress on EvolutionaryComputation, 2005. Vol. 2. pp. 1769–1776.

Bellout, M. C., Ciaurri, D. E., Durlofsky, L. J., Foss, B., Kleppe, J., jul2012. Joint optimization of oil well placement and controls. ComputationalGeosciences 16 (4), 1061–1079.

Bouzarkouna, Z., Ding, D. Y., Auger, A., Jan. 2012. Well placement op-timization with the covariance matrix adaptation evolution strategy andmeta-models. Computational Geosciences 16 (1), 75–92.

Brouwer, D., Jansen, J., dec 2004. Dynamic optimization of waterfloodingwith smart wells using optimal control theory. SPE Journal 9 (04), 391–402.

37

Table 14: The optimal control of each well for the simultaneous procedure and the sequen-tial procedure for Example 3. Values shown are BHP in bar.

(a) Simultaneous procedure

Algorithm NPVP1 [100, 100, 100, 100, 100]P2 [100, 100, 100, 100, 100]I1 [450, 450, 450, 450, 450]I2 [450, 450, 450, 450, 450]

(b) Sequential procedure

Algorithm NPVP1 [103, 103, 103, 103, 103]P2 [103, 103, 103, 103, 103]I1 [450, 447, 447, 447, 450]I2 [447, 447, 447, 447, 447]

Ciaurri, D. E., Mukerji, T., Durlofsky, L. J., 2011. Derivative-Free Optimiza-tion for Oil Field Operations. In: Yang, X.-S., Koziel, S. (Eds.), Compu-tational Optimization and Applications in Engineering and Industry. No.359 in Studies in Computational Intelligence. Springer Berlin Heidelberg,pp. 19–55.

Forouzanfar, F., Poquioma, W. E., Reynolds, A. C., 2015. A covariancematrix adaptation algorithm for simultaneous estimation of optimal place-ment and control of production and water injection wells. In: SPE Reser-voir Simulation Symposium. Society of Petroleum Engineers (SPE).

Gao, G., Zafari, M., Reynolds, A. C., dec 2006. Quantifying uncertainty forthe PUNQ-s3 problem in a bayesian setting with RML and EnKF. SPEJournal 11 (04), 506–515.

GeoQuest, S., 2014. ECLIPSE reference manual. Schlumberger, Houston,Texas.

Hansen, N., Kern, S., Jan. 2004. Evaluating the CMA evolution strategy onmultimodal test functions. In: Yao, X., Burke, E. K., et al. (Eds.), ParallelProblem Solving from Nature - PPSN VIII. No. 3242 in Lecture Notes inComputer Science. Springer Berlin Heidelberg, pp. 282–291.

38

Huber, P. J., feb 1981. Robust Statistics. John Wiley & Sons, Inc.

Humphries, T., Haynes, R., feb 2015. Joint optimization of well placementand control for nonconventional well types. Journal of Petroleum Scienceand Engineering 126, 242–253.

Humphries, T. D., Haynes, R. D., James, L. A., Sep. 2013. Simultaneous andsequential approaches to joint optimization of well placement and control.Computational Geosciences 18 (3-4), 433–448.

Huyer, W., Neumaier, A., 1999. Global optimization by multilevel coordinatesearch. Journal of Global Optimization 14 (4), 331–355.

Isebor, O. J., 2009. Constrained production optimization with an emphasison derivative-free methods. Ph.D. thesis, Stanford University.

Isebor, O. J., Ciaurri, D. E., Durlofsky, L. J., oct 2014a. Generalized field-development optimization with derivative-free procedures. SPE Journal19 (05), 891–908.

Isebor, O. J., Durlofsky, L. J., Ciaurri, D. E., Aug. 2014b. A derivative-free methodology with local and global search for the constrained jointoptimization of well locations and controls. Computational Geosciences18 (3-4), 463–482.

Jones, D. R., Perttunen, C. D., Stuckman, B. E., Oct. 1993. Lipschitzian op-timization without the Lipschitz constant. Journal of Optimization Theoryand Applications 79 (1), 157–181.

Kampstra, P., 2008. Beanplot: A boxplot alternative for visual comparisonof distributions. Journal of Statistical Software 28 (1).

Kennedy, J., 2011. Particle Swarm Optimization. In: Sammut, C., Webb,G. I. (Eds.), Encyclopedia of Machine Learning. Springer Science + Busi-ness Media, pp. 760–766.

Knudsen, B. R., Foss, B., Nov. 2013. Shut-in based production optimizationof shale-gas systems. Computers & Chemical Engineering 58, 54–67.

Lambot, S., Javaux, M., Hupet, F., Vanclooster, M., nov 2002. A globalmultilevel coordinate search procedure for estimating the unsaturated soilhydraulic properties. Water Resources Research 38 (11), 6–15.

39

Li, L., Jafarpour, B., Mohammad-Khaninezhad, M. R., Nov. 2012. A simul-taneous perturbation stochastic approximation algorithm for coupled wellplacement and control optimization under geologic uncertainty. Computa-tional Geosciences 17 (1), 167–188.

Loshchilov, I., jun 2013. CMA-ES with restarts for solving CEC 2013 bench-mark problems. In: 2013 IEEE Congress on Evolutionary Computation.Institute of Electrical & Electronics Engineers (IEEE).

MATLAB, 2012. version 7.14.0 (R2012a). The MathWorks Inc., Natick, Mas-sachusetts.

Merlini Giuliani, C., Camponogara, E., Apr. 2015. Derivative-free methodsapplied to daily production optimization of gas-lifted oil fields. Computers& Chemical Engineering 75, 60–64.

Neumaier, A., 2008. Mcs: Global optimization by multilevel coordinatesearch. https://www.mat.univie.ac.at/~neum/software/mcs/.

Oliveira, D. F., Reynolds, A., oct 2014. An adaptive hierarchical multiscalealgorithm for estimation of optimal well controls. SPE Journal 19 (05),909–930.

Onwunalu, J. E., Durlofsky, L., sep 2011. A new well-pattern-optimizationprocedure for large-scale field development. SPE Journal 16 (03), 594–607.

Onwunalu, J. E., Durlofsky, L. J., 2009. Development and application of anew well pattern optimization algorithm for optimizing large scale field de-velopment. In: SPE Annual Technical Conference and Exhibition. Societyof Petroleum Engineers (SPE).

Posık, P., Huyer, W., Pal, L., dec 2012. A comparison of global search algo-rithms for continuous black box optimization. Evolutionary Computation20 (4), 509–541.

Rios, L. M., Sahinidis, N. V., Jul. 2013. Derivative-free optimization: a re-view of algorithms and comparison of software implementations. Journalof Global Optimization 56 (3), 1247–1293.

Sarma, P., Durlofsky, L. J., Aziz, K., Chen, W. H., Mar. 2006. Efficient real-time reservoir management using adjoint-based optimal control and modelupdating. Computational Geosciences 10 (1), 3–36.

40

https://www.mat.univie.ac.at/~neum/software/mcs/

Shakhsi-Niaei, M., Iranmanesh, S. H., Torabi, S. A., Jun. 2014. Optimal plan-ning of oil and gas development projects considering long-term productionand transmission. Computers & Chemical Engineering 65, 67–80.

Tavallali, M. S., Karimi, I. A., Teo, K. M., Baxendale, D., Ayatollahi, S.,Aug. 2013. Optimal producer well placement and production planning inan oil reservoir. Computers & Chemical Engineering 55, 109–125.

Torczon, V., Feb. 1997. On the Convergence of Pattern Search Algorithms.SIAM Journal on Optimization 7 (1), 1–25.

Vaz, A. I. F., Vicente, L. N., Oct. 2007. A particle swarm pattern searchmethod for bound constrained global optimization. Journal of Global Op-timization 39 (2), 197–219.

Volkov, O., Voskov, D., 2014. Effect of time stepping strategy on adjoint-based production optimization. In: ECMOR XIV - 14th European confer-ence on the mathematics of oil recovery. EAGE Publications.

Wang, C., Li, G., Reynolds, A. C., sep 2009. Production optimization inclosed-loop reservoir management. SPE Journal 14 (03), 506–523.

Yin, S., Cagan, J., 2000. An extended pattern search algorithm for three-dimensional component layout. Journal of Mechanical Design 122 (1), 102–108.

Zandvliet, M., Handels, M., van Essen, G., Brouwer, R., Jansen, J.-D., dec2008. Adjoint-based well-placement optimization under production con-straints. SPE Journal 13 (04), 392–399.

Zhou, K., Hou, J., Zhang, X., Du, Q., Kang, X., Jiang, S., Aug. 2013.Optimal control of polymer flooding based on simultaneous perturba-tion stochastic approximation method guided by finite difference gradient.Computers & Chemical Engineering 55, 40–49.

41

Documents

A Multilevel Coordinate Search Algorithm for Well ... · of most important tasks in solving well placement and control optimization problems. In this work, we introduce and apply