University of Waterloo

Faculty of Engineering

Global Optimization in Slope Analysis

by Simulated Annealing

Rocscience, Inc.

Toronto, Ontario

Prepared by

Su, Xiao

ID 20251755

2A Chemical EngineeringSeptember 1st , 2008

4105 Credit Pointe Dr, Mississauga, Ontario L5M 3K7

September 1st, 2008 Dr. Thomas Duever Chair of Chemical Engineering University of Waterloo Waterloo, Ontario N2L 3G1 Dear Dr Duever,

I am submitting this document, entitled “Global Optimization in Slope Analysis by Simulated Annealing” as my 2A Work Report for Rocscience, Inc. My project at the company investigated the efficiency of simulated annealing methods for finding the minima (critical failure surfaces and accompanying factors of safety) of slope stability equations. The report proposes Adaptive Simulated Annealing (ASA) as a powerful global optimization method for finding the critical failure surface.

During my 2A work term, I worked under the supervision of Dr. Brent Corkum, software development manager at Rocscience. I implemented the simulated annealing algorithm as a new optimization method in SLIDE 5.0, a slope stability analysis program developed by Rocscience. I researched the efficiency of different two SA techniques, namely the very fast simulated annealing (VFSA) and the adaptive simulated annealing (ASA). The multi-dimensional simplex method was also implemented for comparative purposes. In addition to learning how to program, I had the opportunity to study the theoretical and practical aspects of non-convex optimization, especially the performance of stochastic methods when used on multi-modal functions. I also learned soil mechanics and slope stability principles.

This report was written entirely by me and has not received any previous academic credit at Rocscience, University of Waterloo or any other institution. I would like to thank Dr Brent Corkum for his excellent supervision and Dr Reginald Hammah for invaluable references and discussion. I also thank Dr Jim Hazzard, Damir Valiulin and Jeremy Smith for their programming tips and continued help during the program development. Finally, I would like to thank Dr John Curran, Dr Thamer Yacoub, Igor Pashutinski and Ms Lesley-Ann Foulds for helpful discussions. Sincerely, Xiao Su ID 20251755

Contents

Table of Contents ii

Table of Figures iv

List of Tables iv

Summary vi

1 Introduction 1

1.1 Slope Stability Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 The Method of Slices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.3 The Need for Global Optimization . . . . . . . . . . . . . . . . . . . . . . . . 4

1.4 Simulated Annealing (SA) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.5 Improvements on Simulated Annealing . . . . . . . . . . . . . . . . . . . . . 6

2 Methods 8

2.1 Very Fast Simulated Annealing (VFSA) . . . . . . . . . . . . . . . . . . . . . 9

2.2 Re-annealing and Adaptive Simulated Annealing (ASA) . . . . . . . . . . . . 10

2.3 Transformation of Variables - ASAc and ASAt . . . . . . . . . . . . . . . . . 11

2.4 Simplex . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.5 Auto-Refine Search (ARS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

3 Results 18

3.1 Verification cases for SLIDE . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

3.2 Customer Cases for SLIDE . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

4 Discussion and Explanatory Examples 23

4.1 Method Comparison — Verification Cases . . . . . . . . . . . . . . . . . . . 23

4.2 Method Comparison — Customer Cases . . . . . . . . . . . . . . . . . . . . 25

4.3 Simplex, ARS and other Optimization . . . . . . . . . . . . . . . . . . . . . 30

4.4 Annealing — Practical Considerations . . . . . . . . . . . . . . . . . . . . . 31

4.5 Annealing — General Remarks . . . . . . . . . . . . . . . . . . . . . . . . . 32

ii

5 Conclusions 39

6 Recommendations 40

List of Symbols 41

References 43

iii

List of Figures

1 Flowchart for VFSA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

2 Flowchart for ASA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

3 Flowchart for Simplex . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

4 Grid-search performed for Verification Case #87 . . . . . . . . . . . . . . . . 24

5 Auto-Refine Search for Verification Case #87 . . . . . . . . . . . . . . . . . 24

6 Adaptive Simulated Annealing for Verification Case #87 . . . . . . . . . . . 25

7 Grid-Search for Customer Case #4 . . . . . . . . . . . . . . . . . . . . . . . 26

8 Auto-Refine Search for Customer Case #8 . . . . . . . . . . . . . . . . . . . 27

9 Adaptive Simulated Annealing for Customer Case #8 . . . . . . . . . . . . . 28

10 Grid-Search for Customer Case #15 . . . . . . . . . . . . . . . . . . . . . . . 28

11 Auto-Refine Search for Customer Case #15 . . . . . . . . . . . . . . . . . . 29

12 Adaptive Simulated Annealing for Customer Case #15 . . . . . . . . . . . . 29

13 ASAt for Customer Case #15 . . . . . . . . . . . . . . . . . . . . . . . . . . 30

14 Generation range based on Cauchy distribution for T = 10 . . . . . . . . . . 34

15 Generation range based on Cauchy distribution for T = 0.2 . . . . . . . . . . 34

16 Generation range based on Cauchy distribution for T = 0.01 . . . . . . . . . 35

17 Convergence of ASAc for Customer Case #4 . . . . . . . . . . . . . . . . . . 36

18 (a) A function landscape in which SA methods are inefficient; (b) a fractal

landscape in which SA methods have high performance (Sorkin, 1991) . . . . 37

19 Grid Search for Customer Case #17 . . . . . . . . . . . . . . . . . . . . . . . 37

20 Convergence of ASAc for Customer Case #17 . . . . . . . . . . . . . . . . . 38

List of Tables

1 Required variables for safety factor analysis . . . . . . . . . . . . . . . . . . 3

2 Safety factors found for customer cases. Labeled in red are the safety factors

not within 0.01 of the global minimum . . . . . . . . . . . . . . . . . . . . . 19

iv

3 Safety Factors found for Customer Cases. Global minimum found are labeled

in blue, whereas minima within 0.05 of global minimum are considered satis-

factory and are labeled in red. . . . . . . . . . . . . . . . . . . . . . . . . . . 21

4 Running time for customer cases. The running time for example 41 was not

considered due to the large computational effort required for all methods to

convert Hoek-Brown material properties. . . . . . . . . . . . . . . . . . . . . 22

5 Material properties for Verification Case #87 . . . . . . . . . . . . . . . . . . 23

6 Support Properties for Verification Case #87 . . . . . . . . . . . . . . . . . . 23

v

Summary

This work investigates the efficiency of simulated annealing (SA) when applied to slope

stability analysis. The aim is to find the most critical surface along which failure of a soil

or rock mass is likely to occur. This is achieved by minimizing a cost function known as

the factor of safety F(v), for a position parameter v. Traditionally this surface is found

by a variety of methods ranging from trial and error (which relies heavily on the engineer’s

experience) to automated search routines (mostly techniques of local search).

Simulated annealing (SA) is a method of global optimization, and was first developed

for solving combinatorial systems such as the traveling salesman problem and chip placement.

The original SA has recently been applied to slope stability with varying degrees of success.

The current work investigates the performance of state-of-art SA methods such as the

very fast simulated annealing (VFSA) and adaptive simulated annealing (ASA) for circular

slip surface search. The methods were implemented in C++ and incorporated into the

SLIDE 5.032 software developed by Rocscience, Inc. These methods rely on a probabilistic

random walk, coupled with a scheme for decreasing the annealing schedules exponentially.

ASA was originally implemented with centre-point radius as control variables (ASAc). It

was then found that the use of three points on the slip surface greatly increases the efficiency

of the search - and this new version of the method is referred to as ASAt.

The multi-dimensional downhill simplex was also implemented for comparison, and all

methods are compared with the Auto-Refine Search in SLIDE. The performance of both

VFSA and ASA were comparable with ARS. However, when centre-radius parameters are

used the search is exceedingly slow. On the other hand, ASAt is able to locate the global

minimum for all verification cases and customer cases tested, within tolerable speed.

It is concluded that adaptive simulated annealing offers a robust alternative to existing

optimization methods. Major advantages of ASA include reliability and ease of implemen-

tation. The transformation of variables proved successful, and should be implemented for

all circular search algorithms. It is recommended for simulated annealing to be extended to

general non-circular surfaces, and for the local minima to be recorded for current circular

search.

vi

1 Introduction

The analysis of the stability of slopes is essential to modern society, as it examines the safety

of major human constructions such as dams, embankments and structures built along the

sides of slope. At the core of slope analysis lies the challenge of finding the geological surface

with the greatest instability or representing the most probable region in which the mass of

soil or rock will slide. The potential instability is estimated by the factor of safety F , and

critical failure is the surface with the lowest F .

The problem then becomes finding the surface with position vector v(x1, x2, x3, . . . , xn)

which minimizes the safety factor of safety function F . For examples, the parameters x1

to xn can be the centre and radius of a circular surface or the vertices of a non-circular

surface. The challenge lies in that the cost function is often highly multi-modal, and for

some slopes the function domain is strongly discontinuous. Most optimization methods,

especially gradient-based, tend to get trapped in local minima. Simulated annealing (SA),

on the other hand, is a global optimization method that allows up-hill as well as down-hill

moves, being capable of escaping the local minima.

In the current section, some basic slope stability concepts are briefly reviewed. The

original simulated annealing is then presented, and its major drawbacks are discussed, as

well as factors that led to the development of faster annealing algorithms.

1.1 Slope Stability Concepts

The factor of safety F is defined as the ratio of the total shear strength (S) available along

a slip surface to the shear stress (Sm) that is actually mobilized along the surface due to

actions of external loads:

F =S

Sm. (1)

Many methods have been developed to compute factor of safety, based on moment

and/or force equilibrium. Most of them belong to the well established class of limit equilib-

rium methods (LEM). Even though the LEMs do not consider stress-strain relationships, it

provides an accurate estimate of the factor of safety without the need for extensive knowledge

of initial conditions (Cheng et al., 2007a).

1

The LEMs generally result in a problem formulation that is a statically indeterminate.

In order to obtain a solution, different assumptions can be made on the distribution of inter-

nal forces. The quality of the solution is closely tied with the reality of these approximations.

Other techniques exist for computing factors of safety such as strength reduction methods

(SRM). These methods do not require assumptions on inter-slice shear force distributions

and employ finite element techniques (Abramson et al., 2002). However, the SRM have dis-

advantages which include long solution time and the need for detailed knowledge of boundary

conditions, the latter being unknown in many cases.

The flexibility and speed of LEMs lies in the method of slices, in which the earth mass

within the trial surface is divided into vertical slices and at each slice, force and moment

equilibrium resultants must be zero for static equilibrium (Abramson et al., 2002). The factor

of safety at each slice is taken as the factor of safety F of the slope. It results in a non-linear

equation with F as a root, and must be found through iteration. Often, the procedures

for solving the factor of safety equation do not converge (the solution “blows up”), due to

inappropriate force assumptions or negative friction values (Duncan and Wright, 2005).

1.2 The Method of Slices

The method of slices is used by most computer programs, as it can readily accommodate

complex slope geometries, variable soil conditions and the influence of external boundary

loads (Abramson et al., 2002). The method divides the slope geometry into n vertical slices,

creating 6n− 2 unknowns (Abramson et al., 2002), as summarized in Table 1.

A common assumption is that the normal force on the base of the slice acts at the

midpoint thus reducing the number of unknowns to 5n− 2. From moment, force and Mohr-

Coulomb relationships, only n − 2 assumptions are left to make the problem determinate

(Table 1). How these last degrees of freedom are handled differentiate the available methods

of analysis. The most popular methods are reviewed below (Abramson et al., 2002):

• Ordinary Method of Slices (OMS) — This method neglects all interslice forces, there-

fore relying on moment equilibrium alone. It is one of the simplest analysis procedures.

• Bishop’s Simplified Method — assumes that all interslice shear forces are zero, reducing

2

Table 1: Required variables for safety factor analysis

Equations Condition

n Moment Equilibrium for each Slice

2n Force equilibrium in two directions for each slice

n Mohr-Coulomb relationship between shear strength and normal effective stress

4n Total Number of equations

Unknowns Variable

1 Factor of safety

n Normal force at base of each slice

n Location of normal force

n Shear force at base of each slice

n− 1 Interslice force

n− 1 Inclination of interslice force

n− 1 Location of interslice force (line of thrust)

6n− 2 Total number of unknowns

the number of unknowns by n− 1. This leaves 4n− 1 degrees of freedom, leaving the

solution overdetermined as horizontal force equilibrium will not be satisfied for one

slice.

• Janbu’s Simplified Method — Janbu also assumes zero interslice shear forces. Simi-

lar to Bishop’s method, the solution is overdetermined and does not satisfy moment

equilibrium completely. Janbu presents a correction factor f0 to account for this inad-

equacy.

• Spencer’s Method — Spencer proposes a method that rigorously satisfies static equi-

librium by assuming that resultant interslice forces to have a constant, but unknown

inclination. It requires an iterative method for both the factor of safety and the in-

terslice force inclination between the slices, matching the required 4n equations. It is

considered a complete procedure (i.e. force and moment equilibrium are satisfied for

all slices).

3

1.3 The Need for Global Optimization

For all LEMs presented above, the critical failure surface is generally found by trial and error.

Many search methods were developed (Greco, 1996, Malkawi et al., 2001), but in most cases a

good initial guess is needed, and the quality of results depends on the engineers’ experience.

The factor of safety function is often highly multimodal, and it is certainly non-smooth,

being regarded as an N − p complete problem (Cheng, 2007). This multi-modality generally

results from multi-layering of the soil or mineral lenses.

Therefore, finding the critical slip surface pertains to the field of global optimization.

Well-known deterministic methods such as the gradient or Newton’s method inherently per-

form local optimization (Polyak, 1987). Further, Newton’s method does not even distinguish

minima (or maxima) from saddle points (Polyak, 1987).

A class of techniques that has been relatively successful in global search is that of the

stochastic methods. These methods are particularly useful for slope stability, as they are

generally robust and relatively insensitive to problem type.

A promising start-up is simulated annealing (SA), a powerful stochastic method for

global optimization.

1.4 Simulated Annealing (SA)

Simulated annealing (SA) was developed in the early 80s (Kirkpatrick et al., 1983) for com-

binatorial optimization. It differs from other search methods as it allows downhill as well

as uphill moves, based on the underlying similarity between the physico-chemical process of

annealing and the minimization of a function (Kirkpatrick, 1984). In annealing, a material

(i.e. metal) is heated until its molecules acquire sufficient mobility (a melted state). Then, by

decreasing the temperature slowly, the molecules undergo various configuration changes, al-

ways seeking for a lower energy state. If the decrease is sufficiently slow, a perfect crystalline

solid will form, and the system will be at its minimum energy state. If the temperature is

decreased fast, as in quenching, the molecules will collapse into an amorphous solid and have

poor physical properties (its energy state will be at a “local minimum”).

In statistical mechanics, the probability of the atom existing at an energy state can be

4

modeled by a Boltzmann distribution P (E) = exp (−E/T ). It is clear that the higher the

temperature the greater the chance of an atom of existing at a higher energy state. This

concept is implemented as a transition probability for the optimization through the famous

Metropolis criterion (Metropolis et al., 1953).

Similarly for SA, let E1 be the energy of the system at configuration 1. To model the

probability of the system changing to state 2 (at an Energy E2), the Metropolis criterion

consists in the acceptance/rejection probability P (dE) = exp (−dE/T ). Simulated annealing

uses this criterion as the acceptance probability for a random move - let the function F (x)

be defined. For the move to be accepted from x1 to x2, a random number r with uniform

distribution between (0, 1) is generated and compared with P (dF ) = exp (−dF/T ). If r <

P (dF ), then the move is accepted. If higher, the move is rejected. It can be seen that if

dF < 0 (F2 < F1), the move will always be accepted as the move is clearly downhill.

It can be seen that at high temperatures, more uphill moves will be accepted. It is the

goal of simulated annealing to start the temperature T high enough so the search method

can sample every valley of the parameter space. A “melting criterion” was initially proposed

as a 0.8 ratio between accepted and rejected moves (Kirkpatrick, 1984). Then, by reducing

the temperature the search becomes more selective and less up-hill moves are accepted, until

at T = 0 only down-hill moves are accepted.

The algorithm was soon extended to continuous functions (Corana et al., 1987). The

position vector becomes then analogous to the molecular arrangement of a metal, and the cost

function to the energy of the arrangement. The goal becomes to find the perfect arrangement

of atoms which will minimize the overall energy of a system, by performing random-walks

along the function domain.

In a more rigorous formulation, simulated annealing depends on three functions

(Van Laarhoven and Aarts, 1987, Yao, 1995):

1. A generation function GXY (Tk), which generates a state Y based on a state X at the

annealing temperature Tk.

2. An acceptance function AXY (Tk), which is the transition probability from one state to

another. This is determined by the Metropolis criterion.

5

3. An annealing schedule S(Tk), which determines the temperature reduction rule.

During the past two decades, intensive research has been invested in improving the

generation and annealing schedule, such that global convergence is guaranteed at a higher

speed.

1.5 Improvements on Simulated Annealing

The method soon became as popular as the well-known genetic algorithms (GA), being used

in various fields for image and signal processing, biology, geophysical inversion (Pei et al.,

2007) and finance (Van Laarhoven and Aarts, 1987). Similarities between the two methods

were soon drawn, due to the “natural” basis of the algorithms (Davis, 1987).

The original annealing has many issues, and is now considered in fact a simulated

quenching (SQ) algorithm as its convergence is not guaranteed for every case (Ingber, 1993).

These are mainly referred to the algorithms which use an exponentially decreasing schedule

without an adequate generation functon, such as:

T = rTk, (2)

which fails to comply with the necessary convergence conditions. This is not to say that

quenching methods are inefficient, in fact, for many specific problems in which the researchers

know the behavior of the system, simulated quenching has performances that are far superior

and should therefore be used.

Adaptive step-size for the random-walk have been proven to increase performance

(Corana et al., 1987). By increasing the step-size if the search is going well (and vice-

versa if badly), the ratio between accepted and rejected moves is kept close to 1:1. Even

though sufficiency conditions have not been proved for this algorithm, it was shown to be

efficient in finding global minima of very hard test functions, such as Rosenbrock valleys of

up to 1020 local minima.

The concept of using an adaptive step-size has been essential to the development of

better annealing algorithms. The Classical Simulated Annealing (CSA) (Geman and Geman,

1984) was the first annealing algorithm with a rigorous mathematical proof for its global

6

convergence. It was proven to converge if a Gaussian generation is used for GXY (TK),

coupled with an S(Tk) that decreases no faster than To/(log k). However, a logarithmic

decreasing schedule is considered to be too slow, and for many problems the number of

iterations required is considered as “overkill” (Ingber, 1993).

An exponentially faster variant known as fast simulated annealing (FSA) was proposed

(Szu and Hartley, 1987) which uses a Cauchy distribution as GXY (TK). It was heuristically

proven that if coupled with an annealing schedule T = T0/k, the method will converge

rapidly, due to its capabilities of performing longer jumps in the parameter space. Corana

et al. (1987), using a similar proof, proved the convergence of a schedule exponentially faster

than the fast annealing, namely T = T0/ exp (k + 1). The algorithm of VFSA by itself, even

though faster than classical SA, is still slow for many problems (Yao, 1995).

To address this, Ingber (1989) proposed a different generation temperature for each

dimension, and the re-annealing of temperature every couple of iterations. The combination

of the VFSA with a re-annealing is often referred to as adaptive simulated annealing (Chen

and Luk, 1999, Ingber, 1993). It has been shown successful in fields such as signal processing

(Chen and Luk, 1999) as well as wavelet estimation in geophysics (Velis and Ulrych, 1996).

The original simulated annealing, as proposed by Kirkpatrick (1984), was applied by

Cheng (2003) with a fair amount of success. However, in a recent comparison (Cheng et al.,

2007b), simulated annealing was found to require a large amount of iterations. Further,

parameter tuning was also found to be an issue. Due to the number of different slope

geometries, the optimization method is required to be relatively insensitive to change in

function characteristics.

The present study was designed primarily to assess the performance of state-of-art SA

methods when applied to circular search, and determine the most adequate parameters. The

methods of VFSA, ASA, Auto-refine search and simplex are compared based on speed and

reliability. It was noted that a transformation of variables from centre-point/radius to three

points on the slope greatly improves the efficiency of the search, and reasons are given based

on the theoretical framework of annealing.

7

2 Methods

The very fast simulated annealing (VFSA) and the adaptive simulated annealing (ASA)

were implemented in the SLIDE 5.032 software developed by Rocscience, Inc. SLIDE is a

2D slope stability program designed for rock or soil slopes and contains a variety of modeling

capabilities, including material shear strength, external loading, groundwater and support.

The software already employs several automated search methods, the most efficient one

being the Auto-Refine Search (ARS). The auto-refine search is a typical tailored algorithm,

specifically designed for circular surfaces, and is used as the main comparative method.

A standard down-hill search was also implemented for comparison purposes - the multi-

dimension downhill simplex method (Simplex). It is considered a strictly local search method

(Corana et al., 1987, Press et al., 2002), although it is fairly robust and often finds the global

minimum (Corana et al., 1987).

Both the auto-refine search and the simplex are deterministic methods, whereas sim-

ulated annealing is considered a stochastic method, therefore requiring the use of random-

number generators. For all purposes, pseudo-random number generation uses the CRandom-

Number class developed by Rocscience. The CRandomNumber class allows a variety of capabil-

ities and carries a variety of statistical distributions. Finally, CRandomNumber allows the use

of seeds for pseudo-random number generation, therefore making the optimization results

reproducible.

Finally, a crucial issue is the validity of the failure surfaces generated by the optimiza-

tion methods. In many cases, especially for circular search, invalid surfaces are frequently

generated due to a number of reasons (no slope intersection, convergence problems, etc).

Currently, the following error-handling was universally adopted: if an invalid surface is

generated, the optimal surface so far is retrieved and a new random-walk is performed using

this surface. If this procedure fails for a certain number of trials, the iteration breaks and

reports an error flag. This general scheme is used with few modifications for all the methods

described below.

Spencer’s method is used for all cases as it considers both moment and force equilib-

rium, and is considered the simplest of the complete equilibrium procedures (Duncan and

8

Wright, 2005). All methods were implemented with the assumption of a circular failure

surface. They were initially implemented with x, y (centre co-ordinates) and the radius r

being the control parameters.

2.1 Very Fast Simulated Annealing (VFSA)

VFSA was implemented in the Aslide.cpp file under the function void Do_Single

_Centers_VFSA(). The algorithm is derived from a standard Boltzmann simulated anneal-

ing with a more sophisticated generation function and an exponential annealing schedule. A

program flow chart is shown in Figure 1.

In the very fast simulated annealing, the generation function Y = G(X, Tk) is based on

the product of a D-dimensional Cauchy distribution. The generation is based on a random

walk for each dimension i, ruled by the following functional relationship:

Yi = Xi + ri(Li) (3)

In the former equation, L represents the characteristic length of the problem. In the original

formulation by (Ingber, 1989), Li = Ui − Bi in which Ui represents the upper bound of

the ith-dimension and Bi represents the lower bound. In the current algorithm L=diagonal

of slope geometry. If Yi is an invalid surface, it is simply discarded. The random variable

ri ∈ [0, 1] follows a Cauchy distribution, and is mapped from a uniform random variable

ui ∈ [0, 1] using the following function:

ri = sgn(ui − 0.5)Ti

[(1 +

1

Ti

)|2ui−1|− 1

](4)

The generation of the Cauchy variable and the random number is performed by the void

GetStepVFSA(). The walk is performed by the function void PerformWalkVFSA(). The

Cauchy-based generator presented above allows for different sensitivities at different temper-

atures. The simplified version of the VFSA presented here does not perform a re-annealing,

therefore Ti = T for 1 ≤ i ≤ n. This generation function allows for the sensitivity of each

dimension to the parameters; at high Ti the distribution is almost uniform, allowing for

long-range sampling. As Ti is decreased, the distribution becomes a Cauchy, and the further

9

the temperature is decreased the distribution becomes more and more centered around 0

(the lorentzian wings “fold”). For the very fast annealing a higher temperature has a similar

effect to a larger value of L. The behavior of the generation function is further investigated

in section 4.

The annealing schedule S(Tk) is performed by the function void Reduce_Temperature

_VFSA():

Ti = Tin exp (−cik1/n) (5)

The control parameter ci (in this case ci = c for all dimensions) is a user-defined

control parameter. The higher c is set, the faster the schedule is decreased. However,

without implementing the re-annealing, c should preferably be set lower (by lower we mean

between 1 and 3) to guarantee convergence.

Two stopping criterion found in literature were implemented: comparing the fopt[k]

with fopt[k − j] (Cheng et al., 2007b), where j is an integer between 0 and nepsilon. If the

difference fopt[k]− fopt[k− j] < ftol for all j, the program is stopped. In other words, if there

has not been any visible improvement for the global optimum in the previous n� consecutive

runs, the algorithm is to be stopped. An alternative that is to stop when the temperature

becomes sufficiently close to 0 — currently if the temperature T reaches 10−4 × Tin .

2.2 Re-annealing and Adaptive Simulated Annealing (ASA)

The convergence of VFSA is still slow for many applications (Ingber, 1993, Velis and Ulrych,

1996). The reason is that it assumes the same sensitivity for each dimension of the parameter,

the generation temperature used in equation 4 is identical to the acceptance function. In fact,

the parameter space can have completely different sensitivities for each dimension, therefore

it is suggested the use of different generation temperatures for each dimension Ti,gen and a

periodic re-scaling of these temperatures. By re-scaling, the temperature of the insensitive

dimensions is increased relative to the sensitive dimensions every Naccept iterations, and the

sampling range is “stretched out”. This method is often referred to as very fast simulated

re-annealing (VFSR) or adaptive simulated annealing (ASA) (Chen and Luk, 1999).

10

The sensitivity si of each parameter is given by the partial derivative of the function

with relation to the i-th dimension of the position vector:

si =∂F

∂Xi(6)

The temperature Ti,gen is linearly re-scaled proportional to the ratio of the maximum

sensitivity smax of the i-th dimensions:

Ti,gen =smaxsi

Ti,gen (7)

A flow-chart of the algorithm is presented below. At the initialization step, the accep-

tance temperature Taccept = Taccept(0) is first set to the initial value of the cost function, thus

scaling the temperature to the behavior of the cost function and the quality of the first guess.

Further, this adapts the Metropolis criterion to the problem in question, as the ratio dE/T

will be in the same order of magnitude. At every re-scaling, the initial temperature Taccept(0)

will be reset to the last accepted cost function and the acceptance temperature since the

re-scaling iteration will be reset to the best value of the function so far (fopt). The annealing

schedule is also re-scaled for the generation, by an inverse relation from equation 5.

ki = −1

clog

(Ti,gen(ki)

Ti,gen(0)

)n(8)

The acceptance counter is also re-scaled:

ka = −1

clog

(Taccept(ki)

Taccept(0)

)n(9)

This continually adapts S(Tk) to the behavior of the function. After every Ngen itera-

tions, the algorithm is annealed according to the VFSA rule (equation 5), and the generation

function is updated according to the following rule. Adaptive simulated annealing was im-

plemented in a class CASA by inheritance, and can be found under (“XiaoASA.h”). The

base class CVFSA contains the functions implemented in the VFSA.

2.3 Transformation of Variables - ASAc and ASAt

The VFSA and ASA were originally implemented with the control parameter v(X1, X2, X3),

in which X1 = x coordinate of the centre, X2 = y coordinate and X3 = r the radius of the

11

circle. However, due to the theoretical framework of ASA, it was noticed that the original

formulation required improvements. The function domain is unbounded if centre and radius

are used, which goes against the assumptions set by Ingber (1989) for a valid Ui and Li.

Therefore, a transformation of variables is suggested. Instead of the traditional centre

point and radius schemes (Duncan and Wright, 2005), it is well known that three non-colinear

points define a circle. Our control parameters become then two points on the slope surface,

and a third point determined by the tangent vector to the circle. Now, for v(X1, X2, X3),

X1 = pt1, X2 = pt2 and X2 = θ, where pt1 and pt2 are the points on the slope and θ is the

tangent angle that determines the third point. The third point forms a point tangent to the

circle, and by analytical geometry the centre and radius of the circle can be easily found.

With this transformation, the domain of the function becomes bounded. The upper

slope line is parameterized by t ranging from 0 to 1. A one-to-one mapping exists between

each point pt on the slope line and t (t↔pt). The angle θ is also bounded by the line

connecting pt1 and pt2, as well as the vertical line parallel to the y-axis. This Adaptive

Simulated Annealing with Three points (ASAt) has proven efficiency over the ASA with

centre and radius (refer to section 3). To differentiate between the methods, the original

Adaptive Simulated Annealing with Centre-radius parameters is referred to as ASAc. We

refer to the general method simply as ASA.

2.4 Simplex

The Simplex method was implemented into the Aslide.cpp file under the function void

Do_Single_Centers_Simplex(). The Simplex method was originally developed by (Nelder

and Mead, 1965) for estimating Hessian matrices. The version used in the current work was

based on the numerical recipes for C++ (Press et al., 2002), and a detailed description of

the algorithm can be found therein. A simplified flow chart is presented in Figure 3 (y[i]

represents the function value at point i).

The Simplex is known as a multi-dimensional downhill method (Press et al., 2002).

Its allows a geometric interpretation: an n-dimensional Simplex has n + 1 vertices. In two-

dimensions the simplex is a triangle and in three a tetrahedron. Each vertex is a point in the

function, and the algorithm sorts the vertices into high points and low points according to the

12

function value. The Simplex then moves by making reflections and contractions around its

vertices, tumbling downhill in an n-dimensional topography until it finds the local minimum.

When the simplex reaches a narrow valley, the method can contract itself and ooze through

the “eye of needle”, contracting itself in all directions (Press et al., 2002).

Advantages of the Simplex method include easy implementation, robustness, speed,

and the fact that it requires only function evaluations. The main features of the current

simplex method are:

1. Initialization: The Simplex method requires ndim+1 initial circles. The vertices of the

Simplex are stored in the matrix p[i][j] with 1 ≤ i ≤ 4 and 1 ≤ j

2.5 Auto-Refine Search (ARS)

The Auto-Refine Search is the main circular optimization method in SLIDE. It can be seen

as an iterative improvement method. An outline of the method is given below (Rocscience,

2006):

1. The slope is divided into a fixed number of divisions according to the user defined

Divisions Along Slope.

2. Circles are then generated between EACH PAIR OF DIVISIONS, according to the

value of the Circles Per Division.

3. The safety factors are then calculated for these circles, and the average safety factor

associated with EACH DIVISION along the slope, is recorded. This constitutes ONE

ITERATION of the Auto Refine Search method.

4. The value of Divisions To Use In Next Iteration, is then used, to determine how many

divisions along the slope, will be used to define a new, narrowed search area, in the

next iteration. Only the divisions with the lowest (average) safety factors are used, the

divisions with high safety factors are discarded from the analysis. For example, if 10

Divisions Along Slope are set with 50% divisions to use in the next iteration, then after

each iteration 5 divisions will be used. The divisions of the slope which are retained,

are then used to form a new slope poly-line. Using this new, narrowed slope surface,

steps 1 to 4 are then repeated, for the Number of Iterations specified.

14

Figure 1: Flowchart for VFSA

15

Figure 2: Flowchart for ASA

16

Figure 3: Flowchart for Simplex

17

3 Results

The optimization methods (ARS, Simplex, VFSA, ASAc and ASAt) were tested on twenty

SLIDE verification problems as well as twenty-three SLIDE customer cases. The verification

are actual engineering cases (Rocscience, 2006), which include weak layers, bolts, dams and

embankments. The cases related to circular search were selected, and the numbering from

the original files was preserved.

For the customer cases, the running time was recorded as a measure of speed. Indeed,

number of iterations is important for mathematical analysis as it is less sensitive to problem

characteristics (Sorkin, 1991). However, for practical purposes, running time (in seconds) is

paramount, as it includes all the operations performed by the engine including error-checks

and initialization of the algorithm. In some cases (as we will discuss later) a fair amount of

computational effort was wasted due to the generation of invalid surfaces.

Unless otherwise specified, the default tolerance of δ = 0.005 was used for the factor of

safety convergence.

3.1 Verification cases for SLIDE

The parameters of the optimization methods are presented below, and were kept constant

for all the verification cases:

VFSA: L = (1/8)Diagonal, Nt = 50, c = 2 and Tin = 2, with a stopping criterion of

Tstop = Tin10−5 or N� = 50.

ASAc: L =Diagonal, N� = 10, Naccept = 20, Ngen = 200, c = 2 and ftol = 0.0001.

ASAt: N� = 30, Naccept = 50, Ngen = 200 and c = 2, and ftol = 0.0001.

For Simplex, the original parameters from the Numerical Recipes was used (Press et al.,

2002). For ARS the default settings in SLIDE were used (4500 generation surfaces, Divisions

along the slope=10, Circles per division=10 and Divisions to use in the next iteration=50%).

The factor of safeties labeled in red are within 0.01 of the lowest factor of safety found.

ASA demonstrated a better capacity of finding the global minima in all verification cases,

followed by a similar performance between VFSA and the Auto-refine search. The success

rate of each method was also computed. Examples 87 and 88 are such cases, and they are

18

Table 2: Safety factors found for customer cases. Labeled in red are the safety factors not

within 0.01 of the global minimum

Verification # Auto-Refine VFSA ASAc Simplex ASAt[I]

Search

1 0.985 0.985 0.985 0.985 0.985

2 1.590 1.586 1.587 1.586 1.588

3 1.374 1.373 1.373 1.373 1.375

4 0.990 0.990 0.991 0.990 0.992

5 1.959 1.948 1.948 1.948 1.948

6 1.948 1.948 1.948 1.948 1.948

10 1.494 1.494 1.494 1.494 1.495

11 1.063 1.063 1.063 1.452 1.064

12 1.071 1.071 1.071 1.070 1.074

13 1.334 1.334 1.330 1.330 1.338

14 1.405 1.404 1.404 1.403 1.406

15 0.424 0.422 0.423 0.423 0.423

16 1.117 1.117 1.117 1.117 1.118

17 1.342 1.342 1.342 1.342 1.343

87 1.067 1.072 1.044 1.134 1.035

88 1.042 1.091 1.055 1.087 1.049

% found 94% 88% 100% 88% 100%

19

discussed in the next section. It must be noted that for some cases even though the function

values are close, the surface location was found to be completely different.

3.2 Customer Cases for SLIDE

The customer cases for SLIDE were considerably more challenging than the verification

examples, due to the multitude of slope geometries and their strong multi-layering. The

Simplex method was modified: after every down-hill simplex run, the best vertex is obtained.

Three new vertices are added to the best vertex and the algorithm is re-run. Results were

obtained both for this modified simplex as well as the standard.

The VFSA was also modified to investigate the sensitivity of the algorithm to the

characteristic length L. The VFSA algorithm was tested both with a characteristic length

of an eighth of the slope diagonal, as well as the full diagonal. ASAc was tested with the

same parameters used for the verification case.

On the other hand, the parameters for ASAt were found to be too conservative and

therefore drastically reduced speed for some special cases. Therefore, a new set of parameters

was chosen:

ASAt[I]: N� = 30, Naccept = 50, Ngen = 300, c = 2, ftol = 0.001 and δ = 0.0001.

ASAt[II]: N� = 20, Naccept = 30, Ngen = 300 and c = 2, ftol = 0.001 and δ = 0.0001.

The parameter ftol was set lower than δ = 0.0001 to prevent possible precision problems.

In Table 3, the results labeled in blue are the lowest factor of safeties found among

the methods, and the results labeled in red are within 0.05 of the global minimum. This

criterion is purely comparative, and was determined based on a range in which all surfaces

were approximately in the same location. The percentage of minima successfully found is

also summarized in Table 3.

Table 4 presents the running time in seconds for each of the verification cases. It can

be noticed that the speed of a method is highly problem dependent. In general the simplex

was found to be faster for most of the examples, followed by the auto-refine search and lastly

by the annealing methods.

20

Table 3: Safety Factors found for Customer Cases. Global minimum found are labeled in

blue, whereas minima within 0.05 of global minimum are considered satisfactory and are

labeled in red.

C. E. Simplex Simplex VFSA Auto- ASAc ASAt[I] ASAt[II]

# (3 iter) (1 iter) (FULL) Refine

1 0.921 0.921 0.763 0.769 0.763 0.769 0.780

2 1.214 1.287 1.121 1.121 1.123 1.127 1.127

3 1.271 1.450 1.23 1.278 1.275 1.280 1.281

4 1.242 1.389 1.133 0.206 0.238 0.193 0.214

5 1.412 1.444 1.408 1.436 1.409 1.415 1.415

6 1.321 1.321 1.253 1.323 1.263 1.251 1.263

7 1.525 2.167 1.211 1.175 1.175 1.184 1.184

8 1.945 2.290 1.811 2.231 1.804 1.818 1.807

9 2.992 3.485 1.237 1.221 1.239 1.234 1.233

10 1.563 1.408 1.406 1.406 1.406 1.406 1.406

11 2.292 2.292 2.282 2.283 2.286 2.287 2.287

13 2.050 2.578 1.263 1.028 0.937 1.075 0.973

14 1.506 1.510 1.079 1.130 1.114 1.146 1.114

15 0.450 0.215 0.831 0.163 0.844 0.102 0.102

16 1.441 1.306 1.262 1.260 1.284 1.311 1.287

17 1.208 Error 1.1777 1.216 1.176 1.225 1.188

19 0.704 0.651 0.651 0.658 0.651 0.651 0.651

26 2.601 3.596 2.595 2.596 2.608 2.600 2.602

34 1.280 1.682 1.176 1.156 1.152 0.813 0.812

40 1.707 1.322 0.241 0.299 0.244 0.219 0.219

41 1.008 1.069 0.663 0.664 0.662 0.613 0.597

42 0.188 0.189 0.185 0.188 0.185 0.194 0.194

Success % 32% 32% 77% 73% 86% 91% 100%

21

Table 4: Running time for customer cases. The running time for example 41 was not

considered due to the large computational effort required for all methods to convert Hoek-

Brown material properties.

C. E. Simplex Simplex VFSA Auto- ASAc ASAt[I] ASAt[II]

# (3 iter) (1 iter) (FULL) Refine

1 8 3 164 27 1092 260 134

2 96 91 167 17 273 243 92

3 185 32 178 4 459 87 79

4 108 57 367 7 178 103 174

5 3 0 58 0 117 17 12

6 39 5 2129 5 595 125 203

7 40 4 126 5 345 111 103

8 39 12 85 0 134 27 53

9 144 9 418 4 258 62 82

10 16 2 40 2 80 8 8

11 42 19 102 2 124 49 51

13 76 1 120 4 186 53 46

14 3 2 131 5 164 41 102

15 72 31 737 4 1763 6 7

16 14 13 105 6 138 17 24

17 32 17 48 2 156 38 34

19 19 1 21 3 93 15 23

26 39 13 70 1 107 22 21

34 130 22 184 8 851 124 129

40 32 15 183 3 243 28 23

42 8 3 164 27 1092 260 134

Time 55 17 290 14 388 84 84

22

4 Discussion and Explanatory Examples

4.1 Method Comparison — Verification Cases

The results in Table 2 demonstrate that for verification cases all the methods are successful.

For most cases the results were within 0.01 precision of each other. The ASA methods

were particularly efficient, finding the global minima within an accuracy range of 0.01 in

100% of the cases. Among the different cases, #87 and #88 presented the greatest variance.

Verification #87 is a problem taken from the Baseline case studied in the “Geo-synthetic

Reinforced Multi-tiered Walls”, a paper by Leshchinsky, D. and Han, J. (Rocscience, 2006).

The geometry consists of a three-tiered wall with geo-synthetic support. Both cases are based

on the same slope geometry and differ only in the material and support properties. Table 5

presents the material properties for case #87 and Table 6 presents its support properties.

Table 5: Material properties for Verification Case #87

Zone c (kPa) Φ (0) γ (kN/m3)

Reinforced and retained fill 0 34 18

Foundation soil 10 34 18

Blocks 2.5 34 18

Table 6: Support Properties for Verification Case #87

Length (m) Tensile Strength (kN/m) Pullout Strength

6.3 10.0 80%

Figure 4 displays a refined grid-search for case #87. The main challenge for search

methods are the numerous regions with invalid surfaces. There were 853 valid surfaces

against 3987 invalid surfaces, corresponding to a failure of generation rate of 82%. The most

common cause was failure of the factor of safety function to converge within required number

of iterations, occurring in 52% of the cases.

The failure surface found by ARS (Figure 5) comprises the two lowest tiers, and creates

a tension crack due to the reverse curvature of the circle. The safety factor found was 1.067.

23

Figure 4: Grid-search performed for Verification Case #87

Figure 5: Auto-Refine Search for Verification Case #87

24

The global minimum found by ASAc (Figure 6), on the other hand, encompasses all

tiers (1.055). This is a specially interesting case, as the function values are very close even

though the surfaces are different. It must be noticed that in a slope analysis this difference

may well be critical to the safety of the project.

Figure 6: Adaptive Simulated Annealing for Verification Case #87

4.2 Method Comparison — Customer Cases

The results in Table 3 show that ASAt[II] was the most successful in terms of percentage of

minima found within 0.05 of the global minimum, finding the global minimum in 100% of

the cases. The remaining annealing methods (ASAc, VFSA) were quite accurate in terms

of accuracy when compared to ARS. The simplex method proved fairly ineffective, having

significant failures in many cases (such as customer case #8). The restarts did little to

improve the search. This is unsurprising as the simplex method is in essence a downhill local

search by concept. Nevertheless the occasional global effectiveness was displayed, including

the correct minimum for customer case #15.

For most cases, the annealing methods and ARS had very similar results. Cases of

particular interest are #4, #8 and #15. It can be noticed that the VFSA and both Simplex

failed to find the region of correct answer for #4, and for #15 only the Simplex and ARS

found answers close to the global minima (Table 3). For case #8, only the annealing methods

25

Figure 7: Grid-Search for Customer Case #4

found the region with the correct factor of safety.

The grid-search (Figure 7) displays the function landscape for case #4. It can be

noticed that this case is particularly difficult, being strongly multi-modal.

ARS was able to find the minimum very fast, probably because the divisions were

refined enough to create a circle in the weak layer. ASAc was also successful in finding the

region of global minima, albeit at a lower speed. ASAt, on the other hand, tracked down the

global minimum relatively fast (Table 3 - 84s). This case shows both the general parameter-

rescaling advantages of ASA, as well as the influence on speed from the transformation of

variables.

The grid-search representations of the energy landscape might be deceiving. The col-

ored grid is a two dimensional representation of a three-dimensional function. The grid

search displays the best factor of safety from each centre, therefore the graph only shows

the best two-dimensional “slice” of the overall problem (the centre with the best radius). It

should be expected that there are at least n-times as many local minima in the complete

problem.

In several cases, the annealing methods located lower global minimum than the ARS,

including significantly different locations. For customer case #8 ARS (Figure 8) found a

26

Figure 8: Auto-Refine Search for Customer Case #8

safety factor of 2.231, whereas both ASAc (FOS=1.804 - Figure 9), VFSA (FOS=1.811) as

well as ASAt (FOS=1.807) found a more more critical failure surface.

The minimum found by ARS (2.231) comprises a large slip surface starting at the top

of the slope all the way to the base, whereas the safety factor of 1.804 found by ASAc is

restricted to approximately the upper half of the geometry (Figure 9). Again, this difference

may have crucial impact in the design of a slope.

Case #15 presents also a challenge, however the difficulty lies primarily in the discon-

tinuity of the function as opposed to its multimodality. The critical factor of safety lies in a

very narrow range close to the base of the slope, and between the initial guess and the slope

there are many invalid surfaces (Figure 8). There were 23470 invalid surfaces as opposed to

5868 valid surfaces (80% failure generation rate). Errors were due to a variety of reasons,

especially the high probability of multiple slope intersection. The critical failure surface, as

noted by the grid-search and the ARS, tends to an almost vertical tension-crack.

The failure of ASA and VFSA to converge can be explained by the large discontinuity

regions (Figure 13). The methods probably failed to generate valid surfaces around the

global minimum, therefore they were “pushed” towards the left of the slope. Centre-radius is

particularly ill-suited for this problem, as significant computation effort was wasted in error-

checks (1763 seconds - Table 6) and the search method did not span the function domain

27

Figure 9: Adaptive Simulated Annealing for Customer Case #8

Figure 10: Grid-Search for Customer Case #15

28

Figure 11: Auto-Refine Search for Customer Case #15

Figure 12: Adaptive Simulated Annealing for Customer Case #15

29

Figure 13: ASAt for Customer Case #15

properly. The transformation of variables, on the other hand, was highly successful for this

case, quickly finding the global minimum (7s - Table 6).

Finally, it was noticed that multi-layering and an apparently hard geometry do not

imply difficult optimization problems. User example # 34 looks hard but is easy, whereas

user example #15 looks easy but is extremely multi-modal. A significant amount of variance

was found in geometries with support (i.e. verification cases #87 & #88, customer case #8).

Duncan and Wright (2005) point out that reinforcements and external loads might have an

effect on the numerical stability of the solution for the complete equilibrium procedures (such

as Spencer’s method).

4.3 Simplex, ARS and other Optimization

Even though the Simplex is known to adapt to local landscapes (Nelder and Mead, 1965),

it was robust in a global search, finding 95% of the verification cases and 35% of the cus-

tomer cases. The current initialization works well for simple slopes, as noted in section 3.

The simplex was also able to find the global minimum in customer case #15 (Table 3), a

particularly complex slope.

It was noticed that the efficiency of the simplex depended significantly on how the

vertex matrix is initialized. For example, if the circles were all in the same neighborhood

30

(or maybe concentric), the search would be strongly localized. A desirable initialization will

create vertices that roughly span the function domain. As the local landscape tends towards

the global landscape, the simplex acquires global efficiency.

The ARS seems to have an efficient initialization algorithm - it involves the parameter-

ization of the slope surface and point-wise generation of circles. This concept may be used

successfully in the case of the simplex and improve the quality of the search.

It must be kept in mind that the simplex is at its core a local search (Corana et al.,

1987), and 100% global convergence should not be expected. However, the method is still

very useful: for simple slopes the simplex has shown both speed and reliability.

4.4 Annealing — Practical Considerations

In general, ASA with centre-radius were proven to be robust, failing only in a few exceptional

cases for which large amounts of failure surfaces were found. VFSA has a slightly lower but

still satisfactory performance. A further advantage of ASA is that for most problems its

parameters did not need tuning, a very important characteristic for slope problems (Cheng

et al., 2007b). ASA only requires the user to specify a control parameter c, which does not

affect the convergence of the problem.

The main drawback for the centre-radius scheme tested is speed. They are both at

least 20 times slower than ARS. This difference in speed may be unsurprising, as ARS is a

deterministic method, and simulated annealing a stochastic method. Further, there is also a

conceptual imperfection with the use of centre and radius as parameters. Due to its random

sampling nature, the domain for SA methods must be bounded. However, this is not the

case for centre and radius. It is easy to see that even for very far centers valid surfaces

can still be generated, and theoretically the parameter space cannot be completely covered.

Therefore, global convergence is not guaranteed.

The transformation to three points corrects both problems. As two points must always

be constrained to the slope-line, very few invalid surfaces are generated. The number of

degrees of freedom will be maintained, the two parametric points on the slope line and a

third angular vector to represent a third point. All these three parameters are bounded, and

the generation function can now sample the complete range of the slope if enough points are

31

generated.

Finally, it has been noted that all methods run into speed issues when Hoek-Brown

material properties are used. Most slopes tested use Mohr-Coulomb material properties,

with the exception of customer #41. The running time for the case ranged from 40 min to

6h, therefore its speed result was not included in Table 2 as it is considered an outlier.

4.5 Annealing — General Remarks

Standard simulated annealing (SA) has been successfully used on slope stability, (Cheng

et al., 2007a,b) with accuracy comparable to other general algorithms for global optimization.

Its effectiveness lies in its ability to sample large regions of the domain without falling into

a local minimum.

However, this intelligence is purely making up-hill moves as well as down-hill. Two

major drawbacks are the large number of iterations and the necessity for tuning the para-

meters for each solution case. VFSA offers a faster solution than standard annealing, with a

generation function that is highly superior. The quality of the solution from VFSA is similar

to the deterministic optimization in SLIDE, ARS.

However, VFSA is still fairly simple. Unlike genetic algorithms (Davis, 1987), the

algorithm itself does not “learn” much from the function behavior as the search proceeds.

ASA addresses both speed and effectiveness issues (Ingber, 1993). By allowing the search

method to adjust to the sensitivity of the different dimensions of the parameter space, ASA

optimizes its search range by allowing an exponentially decreasing schedule. It must be

noted that the speed of ASA does not rely on the fact that it stops “fast”, as much as the

fact that it can reliably find the local minimum within tolerable time.

The optimization performed by ASA can be understood intuitively. If the search is

going well, decrease the temperature rapidly and if the search is going badly, decrease the

temperature slowly. For ASA, high temperatures signify a larger search range and a less

selective acceptance function. This is very similar to the “dynamic stepping” (Corana et al.,

1987), in which the ratio between accepted and rejected points is used to control the step

size.

This “adaptivenes” addresses the asymptotic convergence noted by many researchers

32

(Van LaarHoven, 1988). As the search approximates the valley with the global minimum,

standard annealing methods will descend slower as most points generated will be uphill. A

stochastic method with a constant probability distribution will become slower and slower as

the method converges to the minima simply because it has a low probability of generating

points around its current state. As annealing converges in distribution, it is essential for

its search range to shrink “intelligently” as it approaches the answer so as to accelerate

convergence. This is more efficient than setting the characteristic length of the search to

be reduced to one fourth or even one tenth of the width of a weak layer, as suggested by

(Duncan and Wright, 2005). Such an approach would slow the search prohibitively, whereas

the probabilistic approach taken by ASA displays robustness as well as a tolerable speed.

The sensitivity approach currently used is more robust, as the derivatives directly

incorporate function information, whereas “acceptance rate” is more indirect. An interesting

question may arise: if the partial derivative of the function is calculated at every Naccept

points, why do we not walk downhill? It must be remembered that the partial derivative is

computed at the point with the lowest factor of safety found so far. At the Naccept iteration

the search method may be far from the current global minimum position, and the partial

derivative of the function at that point becomes now useless. We can pursue the derivational

method and compute the partial derivative at each dimension, however we would be some

type of hybrid “gradient-combinatorial” method that most probably violates the underlying

convergence conditions of annealing, although a proof can not be given at this point.

Annealing has a hierarchical underlying concept: it searches the major regions of local

minima first then refines its search over the small valleys. This is especially obvious with the

temperature-dependent Cauchy distribution as presented by Ingber. At high temperatures

the search ranges over the domain (Figure 14) whereas as the temperature is decreased the

search narrows down on the local valleys. The graphs were generated using 1000 points

distributed according to equation 4. Figure 14 displays an almost uniform distribution at

T = 10, the distribution allows the search method to run through the entire domain.

33

Figure 14: Generation range based on Cauchy distribution for T = 10

Figure 15: Generation range based on Cauchy distribution for T = 0.2

34

Figure 16: Generation range based on Cauchy distribution for T = 0.01

It can be noticed that the distribution approximates a Cauchy distribution as the T

decreases (Figures 14, 15,16), and with a decreasing “standard deviation”. The graphs can

be interpreted as the “range” of the search method, 0 being the current point and (-1, 1)

representing the lower and upper bound respectively. At high temperatures the move-sets

have a long-range, therefore sampling in a “coarse grained” fashion, similar to the FSA

(Szu and Hartley, 1987). As the temperature is reduced, the Cauchy machine refines the

search around the current state (Figure 16). It has to be noted that FSA assumes the same

temperature for generation and for all the dimensions, whereas ASA assumes a different

temperature for each dimension and a different temperature for the acceptance function. As

noted above, this suggests a certain conceptual similarity to the dynamic walk presented by

Corana et al. (1987).

The efficiency of such a method for multi-modal was validated by many of the slope

cases. For analysis purposes, the last accepted factor of safety was plotted for customer case

#4 for 4775 accepted points (Figure 17). The hierarchical convergence can be easily noticed

- at high temperatures the method samples over the space, and at low temperatures the

method refines around the local minimum.

Sorkin (1991) has argued that the effectiveness of annealing depends on how well the

move-set and the energy function are matched. The author suggests annealing is effective

35

Figure 17: Convergence of ASAc for Customer Case #4

for functions possessing a certain degree of “fractalness”, such as customer case #4 discussed

above. Fractal functions possess a hierarchical self-similarity, in other words, big valleys are

self-similar to the small valleys. It is easy to see from Figure 18 why VFSA and ASA would

be extremely efficient and gradient-dependent methods would fail.

On the other hand, Sorkin (1991) argues that for other function landscapes annealing

can be very inefficient (Figure 18). Customer # 17 might be such a case. The grid search

(Figure 19) displays two large plateaus and clear down-hill behavior from the function. ASAc

was 15 times slower than ARS. The problems that annealing faced with case #17 are general

to slopes with large regions with homogeneous material properties, for which the factor of

safeties vary very little at the local minima. For such cases, a few runs from a down-hill

method such as the simplex can easily find the answer (Table 3). Annealing methods are

then “overkill” (see convergence plot Figure 20).

Despite its efficiency, the convergence of ASA relies on the same sufficiency proofs as

CSA (Geman and Geman, 1984), FSA (Szu and Hartley, 1987) and VFSA (Ingber, 1989).

It relies on the fact that any points can be generated infinitely often in time on the func-

tion domain. It was noted by Yao (1995) that generating does not mean visiting, and the

acceptance temperature can prevent the search from visiting a region, even though points

there have been generated. Any annealing method fails if the valley with the minima is not

36

Figure 18: (a) A function landscape in which SA methods are inefficient; (b) a fractal

landscape in which SA methods have high performance (Sorkin, 1991)

Figure 19: Grid Search for Customer Case #17

37

Figure 20: Convergence of ASAc for Customer Case #17

visited (as in customer case #15). Another characteristic is that even ASA does not escape

from the “curse of dimensionality” - the exponent 1/n strongly affects convergence speed at

higher dimensions (Ingber, 1989).

Simulated annealing has shown promising results by finding the global minima in al-

most all cases tested in the current work, some which involve extremely hard minimization

problems. The accuracy of ASA is superior to the deterministic circular search methods,

and it has shown the highest reliability out of the methods tested. However, its speed is

in general lower than the ARS or the Simplex, and in many cases this does not result in a

significant improvement in accuracy.

Simulated annealing is surprisingly successful even for its simplicity, being known as

nature’s optimization. It must be remembered however, that nature is much more powerful

than any current human technology, and the optimization of crystalline structures can be

carried out by millions of atoms in very fast re-arrangements, whereas human computers can

only compute at a limited speed.

Ackley (Davis, 1987) stated in his review on bit vector function optimization that

“There is simply no best strategy for solving all possible problems”. It can be argued however,

that there can be a strategy that probably offers the best solution for most of the problems,

and simulated annealing seems to be a promising answer.

38

5 Conclusions

Based on the analysis of the global optimization methods, the following conclusions are

made:

1. Adaptive simulated annealing is a very powerful algorithm for the global optimization

of slope stability problems. With the proper control variables, ASA was successful and

relatively fast for 100% of the verification and customer cases.

2. The efficiency of circular optimization has been proven to be greater when three points

on the slope are used as control parameters (ARS and ASAt), as opposed to centre

point and radius (ASAc). The use of two points on the slope reduces invalid surface

generation, and the parameterization of the slope line allows complete sampling of the

parameter space.

3. For simple geometries (such as the verification cases) all methods work well. In general

the simplex is the fastest method, however it fails to optimize more complex slopes,

especially with multi-layering.

4. The convergence behavior of ASA was shown to be more efficient for landscapes with

fractal characteristics (highly multi-modal functions with a self-similarity).

39

6 Recommendations

Based on the analysis of the global optimization methods, the following recommendations

are made:

1. Record major local minima:

The current optimization methods only output the global minimum, in other words,

the lowest safety factor found. It is recommended that major local minima should be

also recorded Duncan and Wright (2005). In many cases they indicate regions of failure

and discrete regions where instability might occur.

2. Improve seeding for Simplex method:

The simplex proved significant robustness in the global optimization cases tested, es-

pecially for the verification cases. Further work on method is strongly recommended,

especially the use of better seeding techniques. It is recommended that its seeding to

be based on the initialization of the ARS.

3. Implementation for Parallel Processing:

The current optimization methods were implemented for running in a single processor.

However, parallel processing is recommended as for some slope geometries the number

of iterations required is very large. A second reason is that the methods run very slowly

when Hoek-Brown material properties are used, due to the computation effort used to

convert stress data.

4. Extension of Simulated Annealing for Non-Circular Surfaces:

The simulated annealing methods were proven to be very efficient for circular search

methods. Their implementation for general non-circular surfaces is strongly recom-

mended. Adaptive simulated annealing is expected to favorably compete with current

multi-dimension methods used for non-circular search.

40

List of Symbols

AXY (Tk) acceptance function

ARS Auto-Refine Search

ASA Adaptive Simulated Annealing

ASAt Adaptive Simulated Annealing with Three Points

ASAc Adaptive Simulated Annealing with centre-point/radius

c control parameter for VFSA and ASA or cohesion of soil

d standard deviation of ensemble-averaged simulation

E energy function analogous to objective function

F objective function to be minimized

ftol tolerance for stopping criterion

FSA fast simulated annealing

GXY (Tk) state-generation function

i counter for dimension

j counter for stopping criterion

k counter for each internal temperature iteration (VFSA)

ki counter for each generation temperature at i-th dimension (VFSA)

ka counter for temperature decrease(ASA)

n number of dimensions or number of simplex vertices

n(� iterations required for stopping criterion i

r random number following Cauchy distribution

si sensitivity of the function at the i-th dimension

S(Tk) annealing schedule

T general annealing temperature

Ti annealing temperature for VFSA at each dimension i

Ti,gen generation temperature for ASA at each dimension i

Ti,accept acceptance temperature for ASA at each dimension i

Taccept(0) initial acceptance temperature for ASA (equal for all dimensions)

V FSA very fast simulated annealing

41

v position vector to be evaluated by objective function

X initial position vector

Y Final position vector

Greek Letters

δ tolerance for factor of safety equation

� tolerance level

φ Angle of Friction Envelope

θ angle of tangent vector to the circular surface

42

References

Abramson, L. W., Lee, T. S., Sharma, S., and Boyce, G. M. (2002). Slope Stability and

Stabilization Methods. John Wiley & Sons, New York.

Chen, S. and Luk, B. L. (1999). Adaptive simulated annealing for optimization in signal

processing applications. Signal Processing, 79(1):117–128.

Cheng, Y. M. (2003). Location of critical failure surface and some further studies on slope

stability analysis. Computers and Geotechnics, 30(3):255–267.

Cheng, Y. M. (2007). Global optimization analysis of slope stability by simulated annealing

with dynamic bounds and dirac function. Engineering Optimization, 39(1):17–32.

Cheng, Y. M., Lansivaara, T., and Wei, W. B. (2007a). Two-dimensional slope stability

analysis by limit equilibrium and strength reduction methods. Computers and Geotechnics,

34(3):137–150.

Cheng, Y. M., Li, L., and Chi, S. C. (2007b). Performance studies on six heuristic global

optimization methods in the location of critical slip surface. Computers and Geotechnics,

34(6):462–484.

Corana, A., Marchesi, M., Martini, C., and Ridella, S. (1987). Minimizing multimodal

functions of continuous-variables with the simulated annealing algorithm. Acm Transactions

on Mathematical Software, 13(3):262–280.

Davis, L., editor (1987). Genetic Algorithms and Simulated Annealing. Research Notes in

Artificial Intelligence. Pitman Publishing, London, Great Britain.

Duncan, J. and Wright, S. (2005). Soil Strength and Slope Stability. John Wiley & Sons,

Inc., New Jersey.

Geman, S. and Geman, D. (1984). Stochastic relaxation, gibbs distributions, and the

bayesian restoration of images. Ieee Transactions on Pattern Analysis and Machine In-

telligence, 6(6):721–741.

Greco, V. R. (1996). Efficient monte carlo technique for locating critical slip surface. Journal

of Geotechnical Engineering-Asce, 122(7):517–525.

Ingber, L. (1989). Very fast simulated re-annealing. Mathematical and Computer Modelling,

12(8):967–973.

43

Ingber, L. (1993). Simulated annealing - practice versus theory. Mathematical and Computer

Modelling, 18(11):29–57.

Kirkpatrick, S. (1984). Optimization by simulated annealing — quantitative studies. Journal

of Statistical Physics, 34(5-6):975–986.

Kirkpatrick, S., Gelatt, C. D., and Vecchi, M. P. (1983). Optimization by simulated anneal-

ing. Science, 220(4598):671–680.

Malkawi, A. I. H., Hassan, W. F., and Sarma, S. K. (2001). Global search method for

locating general slip surface using monte carlo techniques. Journal of Geotechnical and

Geoenvironmental Engineering, 127(8):688–698.

Metropolis, N., Rosenbluth, A. W., Rosenbluth, M. N., Teller, A. H., and Teller, E. (1953).

Equation of state calculations by fast computing machines. Journal of Chemical Physics,

21(6):1087–1092.

Nelder, J. A. and Mead, R. (1965). A simplex-method for function minimization. Computer

Journal, 7(4):308–313.

Pei, D. H., Louie, J. N., and Pullammanappallil, S. K. (2007). Application of simulated

annealing inversion on high-frequency fundamental-mode rayleigh wave dispersion curves.

Geophysics, 72(5):R77–R85.

Polyak, B. T. (1987). Introduction to Optimization. Translation Series in Mathematics and

Engineering. Optimization Software, Inc., New York.

Press, W. H., Teukolsky, S., Vetterling, W., and Flannery, B. (2002). Numerical Recipes

C++: The Art of Scientific Computing. Press Syndicate of the University of Cambridge,

Cambridge, UK, 2nd edition.

Rocscience (2006). Slide manual. Technical report, Rocscience, Inc.

Sorkin, G. B. (1991). Efficient simulated annealing on fractal energy landscapes. Algorith-

mica, 6(3):367–418.

Szu, H. and Hartley, R. (1987). Fast simulated annealing. Physics Letters A, 122(3-4):157–

162.

Van LaarHoven, P. J. M. (1988). Theoretical and computational aspects of simulated anneal-

ing. CWI Tract. CWI: Centrum voor Wiskunde en Informatica, Amsterdam, Netherlands.

Van Laarhoven, P. J. M. and Aarts, E. H. L. (1987). Simulated Annealing: Theory and

44

Applications. Mathematics and Its Applications. D. Reidel Publishing Company, Dordretch,

Holland.

Velis, D. R. and Ulrych, T. J. (1996). Simulated annealing wavelet estimation via fourth-

order cumulant matching. Geophysics, 61(6):1939–1948.

Yao, X. (1995). A new simulated annealing algorithm. International Journal of Computer

Mathematics, 56(3-4):161–168.

45