Recent Advances in Optimal Structural Design

By the Technical Committee on Optimal Structural Design of the

Technical Administrative Committee on Analysis and Computation of the

Technical Activities Division of The Structural Engineering Institute of the

American Society of Civil Engineers

Edited by Scott A. Burns

May 3, 2002

PREFACE

In 1980, the ASCE Committee on Optimization of the Committee on Electronic Computation produced a report that was later published in 1981 by ASCE as Structural Optimization: Recent Developments and Applications, edited by Ovadia E. Lev. This work documented important de-velopments in structural optimization over the period 1972 to 1980, and included an extensive bibliography of books and research papers on the topic. It provided a timely resource for students and researchers working in the area.

Since then, the field of structural optimization has experienced many new developments in both computational techniques and applications. The purpose of this present work is to carry forward the previous effort of documenting the recent developments in structural optimization, for the period 1980 to 2000. The ASCE Technical Committee on Optimal Structural Design of the Technical Administrative Committee on Analysis and Computation began this effort in 1996, and completed the first draft in 2000. Members of the committee who directly contributed to this report are:

Jasbir S. Arora, The University of Iowa (Chapter 1) Richard Balling, Brigham Young University (Chapter 2) Scott A. Burns, University of Illinois at Urbana-Champaign (Chapter 11) Franklin Y. Cheng, University of Missouri-Rolla (Chapter 9) Allen C. Estes, United States Military Academy (Chapter 10) Christopher M. Foley, Marquette University (Chapter 8) Dan M. Frangopol, Masaru Miyake, Jung S. Kong, and Emhaidy S. Gharaibeh,

University of Colorado at Boulder (Chapter 10) Donald E. Grierson, and Siavash Khajehpour, University of Waterloo (Chapter 4) M. Ohsaki, Kyoto University (Chpater 5) Shahram Pezeshk and Charles V. Camp, The University of Memphis (Chapter 3) G. P. Steven, and O. M. Querin, The University of Sydney (Chapter 6) Colby C. Swan, University of Iowa (Chapter 5) Y. M. Xie, X. Y. Yang, and Q. Q. Liang, Victoria University of Technology (Chapter 6) Lei Xu, University of Waterloo (Chapter 7)

The report is organized into four main sections. The first group of chapters concerns advances in computational methods and solution algorithms. Chapter 1 reviews recent developments in methods that treat discrete variables. Many applications in structural optimization involve design variables that are not continuous in nature and are meaningful only at discrete values, such as commercially available hot-rolled steel sections. Chapter 2 documents recent developments in decomposition methods. Structural optimization problems are sometimes solved more efficiently when they are subdivided into multiple, smaller problems that can be solved in parallel. Chapter 3 addresses the issue of discrete design variables as well, in addition to addressing issues of find-ing a global optimum and treating multiple objective functions, through the application of ge-netic algorithms.

The next group of chapters focuses on the role of optimization during the various stages of struc-tural design. Chapter 4 is concerned with the wide range of issues facing a designer during the conceptual stage of design. Another important issue that must be confronted early in the design process is selection of topology and layout. Chapters 5 and 6 present two different approaches to doing this. In recent years, increasing attention has been paid to the effects of semi-rigid connec-tions in steel design. Chapter 7 addresses how this aspect of behavior may be incorporated into optimal structural design.

The third group of chapters addresses the stochastic nature of design in relation to structural op-timization. There has been great interest recently in designing structures for multiple perform-ance objectives corresponding to hazards with different mean recurrence intervals. This so-called performance-based design is the topic of Chapter 8. Chapter 9 addresses life-cycle optimal de-sign of structures intended to resist seismic activity, where conflicting objectives give rise to compromise solutions. Life-cycle design also concerns expected repair and maintenance costs; Chapter 10 summarizes recent work in optimal bridge maintenance planning.

The last chapter of the report comprises an extensive collection of journal papers on structural optimization, published between the 1987 and 1999. The citations are presented alphabetically, and are organized by an index grouping the citations according to topic.

This report follows on the heels of the 1997 OSD Committee publication, Guide to Structural Optimization, edited by Jasbir S. Arora. The OSD Committee was awarded the 1998 ASCE State-of-the Art Award for this document.

I would like to thank the authors for their hard work and dedication in preparing their contribu-tions to this report. I sincerely appreciate their efforts, including the many hours of committee discussion on content and direction, and repeated chapter reviews and revisions.

Scott A. Burns Urbana, IL April 2002

TABLE OF CONTENTS

preface......................................................................................................................................................... iii

Chapter 1.......................................................................................................................................................1 Methods for Discrete Variable Structural Optimization............................................................1

Introduction .........................................................................................................................1 Problem Definition..............................................................................................................2 Classification of Problems ..................................................................................................3 Overview of Literature ........................................................................................................4 Methods for Discrete Variable Problems ............................................................................7

Branch and Bound Method ....................................................................................7 Basic BBM................................................................................................7 BBM with Local Minimization.................................................................9 BBM for MD-NLP..................................................................................10 BBM with Multiple Branches .................................................................12

Integer Programming ...........................................................................................13 Sequential Linearization.......................................................................................14 Simulated Annealing............................................................................................15

Algorithm SA: Simulated Annealing ......................................................16 Genetic Algorithms ..............................................................................................18

Algorithm GA: Genetic Algorithm .........................................................22 Other Methods......................................................................................................22

Dynamic Rounding-off ...........................................................................22 Penalty Approach....................................................................................23 Lagrangian Relaxation Techniques.........................................................23 Neighborhood Search Method ................................................................24 Cutting-Plane Technique.........................................................................24 Heuristic Techniques...............................................................................24

Methods for Linked Discrete Variables ............................................................................25 Problem Formulations..........................................................................................26

Single Design Variable Formulation 1....................................................26 Single Design Variable Formulation 2....................................................26 Single Design Variable Formulation 3....................................................26 Multiple Design Variable Formulation 1 ................................................26 Multiple Design Variable Formulation 2 ................................................26

Solution Strategies for LD-NLP Problems...........................................................27 Single Variable Approaches with Approximations.................................27 Single Variable Approaches without Approximations............................30 Mixed Single and Multiple Variable Approaches...................................32 Rounding-off Methods............................................................................33

Concluding Remarks .........................................................................................................34 References .........................................................................................................................35

Chapter 2.....................................................................................................................................................41 Decomposition Methods in Structural Optimization ................................................................41

Introduction .......................................................................................................................41 Efficiency ..........................................................................................................................42 Decomposed Optimization................................................................................................43 Analysis Autonomy...........................................................................................................44 Design Autonomy .............................................................................................................47 Other Developments..........................................................................................................51 References .........................................................................................................................51

Chapter 3.....................................................................................................................................................55 State of the Art on the Use of Genetic Algorithms in Design of Steel Structures...................55

Introduction .......................................................................................................................55 Introduction To Genetic Algorithms .................................................................................56

The Fundamental Theorem of Genetic Algorithms .............................................57 Formulation of Structural Optimization Problems...............................................58 Penalized Objective Function ..............................................................................59

Multiple Segment Penalty Function........................................................59 Nonlinear Penalty Function.....................................................................59

Coding and Decoding...........................................................................................61 Binary Coding .........................................................................................61

Selection Strategies ..............................................................................................62 Proportional Selection.............................................................................62 Inverse Fitness.........................................................................................63 Fitness Scaling ........................................................................................63 Group Selection.......................................................................................66

Crossover .............................................................................................................66 One-Point Crossover ...............................................................................67 Two-Point Crossover ..............................................................................67 Uniform Crossover..................................................................................68 Adapting Crossover.................................................................................68 Techniques for Adapting Crossover........................................................69

Mutation ...............................................................................................................69 Basic Parameters Of Genetic Algorithms .........................................................................70 Related Research Findings ................................................................................................70 Summary ...........................................................................................................................75 References .........................................................................................................................76

Chapter 4.....................................................................................................................................................81 Conceptual Design Optimization of Engineering Structures ...................................................81

Introduction .......................................................................................................................81 The Conceptual Design Problem.......................................................................................82 Computer-Based Conceptual Design: 1989 to 1999 .........................................................85 References .........................................................................................................................91

Chapter 5.....................................................................................................................................................97 Topology and Geometry Optimization of Trusses and Frames ...............................................97

Introduction .......................................................................................................................97 Discrete Structural Topology Optimization ......................................................................98

Topology Optimization Problem..........................................................................99 Optimization Methods........................................................................................100

Stress Constraints ...............................................................................................102 Frequency Constraints........................................................................................104 Simultaneous Optimization of Topology and Geometry ...................................106

Continuum Structural Topology Optimization................................................................107 Description of Structural Material Arrangements ..............................................107 Design Problem Formulation .............................................................................109 Structural Analysis and Sensitivity Analysis Problems .....................................111

Linear Quasi-Static Analysis.................................................................111 Vibrational Eigenvalue Analysis...........................................................112 Treatment of Buckling Instabilities.......................................................113

Uniqueness of Solutions.....................................................................................114 Representative Examples ...................................................................................115

The Canyon Bridge Problem.................................................................115 The MBB Beam Problem......................................................................116 The Cantilever Problem ........................................................................117 Plate Vibration Problem........................................................................118

Discussion of Continuum Topology Optimization Characteristics....................118 Discussion/Recommendations For Future Research And Practical Considerations .......119 References .......................................................................................................................119

Chapter 6...................................................................................................................................................125 Evolutionary Structural Optimization .....................................................................................125

Introduction .....................................................................................................................125 Basic ESO .......................................................................................................................126 Examples of Basic ESO ..................................................................................................127

A Michell Type Structure...................................................................................127 A Structure with Uniform Surface Stress...........................................................130

ESO For Structures With Stiffness/Displacement Constraint .........................................132 ESO for Overall Stiffness Constraint .................................................................132 ESO for Displacement Constraint ......................................................................134 An Example of Optimization with Three Displacement Constraints.................135 Application to the Optimal Layout Design of Reinforcing Steel in Concrete ...136

ESO For Frequency Optimization...................................................................................138 ESO For Optimization Against Buckling........................................................................139 Bi-Directional Evolutionary Structural Optimization .....................................................141

BESO for Stiffness/Displacement Optimization................................................141 BESO for Frequency Optimization ....................................................................143

Conclusions .....................................................................................................................146 Acknowledgement...........................................................................................................146 References .......................................................................................................................146

Chapter 7...................................................................................................................................................147 Design and Optimization of Semi-Rigid Framed Structures .................................................147

Introduction .....................................................................................................................147 Semi-Rigid Steel Construction in Design Specifications................................................149 Connection Behaviour and Modeling .............................................................................150 Semi-Rigid Frame Analysis ............................................................................................152 Design and Optimization of Semi-Rigid Steel Frames ...................................................157

Design of Semi-Rigid Frames............................................................................157 Least Weight Design of Semi-Rigid Steel Frames.............................................158 Minimum Cost Design of Beam-to-Column Connections.................................159

Optimum Member and Connection Design of Semi-Rigid Steel Frames ...........160 Concluding Remarks .......................................................................................................164 Acknowledgment ............................................................................................................164 References .......................................................................................................................164

Chapter 8...................................................................................................................................................169 Optimized Performance-Based Design for Buildings .............................................................169

Introduction .....................................................................................................................169 Performance-Based Engineering.....................................................................................170 Quantifying Performance Seismic Loading .................................................................176

Design Actions...................................................................................................176 Acceptance Criteria General ...........................................................................178 Acceptance Criteria Steel Structures...............................................................179 Acceptance Criteria - Reinforced Concrete Structures ......................................183 Acceptance Criteria - Analysis Procedures........................................................184

Quantifying Performance Wind Loading.....................................................................191 Quantifying Performance Gravity Loading..................................................................193 Quantifying Performance Nonstructural Elements ......................................................194 Quantifying Performance Using damage Structural Elements.....................................197 Quantifying the Loads.....................................................................................................200

Live (Gravity) Loading ......................................................................................201 Wind Loading.....................................................................................................204 Earthquake Loading ...........................................................................................206

Performance-Based Optimization ...................................................................................215 General Optimization Problem Statements ........................................................216 PBD Optimization Historical Perspective.......................................................219 PBE Optimization Problem Format ...................................................................224 Objective Function .............................................................................................225 Probabilistic (Expected) Loading ......................................................................228 Problem Constraints ..........................................................................................229

Solution Methods Available............................................................................................233 Concluding Remarks .......................................................................................................236 References .......................................................................................................................236

Chapter 9...................................................................................................................................................241 Multiobjective Optimum Design of Seismic-Resistant Structures.........................................241

Introduction .....................................................................................................................242 Multiobjective Optimization and Pareto Optimum.........................................................243 Game Theory Algorithms................................................................................................244 Multiobjective Optimization With Genetic Algorithms..................................................244 Pareto GA With Constrained Multiobjective Optimization ............................................245 Life-Cycle Cost And Multilevel Optimization................................................................245 Numerical Examples .......................................................................................................247

Optimum Design of Seismic Structure with/without Control ............................247 Optimum Design for Selection of a Structural System......................................249 Sensitivity of Cost Function on Structural Probability Failure and Optimum Solution ..............................................................................................................250

Conclusions .....................................................................................................................252 Acknowledgments...........................................................................................................253 References .......................................................................................................................253

Chapter 10.................................................................................................................................................257 Reliability- and Cost- Oriented Optimal Bridge Maintenance Planning..............................257

Background .....................................................................................................................257 Probabilistic Maintenance Models ..................................................................................258

Numerical Examples ..........................................................................................262 optimum maintenance and repair methods......................................................................263

Optimization Methodology ................................................................................263 Concrete Bridge Deck Example.........................................................................264 Optimizing and Updating...................................................................................268

Concluding Remarks .......................................................................................................268 Acknowledgements .........................................................................................................268 References .......................................................................................................................268

Chapter 11.................................................................................................................................................271 Journal Publications on Optimal Structural Design, 1985-1999............................................271

Index into References......................................................................................................369

Chapter 1

METHODS FOR DISCRETE VARIABLE STRUCTURAL OPTIMIZATION

Discrete design variables are encountered in most practical engineering design applications. Promising optimization methods that can treat such design variables in their solution process are reviewed. Many times selection of solution method depends on the type of discrete design vari-ables, and cost and constraint functions for the problem. Depending on these quantities, discrete design variable optimization problems are classified into six different types first. Then methods suitable for each problem type are identified, described and tabulated. Recent applications of dis-crete variable optimization methods are briefly described. Formulations and methods for struc-tural design optimization with commercially available sections are also discussed.

INTRODUCTION In practical applications of optimization, discrete design variables occur naturally in the problem formulation. For example,

plate thickness that must be selected from the available ones structural members must be selected from commercially available ones number of rebars in an RC member must be an integer diameter of rebars must be selected from the available ones number of bolts must be an integer size of the RC members must correspond to the available form work material properties must correspond to the available materials number of strands in a prestressed beam or column must be an integer Such variables must be treated properly in procedures for optimum design of systems. To de-scribe these procedures, a general mixed continuous-discrete variable optimization problem is

Contributed by Prof. Jasbir S. Arora, The University of Iowa

defined first. Then six categories of the problem with mixed continuous discrete design variables are presented. Characteristics of each problem type are described and discussed. Basic ideas and procedures of various methods for solving such problems are presented. These include, branch and bound, sequential linearization, integer programming, simulated annealing, genetic algo-rithm, rounding-off, and some other methods. An overview of the recent applications of the methods is also presented. The paper presents a synthesis of the material on the subject devel-oped by the author and his co-workers during the last decade. It is taken from several recent pub-lications, such as Arora, et al (1994), Arora and Huang (1996), Huang and Arora (1997), and several references cited in there.

PROBLEM DEFINITION A general mixed discrete-continuous variable nonlinear optimization problem (MD-NLP) is de-fined a follows:

( )minimize f x 0, 1,...,

0, 1,...,

j

j

g j psubject to

g j p m= =

= + (1)

( )1 2, , ,...., ; 1,....,ii i i i i iq dx D D d d d i n = = ; 1,....iL i iU dx x x i n n = +

where f and gj are objective (cost) and constraint functions respectively, xiL and xiU are lower and upper bounds for the design variable xi, p, m and n are the numbers of equality constraints, total constraints and design variables, respectively, nd is the number of discrete design variables, Di is the set of discrete values for the ith variable, qi is the number of available discrete values for the ith variable, and dik is the kth discrete value for the ith variable. Note that the foregoing problem definition includes integer variable as well as 0-1 variable problems. If the problem as only continuous variables, and the functions f and gj are assumed to be twice continuously dif-ferentiable, we obtain the standard nonlinear programming problem (NLP). Methods to solve such problems have been discussed extensively in many textbooks (Gill et al 1981; Luenberger 1984; Vanderplaats 1984; Minoux 1986; Arora 1989).

The formulation in Eqs. (1) can also be used to solve design problems with linked discrete vari-ables (Arora and Huang 1996; Huang and Arora 1997a). A linked discrete variable is defined as the one whose value specifies the values for a group of parameters related to it. There are many design applications where such linked discrete variables are encountered. Solution methods for such problems are discussed later in the paper.

It is important to note that the discrete variable optimization problems usually require consider-able more computational effort compared to the continuous variable problems. This is true even though the number of feasible points with discrete variables is finite and they are infinite with continuous variables.

CLASSIFICATION OF PROBLEMS Depending on the type of design variables, and cost and constraint functions, the mixed continu-ous-discrete variable problems are classified into six types as described in Table 1. Problem Type 1 is the standard NLP that is obtained from Problem (1) when there are no discrete vari-ables. Problem Type 2 is discussed most often in the structural optimization literature. Many methods have been presented to solve this problem type. For Problem Type 4, some of the prob-lem functions can be evaluated only at the discrete values during the solution process. Examples for such variables are: number of strands in a prestressed beam or column, number of reinforcing rods, number of teeth in a gear, and the number of bolts for a joint. On the other hand, a problem is not classified as Type 4 if the effects of the non-discrete design points can be some how simulated. For instance, a coil spring must have integer number of coils. However, during the solution process, having non-integer number of coils is acceptable as long as function evalua-tions are possible (they may or may not have physical meaning). Problem Type 5 covers many practical applications, such as structural optimization with sections available in a catalog. Prob-lem Type 6 is a combinatorial problem, which is a nondifferentiable discrete problem. An exam-ple of such problems is the traveling salesman problem where the total distance traveled to visit a number of cities needs to be minimized. A set of integers (cities) can be arranged in different or-ders to specify a travel schedule (a design). A particular integer can appear only once in a se-quence. A variation of this would be to select a unique set of nd numbers from a much larger pool of contenders. Examples of applications of this type of engineering problems where a se-quence of operations needs to be determined can be found in Kincaid and Padula (1990) and Huang, Hsieh and Arora (1997).

It will be seen later that some of the discrete variable methods assume that the functions and their derivatives can be evaluated at non-discrete points. Such methods are not applicable to some of

Table 1. Classification of Discrete Variable Design Problems

Problem type

Problem description

1 Continuous design variables; functions are twice continuously differentiable (stan-dard NLP problem).

2 Mixed design variables, functions are twice continuously differentiable; discrete variables can have non-discrete values during the solution process (i.e., functions can be evaluated at non-discrete points).

3 Mixed design variables; functions are non-differentiable; discrete variables can have non-discrete values during the solution process.

4 Mixed design variables; functions may or may not be differentiable; some of the discrete variables must have only discrete value in the solution process.

5 Mixed design variables; functions may or may not be differentiable; some of the discrete variables are linked to others; assignment of a value to one variable speci-fies values for others.

6 Combinatorial problems; purely discrete non-differentiable problems, such as the traveling salesman problem

the problem types defined in Table 1. Various characteristics of the six problem types are sum-marized in Table 2.

OVERVIEW OF LITERATURE Enumerating on the allowable discrete values for each of the design variables can always solve discrete variable optimization problems. The number of combinations Nc to be evaluated in such a calculation is given as

i

n

1ic qN

d

=

= (2)

It is seen, however, that the number of combinations to be analyzed increases very rapidly with an increase in the number of design variables nd and the number of allowable discrete values for each variable qi. This can lead to an extremely large computational effort to solve the problem. Thus most of the discrete variable optimization methods try to reduce the search to only a partial list of possible combinations using various strategies and heuristic rules. Most of the methods guarantee optimal solution for only a very restricted class of problems (linear or convex). For more general nonlinear problems, however, good usable solutions can be obtained depending on how much computation is allowed. In this section an overview of the methods for discrete vari-able optimization is presented.

Toakley (1968) recognized the need for discrete variable structural optimization. However, since the continuous variable optimization algorithms were not fully developed at that time, emphasis shifted to the development of algorithms for such problems. There were very few papers on dis-crete variable structural optimization at that time. The integer variable linear programming (LP) methods (called integer programming (IP) or 0-1 programming), and the branch and bound methods had been developed for general discrete optimization problems, and so it was natural to apply these methods for discrete structural optimization. Toakley (1968) applied a few of these methods for optimal design of plastic and elastic structures. Design of plastic structures was for-mulated as an LP problem which was transformed to a mixed integer-continuous variable LP problem. The elastic design of determinate trusses subjected to displacement constraints was also

Table 2. Characteristics of Design Variables and Functions for Problem Types

Problem type #

Variable types Functions differ-entiable?

Functions de-fined at non-

discrete points?

Non-discrete values allowed

for discrete variables?

Are discrete variables linked?

1 Continuous Yes NA NA NA 2 Mixed Yes Yes Yes No 3 Mixed No Yes Yes No 4 Mixed Yes/No No No No 5 Mixed Yes/No No No Yes 6 Discrete No No No Yes/No

NA: Not Applicable

formulated as a mixed integer-continuous variable LP problem. The cutting plane method (Go-mory 1960), branch and bound method and heuristic techniques were used.

Reinschmidt (1971) used a branch and bound method to solve the problem of plastic design of building frames (which is a linear programming problem). The problem was transformed to an IP problem and a branch and bound method, based on Geoffrions (1967) implicit enumeration ap-proach was used. Elastic design of trusses subjected to stress, displacement, and member size constraints was also considered (which is a nonlinear programming (NLP) problem). The NLP problem was linearized and solved as a sequence of linear IP problems using the same branch and bound algorithm. Cella and Logcher (1971) solved the nonlinear problem of designing trusses subjected to stress constraints using the branch and bound method. The nonlinear prob-lem was attacked directly without linearization or introduction of integer variables. A filtered pattern search was used during the branching phase of the algorithm. The method simply evalu-ated each trial design and either accepted it or rejected it. Since each trial design required struc-tural response, an approximate reanalysis approach was used to reduce the computational effort.

A direct method combining Boxs algorithm and Hooke and Jeeves method was used by Lai and Auchenbach (1973) for structural optimization. Liebman et al (1981) transformed the discrete variable optimization problem to a sequence of unconstrained problems that were solved using an integer discrete gradient algorithm. Hua (1983) developed a special enumeration algorithm for discrete variable optimization of trusses with stress and displacement constraints. The method exploits the structure of the problem to develop heuristics that reduce the size of enumeration.

A comprehensive review of literature for mixed variable optimization problems has been pre-sented (Arora et al. 1994). A good overview of the methods for discrete variable optimization has also been presented by Bremicker et al. (1990), Loh and Papalambros (1991) and Vander-plaats and Thanedar (1991). Usually, the algorithms for mixed continuous-discrete variable op-timization problems are classified according to the type of variables they can handle (Loh and Papalambros 1991): Binary (zero-one) versus non-binary, purely discrete versus mixed continu-ous-discrete, and integer versus arbitrary discrete. Some methods cover more than one classifica-tion while others are strictly for a specific model. Loh and Papalambros (1991) first review the methods for mixed-discrete linear programming (MDLP) and then for mixed-discrete nonlinear programming (MDNLP). For MDLP, the methods reviewed are branch and bound, cutting plane, dynamic programming, implicit enumeration, and Lagrangian relaxation. A similar set of meth-ods is then reviewed for MDNLP.

Belegundu and Arora (1979) presented a survey of discrete variable optimization methods rela-tive to the structural design problems. Plastic and elastic structural design problems were formu-lated and discussed. It happens that the plastic design of frames can be formulated as a linear programming (LP) problem. For discrete variable optimization, such problems can be trans-formed to integer (0-1) linear programming (ILP) problems. Many algorithms are available to solve such problems. The elastic structural design problem is nonlinear. For such problems, se-quential linearization methods were discussed where the nonlinear problem was linearized and converted to an integer LP problem. The methods for nonlinear problems were classified into two broad categories: integer programming (IP), and non-integer programming such as branch and bound and heuristic methods. The IP techniques included the cutting plane method, branch

and bound (implicit enumeration), group theoretic techniques, network approach, and heuristic (approximate) techniques.

Vanderplaats and Thanedar (1991) and Thanedar and Vanderplaats (1994) classify the discrete variable structural optimization methods for Problem Type 2 into three categories: branch and bound, approximation, and ad hoc methods. In the approximation methods, the branch and bound method is used on an approximate problem after a continuous solution has been obtained for the original nonlinear problem. The approximate problem can be defined in several ways; e.g., lin-earized problem using Taylors expansion followed by solution using discrete linear program-ming methods, and conservative approximations which result in nonlinear approximate problem having explicit variables. Ad hoc methods consist of several different strategies, e.g., the penalty approach is classified as an ad hoc approach. Shin et al. (1990) introduce a sine penalty function to penalize the non-discrete values for the variables. Fu et al. (1991) have also used the penalty function approach to solve five small-scale engineering optimization problems. The difficulties with this approach are: introduction of additional local minima and repeated minimization by adjusting the penalty parameters. The dual method proposed by Schmit and Fluery (1980), which has its roots in the Lagrangian relaxation concept (Geoffrion 1974), is also classified as an ad hoc approach. Here, a separable approximation to the original nonlinear problem is first created which is then solved by the dual approach in a closed form. The method is not good if the origi-nal problem is not well approximated as a separable problem, or the allowable discrete values are widely separated.

Simulated annealing and genetic algorithms are stochastic methods that have their origins in natural processes. These methods can solve most types of discrete variable optimization prob-lems. The methods have been quite popular in the recent literature, especially the genetic algo-rithms for a variety of engineering applications.

It is observed that some of the methods for discrete variable optimization use the structure of the problem to speed up the search for the discrete solution. These methods are not suitable for im-plementation into a general-purpose application software. The branch and bound method, simu-lated annealing and genetic algorithm are the most general methods; however, they are also the most time consuming ones. In general, the numerical methods for solving MDNLP are classified as follows:

1. Branch and bound methods 2. Sequential linearization methods followed by branch and bound, integer program-

ming, simulated annealing, and others 3. Simulated annealing 4. Genetic algorithm 5. Other methodsrounding-off, penalty function approach, Lagrangian relaxation

methods, cutting-plane, heuristics, etc.

METHODS FOR DISCRETE VARIABLE PROBLEMS

Branch and Bound Method Branch and bound (BBM) method was originally developed for discrete variable linear pro-gramming (LP) problems for which a global solution is obtained. It is sometimes called an im-plicit enumeration method because it tries to reduce the entire enumeration in a systematic man-ner. It is one of the earliest and the best-known methods for discrete variable problems that have also been used to solve MD-NLP problems. The concepts of branching, bounding and fathoming are used to perform the search as explained later. The following definitions are useful for de-scription of the method, especially when applied to continuous problems (Reinschmidt 1971):

1. Half-Bandwidth. When r allowable discrete values are taken below and (r-1) values are taken above a given discrete value for a variable, giving 2r allowable values, the parameter r is called the half-bandwidth. It is used to limit the number of allowable values for a discrete variable, e.g., based on the rounded-off continuous solution.

2. Completion. Assignment of discrete values from the allowable ones to all the vari-ables is called a completion.

3. Feasible Completion. It is a completion that satisfies all the constraints. 4. Partial Solution. It is an assignment of discrete values to some but not all the vari-

ables for a continuous discrete problem. 5. Fathoming. A partial solution for a continuous problem or a discrete intermediate so-

lution for a discrete problem (node of the solution tree) is said to be fathomed if it is determined that no feasible completion of smaller cost than the one previously known can be determined from the current point. It implies that all possible completions have been implicitly enumerated from this node.

The first use of the branch and bound method is attributed to Land and Doig (1960) for linear problems. Dakin (1965) later modified the algorithm that has been subsequently used by many researchers. Garfinkel and Nemhauser (1972) have also described the use of BBM on linear pro-gramming problems. Reinschmidt (1971) used BBM to solve the integer linear programming problem related to plastic design of frames. The problem was posed as a linear programming problem and solved by a modified version of Geoffrions (1967,1969) implicit enumeration method for discrete programming. This algorithm, designed to solve linear 0-1 programming problems, is based on an algorithm due to Balas (1965). If q is the number of discrete values for each variable, then Geoffrions method will have a maximum of 2qn combinations. None of the algorithms ever enumerates all the possible combinations.

Basic BBM

There are two basic implementations of the BBM. In the first one, non-discrete values for the discrete variables are not allowed (or are not possible) during the solution process. This imple-mentation is quite straightforward; the concepts of branching, bounding and fathoming are used directly to obtain the final solution. No subproblems are solved; just the problem functions are evaluated for different combinations of design variables. In the second implementation, non-discrete values for the design variables are allowed. Forcing a variable to have a discrete value

generates a node of the tree. This is done by defining additional constraints to force out a discrete value for the variable. The subproblem is solved using either LP or NLP methods depending on the problem type.

To illustrate these procedures, let us consider the following LP problem:

21 1020 xxfMinimize = subject to 0751020 211 + xx:g 055x7x12:g 212 + 0901025 213 + xx:g }3,2,1,0{x1 , }6,5,4,3,2,1,0{x2

Huang and Arora (1997) also use this example. First let us solve the problem when non-discrete values for the variables are not allowed during the solution process. Since x1 and x2 can have 4

x=(3,6)T f=-120g=(-45,23,45)T

x=(3,5)T f=-110g=(-35,16,35)T

x=(2,6)T f=-100g=(-25,11,20)T

x=(3,4)T f=-100g=(-25,9,25)T

x=(2,5)T f=-90g=(-15,4,10)T

x=(1,6)T f=-80g=(-5,-1,-5)T

x=(2,4)T f=-80g=(-5,-3,0)T

x=(1,5)T f=-70g=(5,-8,-15)T

Node 2

Node 4

Node 7

Node 5

Node 9

Node 3

Node 6

Node 1

x=(3,3)T f=-90g=(-15,-2,15)T

Node 8

x=(3,2)T f=-80g=(-5,5,5)T

Node 10x=(2,3)T f=-70g=(5,-10,-10)T

Node 11

STOP - since noother feasible pointswith smaller cost

STOP - since noother feasible pointswith smaller cost

STOP - since costis larger than -80

STOP - since costis larger than -80

STOP - feasible costwill be higher than -80

Figure 1. Branch and bound method without solving continuous subproblems

and 7 possible discrete values respectively, the full enumeration requires evaluation of problem functions for 28 combinations; however, BBM can find the final solution in much fewer evalua-tions. For the problem, the derivatives of f with respect to x1 and x2 are always negative. This information can be used to advantage in BBM. One can enumerate the discrete points in the de-scending order of x1 and x2 to ensure that the cost function is always increased when one of the variables is perturbed to the next lower discrete value. The BBM for the problem is illustrated in Figure 1. For each point (called node), the cost and constraint function values are shown. From each node, assigning the next smaller value to each of the variables generates two more nodes. This is called branching. At each node all the problem functions are evaluated again. If there is any constraint violation at a node, further branching is necessary from that node. Once a feasible completion is obtained, the node need not be branched further since no point with lower cost is possible from there. Such nodes are said to have fathomed; i.e., reached their lowest point on the branch and no further branching will produce a solution with lower cost. Nodes 6 and 7 are fath-omed this way where the cost function has a value of -80. For the remaining nodes, this value becomes an upper bound for the cost function. This is called bounding. Later any node having cost function value higher than the current bound is also fathomed. Nodes 9, 10 and 11 are fath-omed because the designs are infeasible with the cost function value larger than or equal to the current bound of -80. Since no further branching is possible, the global solution for the problem is found at Nodes 6 and 7 in 11 evaluations.

BBM with Local Minimization

For optimization problems where the discrete variables can have non-discrete values during the solution process and all the functions are differentiable, we can take advantage of the local minimization procedures to reduce the number of nodes in the solution tree. In such a BBM pro-cedure, initially an optimum point is obtained by treating all the discrete variables as continuous. If the solution is discrete, an optimum point is obtained and the process is terminated. If one of the variables does not have a discrete value, then its value lies between two discrete values; e.g., dij < xi < dij+1. Now two subproblems are defined, one with the constraint xi dij and the other with xi dij+1. This process is also called branching which is slightly different from the one ex-plained earlier for purely discrete problems. It basically eliminates some portion of the continu-ous feasible region that is not feasible for the discrete problem. However, any of the discrete fea-sible solutions is not eliminated. The two subproblems are solved again, and the optimum solu-tions are stored as nodes of the tree containing optimum values for all the variables, the cost function and the appropriate bounds on the variables. This process of branching and solving con-tinuous problems is continued until a feasible discrete solution is obtained. Once this has been achieved, the cost function corresponding to the discrete feasible solution becomes an upper bound on the cost function for the remaining subproblems (nodes) to be solved later. The solu-tions that have cost values higher than the upper bound are eliminated from further consideration (i.e., they are fathomed).

The foregoing process of branching and fathoming is repeated from each of the unfathomed nodes. The search for the optimum terminates when all the nodes have been fathomed due to one of the following reasons: (i) a discrete optimum solution is found, (ii) no feasible continuous so-

lution can be found, or (iii) a feasible solution is found but the cost function value is higher than the established upper bound.

Figure 2 shows implementation of the BBM where requirements of discreteness and non-differentiability of the problem functions are relaxed during the solution process. Here one starts with a continuous solution for the problem. From that solution two subproblems are defined by imposing an additional constraint requiring that x1 be not between 1 and 2. Subproblem 1 im-poses the constraint x1 1 and the Subproblem 2 x1 2. Subproblem 1 is solved using the con-tinuous variable algorithm that gives a discrete value for x1 but non-discrete value for x2. There-fore further branching is needed from this node. Subproblem 2 is also solved using the continu-ous variable algorithm that gives discrete values for the variables with the cost function of -80. This gives an upper bound for the cost function and no further branching is needed from this node. Using the solution of Subproblem 1, two subproblems are defined by requiring that x2 be not between 6 and 7. Subproblem 3 imposes the constraint x2 6 and Subproblem 4 x2 7. Sub-problem 3 has a discrete solution with f = -80 which is same as the current upper bound. Since the solution is discrete, there is no need to branch further from there by defining more subprob-lems. Subproblem 4 does not lead to a discrete solution with f = -80. Since further branching from this node cannot lead to a discrete solution with a cost smaller than the current upper bound of -80, the node is fathomed. Thus the Subproblems 2 and 3 give the two optimum discrete solu-tions for the problem, as before.

Since the foregoing problem has only two design variables, it is fairly straightforward to decide how to create various nodes of the solution process. When there are more design variables, node creation and the branching processes are not unique. These aspects are discussed further for nonlinear programs.

BBM for MD-NLP

In structural optimization literature, the latter version of the BBM has been used most often, where functions are assumed to be differentiable and the design variables can have non-discrete values during the solution process. Reinschmidt (1971) has applied the algorithm to nonlinear problems along with the sequential linearization procedure. Gupta and Ravindran (1983) have used Dakins method together with a generalized reduced gradient method, to solve nonlinear mixed-integer problems. Mesquita and Kamat (1987) combine branch and bound with a sequen-tial quadratic programming (SQP) method to solve the problem of stiffened laminated composite plates. Sandgren (1990) used an approach similar to the one by Gupta and Ravindran, but in-cluded equality constraints having zero-one type variables for topological optimization problem. Branch and bound was combined with the exterior penalty function and SQP methods to treat the mixed-discrete NLP problem. John, Ramakrishnan and Sharma (1988) combined BBM with se-quential linearization for discrete optimal design of trusses. Hajela and Shih (1990) use BBM to solve multiobjective optimization problems with discrete and integer variables. Large storage and an exponential growth in computational effort may limit applicability of BBM to higher di-mensional problems.

Branch and bound method has been used successfully to deal with discrete design variable prob-lems and has proved to be the quite robust. However, for problems with large number of discrete design variables, the number of subproblems (nodes) becomes large. Therefore considerable ef-fort has been spent in investigating strategies to reduce the number of nodes by trying different fathoming and branching rules. For example, Mesquita and Kamat (1987) suggest fixing the variable that is used for branching to its proper value for the two subproblems, and eliminating it from further consideration. This reduces dimensionality of the problem that can result in effi-ciency. As the iterative process progresses, more and more variables are fixed and size of the op-timization problem keeps on reducing. Many other variations of the BBM for nonlinear continu-ous problems have been investigated to improve its efficiency. Two basic strategies, depth first search and breadth first search are discussed by Ringertz (1988), each having its advantages and drawbacks. Two detailed algorithms generalized from Ringertz (1988) and Sandgren (1990) are given in Arora, Huang and Hsieh (1994). Many enhancements enable the method to create fewer subproblems (nodes) without neglecting subproblems with possible lower cost. BBMs with enhancements can be found in Tseng, Wang and Ling (1995), Huang and Arora (1995, 1997a), and Huang (1995).

Hager and Balling (1988), Tseng et al. (1992,1995) and Arora and Huang (1997a) have investi-gated variations of the basic BBM to improve its efficiency. Since an early establishment of a good upper bound on the cost is important, it may be possible to accomplish this by choosing an appropriate variable for branching. More nodes or subproblems may be fathomed early if a smaller upper bound is established. Several ideas have been investigated in this regard. For ex-

x = (16/11, 59/11)f = - 82.7

x = (1, 6.15)f = - 81.5

Subproblem 1

Continuous solution

Subproblem 2

x = (1, 6)f = - 80

Subproblem 3

x = (0.5, 7)f = - 80

Subproblem 4

x2 6

x1 1x1 2

x2 7

x = (2, 4)f = - 80

Stop Discretefeasible solution

Stop Discretefeasible solution

Stop Discrete solutionwill have cost higherthan -80

Figure 2. Branch and bound method with solution of continuous problems

ample, distance of a continuous variable from its discrete values, and the cost function value when a variable is assigned discrete values can be used to decide the variable to be branched.

Tseng et al. (1992,1995) have also tested three different algorithms for branching a node. The idea of the branching algorithms is to avoid the middle nodes (subproblems) of the branched trees. Three algorithms suggested are: single branch algorithm, multi-branch algorithm, unbal-anced branch algorithm. In single branch procedure, only one discrete design variable is treated, and a new constraint is added to the new subproblems (as shown in Figure 2). In the multi-branch algorithm, all discrete design variable are treated and hence it generates 2n subproblems, where n is the number of discrete design variables. Unbalanced branch algorithm is a combina-tion of the first two algorithms. The first algorithm is used to generate two branches by treating only one discrete variable. One of the two branches is treated as a subproblem and using the sec-ond algorithm, another 2n-1 subproblems are created.

It is important to note that the BBM is guaranteed to find the global optimum only if the problem is linear or convex. In the case of general nonlinear nonconvex problems, there is no such guar-antee. It is possible that a node is fathomed too early and one of its branches actually contains the true global solution.

BBM with Multiple Branches

Until now, the BBM has only been used by very few researchers when dealing with linked dis-crete variables, such as the frame design problems using available sections. With such applica-tions in mind, Hager and Balling (1988) have proposed a modified BBM where multiple branches are allowed from a node. The method uses the basic concepts of branching, bounding and fathoming, and solves continuous variable optimization problems at each node. The method has several levels of nodes, the total number being equal to the number of design variables for the problem. The first level consists of a number of nodes where the first design variable is as-signed an allowable discrete value. Thus the number of nodes at the first level is equal to the number of allowable discrete values for design variable 1. Treating all other variables as con-tinuous, each node is solved using a local minimization procedure. The cost function for each node is noted, and from a node with the smallest value for the cost function, many nodes are branched. For each new node, the second design variable is also assigned a value from the avail-able ones. Therefore, the number of new nodes created from a first level node is equal to the number of allowable values for the second design variable. Each new node is optimized keeping the first two variables fixed and treating the remaining variables as continuous. If any of the nodes corresponds to an infeasible problem, it is fathomed and no further branching is necessary from there. Now all the nodes from the first and the second level are searched for the smallest cost function value. If this node is at level one, then it is branched into many new nodes; for each new node, the second variable is assigned an allowable value. If the node to be branched is at level two, then multiple nodes are branched from there, and the third design variable is fixed to an allowable value for each new node. This creates nodes at the third level and the number of nodes branched from that node is equal to the number of allowable values for the third discrete design variable. Once the new nodes are created, the local minimization procedure is used to solve for the remaining continuous variables and the cost function value for the node.

The foregoing procedure of creating new nodes and new levels is continued until a discrete fea-sible solution is obtained. Note that this will occur when some nodes have been created at all the levels. The node with a discrete feasible solution and the smallest value for the cost function gives an upper bound for all the remaining nodes at all the levels. Now all the nodes at all levels that have cost function value higher than the established upper bound are fathomed. Branching is done from the unfathomed nodes until all the remaining nodes are fathomed or a discrete solu-tion with the smaller cost function value is obtained. The new discrete solution establishes a bet-ter upper bound for the cost function. At the end of the procedure, the best discrete solution is taken as the optimum design.

The foregoing form of the BBM along with linearization of the problem has been demonstrated for steel frame optimization problems (Hager and Balling, 1988). However, the method can be applied to other discrete variable optimization problems as well. Recently, Huang and Arora (1997b) have also combined a continuous variable algorithm with BBM to solve structural opti-mization problems where the final solution has commercially available sections.

Integer Programming Optimization problems where the variables are required to take on integer values are called inte-ger programming (IP) problems. If some of the variables are continuous, then we get mixed vari-able problems. With all functions as linear, an integer linear programming (ILP) problem is ob-tained, otherwise it is nonlinear. The ILP problem can be converted to 0-1 programming prob-lem. Linear problems with discrete variables can also be converted to 0-1 programming prob-lems. Several algorithms are available to solve such problems, such as the branch and bound method discussed earlier. Nonlinear discrete problems can also be solved by sequential lineariza-tion procedures if the problem functions are continuous and differentiable.

The ILP problem can be converted to a 0-1 programming problem that can be solved using any of the several algorithms. Therefore, we first discuss transformation of ILP to a 0-1 programming problem. Consider an LP defined as follows:

minimize Tf c x= subject to Ax b xi 0 integer; i = 1, nd; xiL xi xiU; i = nd+1, n (3)

Define zij as the 0-1 variables (zij = 0 or 1 for all i and j). Then the ith integer variable can be ex-pressed as

1

iq

i ij ijj

x z d=

= ; 1ijj

z = ; 1 d i , n= (4)

Substituting this into the foregoing mixed ILP problem, it is converted as a 0-1 programming problem in terms of zij.

The branch and bound methods can be used to solve the 0-1 ILP problem. Other algorithms are (Gue et al. 1968): Balas Additive Algorithm (1965) that uses a unique value criterion to decide

if a particular variable should take on the value of 0 or 1; Glover Algorithm (1965) where the concept of a surrogate constraint is used; Balas Filter Algorithm (1967) where the earlier addi-tive algorithm is modified to incorporate the surrogate constraint concept; Lawler-Bell Algo-rithm (1966) where the cost and constraint functions are not required to be linear functions of the zero-one variables; and Geoffrion Algorithm (1967,1969) where a general framework is pre-sented to describe most branch and bound algorithms for zero-one programming problems.

It is important to note that many modern computer programs for linear programming have an op-tion to solve discrete variable LP problems; e. g., LINDO (Schrage 1983).

Sequential Linearization If functions of the problem are differentiable, a reasonable approach to solve the MD-NLP is to linearize the nonlinear problem at the current point. Then discrete-integer linear programming methods can be used to solve the linearized subproblem. There are several ways in which the lin-earized subproblems can be defined and solved. For example, the linearized subproblem can be converted to 0-1 variable problem (Templeman and Yates 1983; Ming-Zhu 1986). This way the number of variables is increased considerably; however, many methods are available to solve integer linear programming problems. Many LP codes have an option of solving such problems, so the MD-NLP can be solved using the sequential LP approach and existing codes. A modifica-tion of this approach is to obtain a continuous optimum point first, and then linearize and use in-teger-programming methods (Olsen and Vanderplaats 1989). This process can reduce the number of integer LP subproblems to be solved. Restricting the number of discrete values to be in the neighborhood of the continuous solution can also reduce the size of the LP problem.

Ghattas and Grossman (1991) formulate the discrete variable structural optimization problem as the so-called simultaneous analysis and design problem. In this formulation, the design variables and the state variables (nodal displacements, stresses) are treated simultaneously as variables of optimization. All the state equations are treated as equality constraints. The formulation results in a large scale NLP problem; however, it is quite sparse. Since all functions of the problem are ex-plicit in terms of the variables, special procedures for calculating gradients are not required, as is the case with the standard formulation. The discrete variable problem thus formulated is then converted to 0-1 programming problem and solved using an integer LP algorithm. The approach is demonstrated on two example problems of truss design. In the allowable set of discrete values for each variable, zero value is also included. This allows some of the members of the structure to be eliminated, so topology of the structure can also be optimized. The branch and bound method was also used to solve the problem and compare the results. The proposed method was more efficient.

Balling and co-workers (Hager and Balling 1988; Balling and May 1990; May and Balling 1991) have discussed the discrete variable optimization problem for design of steel frames. They first obtain a continuous optimum point using an NLP algorithm. Then approximation concepts are used to convert the problem to an explicit design problem. In one approach, the problem is con-verted to a linear program at the continuous solution. The LP problem is then solved using the branch and bound approach with the number of discrete values for each variable restricted to the nearest three or four values. After a discrete solution is obtained, a new linear approximation is

constructed and the process is repeated one more time. In another approach, the simulated an-nealing (Kincaid and Padula 1990; Balling 1991) is used after a base design is obtained and an explicit approximate problem is generated. The simulated annealing requires repeated testing of random trial designs. The trial designs are restricted to a neighborhood of the base design. The approximate analysis approach is used to evaluate the functions in the neighborhood of the base design. A smart simulated annealing approach is also presented to sieve out some undesirable designs, speeding-up the iterative process.

Bremicker et al. (1990) have also used sequential linearization followed by BBM to solve dis-crete variable structural optimization problems. The linearized discrete problem is solved by the Simplex method to obtain information at each node of the tree. The approach is compared with the pure BBM, where the SQP method is used to solve the nonlinear problem to obtain informa-tion at each node. The study showed the SLP method to be superior to the pure BBM. In another study, Loh and Papalambros (1991) propose a different procedure for solving MDNLP. In this procedure, the problem is first linearized in terms of the discrete variables only keeping the con-tinuous variables fixed at their current values. The linearized subproblem is solved using the BBM and the Simplex method. Then the discrete variables are kept fixed at their current values, and the continuous subproblem is solved using the GRG method, although other methods may also be used. Good results are reported with this procedure.

Simulated Annealing Simulated annealing (SA) is a simple stochastic technique that can be used to find global minima for continuous-discrete-integer variable nonlinear programming problems (Nemhauser and Wol-sey 1988; Aarts and Korst 1989). The approach does not require or use continuity or differenti-ability of the problem functions. The basic idea of the method is to generate random points in a neighborhood of the current best point and evaluate the problem functions there. If the cost func-tion (penalty function for constrained problems) value is smaller than its current best value, then the point is accepted, and the best function value is updated. If the function value is higher than the best value known thus far, then the point it is sometimes accepted and sometimes rejected. The acceptance is based on the value of the probability density function of Bolzman-Gibbs dis-tribution. If this probability density function has a value greater than a random number, then the trial point is accepted as the best solution even if its function value is higher than the known best value. In computing the probability density function, a parameter called the temperature is used. For the optimization problem, this temperature can be a target value for the function. Initially, a larger target value is selected. As the trials progress, the target value is reduced (this is called the cooling schedule), and the process is terminated after a large number of trials. The acceptance probability steadily decreases to zero as the temperature is reduced. Thus in the initial stages, the method accepts worse designs sometimes while in the final stages, the worse designs are almost always rejected. This strategy avoids getting trapped at a local minimum point. The deficiencies of the method are the unknown rate for reduction of the target level, uncertainty in the total num-ber of trials and the point at which the target level needs to be reduced.

The idea of simulated annealing originated in statistical mechanics (Metropolis et al. 1953). The approach is ideally suited for solving combinatorial optimization problems, such as the discrete-integer programming problems. However, first attempts in use of the method for solving such

problems (e.g., the traveling salesman problem) were in 1980s by Kirkpatrick et al. (1983) and Cerny (1985). The problem of combinatorial optimization can be related to finding a configura-tion at the ground state of a matter that occurs at very low temperatures. The ground state corre-sponds to a state with the absolute lowest internal energy. In practical context, low temperature is not a sufficient condition for finding a ground state of the matter. Experiments that determine the low temperature state of a material are done by careful annealing; for example, growing a crystal from a metal. First the material is melted, then the temperature is lowered slowly to cool the metal. This process is repeated several times. Longer time is spent at temperatures in the vicinity of the freezing point than at higher temperatures. If this is not done, i.e., long enough time is not spent at each temperature so that the equilibrium is not reached, the resulting crystal may have many defects or the material may even become glass with no crystalline order. This corresponds to metastable state and the corresponding configurations will correspond to only locally optimal structures.

The idea of combinatorial optimization and its similarity with simulated annealing in statistical mechanics becomes clear from classical traveling salesman problem: given a list of N cities and a means to calculating the cost of traveling between any two cities, one must plan the salesman route which passes through each city once and returns finally to the starting point while minimiz-ing the total cost. All exact methods known for determining an optimal route require a computing effort that increases exponentially with N. Iterative improvement, commonly applied to such problems, is much like the microscopic rearrangement process modeled by statistical mechanics. However, accepting only rearrangements that lower the cost function of the system is like ex-tremely rapid quenching from high temperatures to very low temperature. Thus what we get is a system corresponding to metastable state, i.e., a local minimum. Optimization procedures that borrow this simulated annealing idea from statistical mechanics, such as in Metropolis algorithm, provide a generalization of iterative improvement in which controlled uphill steps can also be incorporated in the search for a better solution.

Algorithm SA: Simulated Annealing

The following algorithm is based on the implementations by Kincaid and Padula (1990), and Huang and Arora (1997a).

Step 1: Choose an initial temperature T0 (expected global minimum for the function) and a feasible trial point x(0). Compute f(x(0)). Select an integer L (e.g., a limit on the number of trials to reach the expected minimum value), and a parameter r

)exp()f(pKTf = (5)

If z < p(f), then take x(k) as the new best point x(0) and go to Step 4. Otherwise

go to Step 2.

Step 4: If k < L, then set k = k+1 and go to Step 2. If k > L and any of the stopping criteria is satisfied, then stop. Otherwise, go to Step 5.

Step 5: Set K = K+1, k =1; set TK = rTK-1; go to Step 2.

The following points are noted in implementation by Huang and Arora (1997a):

1. L is suggested as 100, and r is suggested as 0.9.

2. In Step 2 only one point is generated at a time within a certain neighborhood of the cur-rent point. Thus, although SA randomly generates design points without the need for function or gradient information, it is not a pure random search within the entire design space. At the early stage, a new point can be located far away from the current point to speed up the search process and to avoid getting trapped at a local minimum point. After the temperature gets low, the new point is usually created nearby in order to focus on the local area. This is controlled by defining a step size procedure.

3. In Step 2, the newly generated point is required to be feasible. If it is not, another point is generated until feasibility is attained. Another method for treating constraints is to use the penalty function approach; i.e., the constrained problem is converted to an unconstrained one. However, the choice of a good penalty parameter can vary from one problem to an-other.

4. The following stopping criteria are suggested in Step 4: (i) The program stops if change in the best function value is less than 0.0001 for the last 4 consecutive iterations. (ii) The program stops if I/L < 0.05, where L is a limit on the number of trials (or number of fea-sible points generated) within one iteration, and I is the number of trials that satisfy f < 0 (see Step 3). (iii) The program stops if K reaches a pre-set value.

The foregoing ideas from statistical mechanics can be used to develop methods for global opti-mization of continuous variable problems (Aluffi-pentini et al. 1985). For such problems, simu-lated annealing may be combined with a local minimization procedure. However, in that case the temperature T is continuously decreased very slowly so that the effect is similar to annealing. Using the probability density function given in (a), a criterion has been developed to decide whether or not to start a local search from a particular point (Lucidi and Piccioni 1989). Re-cently, May and Balling (1992) proposed a filtered simulated annealing strategy. Based on the calculated gradient information, this method can probabilistically filter out many of the poorer candidates.

Genetic Algorithms Like the simulated annealing methods, the genetic algorithms (GA) are also in the category of stochastic search methods. They can also be classified as implicit enumeration procedures. Both the methods have their basis in natural processes. The basic ideas of the GAs can be found in Holland (1975). The methods have been applied to a variety of engineering design problems (Goldberg and Kuo 1987; Goldberg 1989; Hajela 1989; Lin and Hajela 1992; Sugimoto 1992; Huang 1995; Huang and Arora 1995, 1997a, 1997b; Huang et al 1997; Kocer and Arora 1996a, 1996b, 1997, 1999).

The basic idea of the methods is to start with a randomly generated set of design alternatives us-ing the allowable values of each variable. Each design alternative is represented by a unique fi-nite length binary string of 0s and 1s. This set of designs is called the population in a given gen-eration (iteration). Each design is also assigned a fitness value (usually the cost function or the penalty function). From the current population, a set of designs is selected randomly with a bias allocated to more fit members of the population. Random processes are used to generate a new set of designs for the next generation. The size of population of each generation is kept fixed. Since more fit members of the population are used to create new designs, the successive genera-tions have a higher probability of having designs with better fitness values. An advantage of this approach is that continuity and differentiability of functions are not required, as for the SA method.

Three operators are needed to implement a genetic algorithm: (1) reproduction; (2) crossover; and (3) mutation. Reproduction is an operator where an old design (string) is copied into the new population according to the designs fitness. More highly fit designs receive higher numbers of offspring. There are many different strategies to implement this reproduction operator. The next operator - crossover - corresponds to allowing selected members of the population to exchange characteristics of the design among themselves. Crossover entails selection of starting and end-ing positions on a pair of mating strings at random, and simply exchanging the string of 0s and 1s between these positions. Mutation is the third step that safeguards the process from a com-plete premature loss of valuable genetic material during reproduction and crossover. In terms of a binary string, this step corresponds to selection of a few members of the population, determin-ing a location on the strings at random, and switching the 0 to 1 or vice versa.

The foregoing three steps are repeated for successive generations of the population until no fur-ther improvement in the fitness is attainable. The member in this generation with the highest level of fitness is taken as the optimum design. Some details of the GA algorithm implemented by Huang and Arora (1997a) are described in the sequel.

1. Design representation. A method is needed to represent a design point using values from the allowable sets. An L-digit string with 0 or 1 for each digit is used for this purpose, where L is the total number of binary digits required to specify a design point. This is called a genetic string that is constructed by linking n binary strings together, where n is the number of design variables. A binary string of 0s and 1s represents a design variable. An m-digit binary string has 2m possible 0-1 combinations implying that 2m discrete values can be represented. The following method can be used to convert a 0-1 combina-

tion to its corresponding discrete value of a variable with nd discrete values: Calculate m

which is the smallest integer satisfying 2m > nd. Calculate the integer j:

( 1)1

( )2 1m

i

ij ICH i

=

= + ; If j > nd , ( )2d

dmd

nj INT j n

n

=

(6)

where ICH(i) is the value of the ith digit (either 0 or 1), and INT(x) is the integer part of

x. Thus the jth discrete value corresponds to this 0-1 combination. Note that when j > nd, procedures different from the one given in Eq. (6) can be used to adjust j such that j nd.

2. Random design generation. With a method to represent a design defined, the first popula-tion with Np members (genetic strings) needs to be created. These members are generated randomly via the use of a random number generator. Note that there is no need to use one random number to make decision (between 0 and 1) for each of the L digits in a genetic string. Instead, the following procedure shows a way to produce a 30 digit genetic string:

(i) Generate two double precision random numbers. (ii) Assuming the two numbers are 0.6832212546532534 and 0.1257213658623255, a string 68322125465325341257213658623255 is then created. (iii) The first 30 digits of the above string are converted to 110000010110010000110001111000 in which 0 is converted from any value in 0 and 4 and 1 from any value in 5 and 9. 3. Reproduction procedure. Reproduction is to select the more fit members of the popula-

tion into the mating pool from which members for crossover and mutation transforma-tions are selected. The ith member in the original population has a probability of selection Pi = Fi/Fj (j = 1,..., Np), where Fi is the fitness of the ith design. Figure 3 shows how a member is selected from the original population according to the probability of selection Pi. After the wheel in the figure is spun according to the value of a random number be-tween 0 and 1, the arrow at the starting position points to the member that is selected for

0

Spun by a random number

P1

P5

P4P3 P2

PNp

..

..

The second member is selected

Figure 3. Reproduction process in a genetic algorithm - selection of the mating pool

reproduction. This member is copied to the mating pool (a new population of same size, Np), but it still remains in the original population for further selection. Thus the new population may contain identical members and may not contain some members that were in the previous population. The average fitness of the new generation will be increased, however. The fitness function is defined as follows:

Fi = (1 + )fmax fi, (7)

where fi is the cost (penalty) function for the ith design, fmax is the largest recorded cost

(penalty) function value, and is a small value (e.g., 210-7) to prevent numerical diffi-culties due to Fi = 0.

4. Amount of crossover and mutation. For each generation (iteration), three operators, re-production, crossover and mutation, are performed. While the number of the reproduction operations is always equal to the size of the population, the amount of crossover and mu-tation can be adjusted to fine-tune the performance of the algorithm. In the implemented GA, the following procedure is used at each generation for crossover and mutation. First determine Imax and Im which control the amount of crossover and mutation using the following scheme:

(i) The program starts with normal rates of crossover and mutation:

Imax = 1, and Im = INT(PmNp), (8) where Pm, a fraction of the population selected for mutation, is set to 0.3 and Np is the

size of the population. It is concluded based on our experience with the test problems pre-sented later that too many crossovers result in poorer designs. Apparently, it produces new designs that are far away from the mating designs. Therefore, crossover is carried out only once between two members (Imax = 1). The mutation, however, changes designs in a neighborhood of the current design; therefore a larger amount of mutation is allowed. Note also that the population size for the present applications is selected as twice the number of design variables with minimum and maximum limits as 100 and 300, respec-tively.

(ii) Let *Kf denote the best cost (or penalty) function value among the entire population at the Kth iteration.

If the improvement on *Kf is less than (=0.0001 is used) for the last two consecutive iterations, then Imax is doubled temporarily. This doubling strategy continues at the subsequent iterations and returns to 1 as soon as *Kf is reduced. The concept behind this is that we do not want too much crossover or mutation to ruin the good genes (in genetic strings) if they keep producing better offsprings. On the other hand, we need more cross-over and mutation to trigger changes when progress stops.

(iii)If the improvement on *Kf is less than for the last Ig (Ig = 30 is used) consecutive it-erations, then Pm = 0.6 is used. The crossover and mutation are performed as follows:

FOR i = 1, Imax Generate a random number z uniformly distributed in [0,1] If z > 0.5, perform crossover as follows:

Randomly select two mating parents from the mating pool, and randomly choose two sites on the genetic strings and swap strings of 0s and 1s between the two chosen sites; e.g., 010110 and 110001 become 010000 and 110111.

If z 0.5, skip crossover. FOR j = 1, Im Generate a random number z uniformly distributed in [0,1] If z > 0.5, perform mutation as follows:

Randomly choose a member from the mating pool and switch a 0 to 1 or vice versa at a randomly selected site; e.g., 110111 becomes 110011.

If