Computational Intelligence Based Methodologies for Modeling and

Optimization

Somnath Nandi

Asst. ProfessorDept. of Petroleum and Petrochemical Engineering

MIT - Pune

Contents

• Modeling• Artificial Neural Networks• Optimization• Genetic Algorithms• Differential Evolution• Case Study• Conclusion

Modeling

• Engineers and scientists required to analyze the complex processes and develop mathematical models which simulate their steady-state and / or dynamic behavior.

• The objective is to construct, from theoretical and empirical knowledge of the process, a mathematical description.

• A mathematical model provides information on the process behavior, over important ranges of operating variables, in terms of equations, which reflects at least the major features of the underlying mechanisms.

Modeling (contd… )

• Phenomenological Approach:

- Process behavior described in terms of the appropriate mass, momentum and energy balance equations together with the pertinent chemical engineering principles.

- Mathematical formulation describing the physico-chemical phenomena underlying in the process is formulated followed by model fitting.

- Regression techniques based on the least squares minimization

Modeling (contd… )• Advantages :

- It provides a valuable insight into the process behavior- It possesses extrapolation ability

• Disadvantages:- Owing to the complex nature of many processes, the underlying physico-chemical phenomenon is seldom fully understood- Collection of the requisite phenomenological information is costly, time-consuming and tedious- Nonlinear behavior common for many processes leads to complex nonlinear models, which in most cases are not amenable to analytical solutions; thus, computationally intensive numerical methods must be utilized for obtaining solutions

Modeling (contd… )• Empirical Approach:

- Process behavior is modeled using appropriately chosen empirical equations, for instance, polynomial expressions.

- Model can be constructed solely from the process input-output data without explicitly invoking the process phenomenology.

- An appropriate functional form that possibly fits the process data is selected in advance following which the unknown model parameters are estimated using a suitable function fitting procedure.

Modeling (cond… )• Artificial Intelligence based Approach:

- AI is science and engineering of making “intelligent” systems, especially intelligent computer programs

- Related to task of using computers to understand the “human intelligence”

- Intelligence can be broadly defined as computational part of our ability to efficiently achieve goals in the world.

AI based Modeling Approaches

• Artificial Neural Networks (ANN)

• Support Vector Regression (SVR)

• Genetic Programming (GP)

• Fuzzy Logic (FL)

Artificial Neural Networks• Efforts to develop computer models of the

information processing of human nervous system (Rumelhart et. al., 1986).

• Simplified mathematical models describing the biological nervous system and functioning.

• A highly interconnected system of simple processing elements can learn complex interrelationships between the independent and the dependent variables in a data set.

Artificial Neural Networks

1 2 3 NO

1 2 NH

1 2 3 NI

Output layer

Hidden layer

Input layer

Inputs

Outputs

x1 x2 x3 xNI

y1 y2 y3 yNO

Artificial Neural Networks

Kkfy kkk ...,2,,1 ,, wx

P

i

K

k

pkiki yyRMSE

1 1

2

,,ˆ

Input x2

x1

xN

Outputy1

yK

Artificial Neural Networks• The distinct advantages of the ANN formalism

are:

– Can be developed solely from process input-output data.

– MIMO relationships can be approximated– Possesses good generalization ability– Can tolerate noisy data or incomplete information– Can be developed even using qualitative data.– Use a generic nonlinear function for function

approximation and thus there is no need to specify system-specific data fitting function as done in traditional regression.

Artificial Neural Networks

• Principal applications of ANNs are: (i) nonlinear function approximation (i.e., process modeling), (ii) pattern recognition and classification, (iii) data reduction and compression, (iv) signal processing, (v) noise reduction.

What is Optimization ?• Optimization is use of specific methods to

determine the most cost-effective and efficient solution to a problem or design for a process

• A wide variety of problems in the design, construction, operation, and analysis of industrial processes can be resolved by optimization

• The field of statistics treats various principles termed "maximum likelihood," "minimum loss," and "least squares," and business makes use of "maximum profit," "minimum cost," "maximum use of resources," "minimum effort," in its efforts to increase profits

What is Optimization ?• A typical engineering problem can be posed as

follows: a process can be represented by some equations or perhaps solely by experimental data. You have a single performance criterion in mind such as minimum cost

• The goal of optimization is to find the values of the variables in the process that yield the best value of the performance criterion

• A trade-off usually exists between capital and operating costs. The described factors-process or model and the performance criterion-constitute the optimization "problem."

What is Optimization ?

• Optimization is minimization or maximization of an objective function (also called a performance index or goal function) that may be subject to certain constraints

min f (x) Goal function

subject to, g (x) = 0 Equality constraints

h (x) < 0 Inequality constraints

Need for Optimization• Typical problems in engineering process design or

plant operation have many (possibly an infinite number) solutions

• Optimization is concerned with selecting the best among the entire set by efficient quantitative methods

• Computers and associated software make the necessary computations feasible and cost effective

• To obtain useful information using computers, however, requires

(1) critical analysis of the process or design, (2) insight about what appropriate performance objectives are (what is to be accomplished), (3) use of past experience, sometimes called

engineering judgment.

Applications of Optimization• Determining the best sites for plant location• Routing tankers for the distribution of crude and

refined products• Sizing and layout of a pipeline• Designing equipment and an entire plant• Scheduling maintenance and equipment replacement• Operating equipment, such as tubular reactors,

columns, and absorbers• Evaluating plant data to construct a model of a

process• Minimizing inventory charges• Allocating resources or services among several

processes• Planning and scheduling construction

1-Dimensional Search

2-Dimensional Search

2-Dimensional Search

2-Dimensional Search

Unimodal Optimization

Multi-modal Optimization

A function exhibiting different types of stationary points. a-inflection point (scalar equivalent to a saddle point); b-global maximum; c-local minimum; d-local maximum

Global Methods of Optimization

Performance of Classical Techniques

Multiobjective Optimization• A MOO problem will have two or more objectives

involving many decision variables and constraints• Consider an MOO problem with two objectives:

f1(x) and f2(x), and several decision variables (x)

Minimize f1(x) (1)Minimize f2(x) (2)

With respect to xSubject to xL ≤ x ≤ xU (3)h (x) = 0 (4)g (x) ≤ 0 (5)

Multiobjective Optimization

Different Evolutionary Techniques

• Genetic Algorithms (GA)• Simulated Annealing (SA)• Ant Colony Optimization (ACO)• Tabu Search (TS)• Particle Swarm Optimization (PSO)• Differential Evolution (DE)• Memetic Algorithm (MA)• Simultaneous Perturbation Stochastic

Approximation (SPSA)

What is GA ?

• GAs are computer based search and optimization algorithms based on mechanics of natural genetics and natural selection

• A population of initial solution is generated within feasible region

• The main idea is - Survival of the fittest- Evolution of species with time

• Only best solution will survive till end

What is GA ?

• Genetic Algorithms (GAs) were invented by John Holland and developed by him and his students and colleagues. This lead to Holland's book "Adaptation in Natural and Artificial Systems" published in 1975.

• All living organisms consist of cells. In each cell there is the same set of chromosomes

• A chromosome consists of genes, blocks of DNA. Each gene encodes a particular protein

• Complete set of genetic material (all chromosomes) is called genome.

• Particular set of genes in genome is called genotype.

Working Principle

Let us consider the maximization problem:

• Coding: - Variable xi are first coded into binary

strings - Length of string is determined based on

desired accuracy of solution

nixxxxf Uii

Li ,...,2,1,,Maximize

TTUU

TTLL

xx

xx

11111111,

00000000,

21

21

Working Principle

• Fitness function - GA are based on survival-of-the-fittest - Naturally suitable for solving maximization

problems - Minimization are transformed to suitable maximization ones - Fitness function is a measure of goodness of

the string - Our target is to keep on increasing the overall

fitness functions of all the strings - Genetic operators perform duty to manipulate

binary strings so that fitness function is keep on increasing on successive iterations

Working Principle

• GA Operators- Reproduction / Selection

Selects good strings of a population Forms a mating pool Above – average stings are picked from

current population Multiple copies of selected strings are placed

in mating pool in a probabilistic manner No new strings are formed in this phase Roulette – Wheel or Stochastic Remainder

Selection methods

Working Principle• Crossover

- New strings are created

- It exchanges information among strings of mating pool - 2 strings are picked at random

- Point of crossover is probabilistically

chosen 0 0 0 1 1 1 0 0 0 0 1 0

1 1 1 0 1 0 1 1 1 1 1 1

Parent Strings Children Strings

Working Principle

• Mutation- It changes 1 to 0 and vice versa

- Small probability pm generally < 0.1

- Need is to create a point in the neighborhood of the current point

- Performs local search around current solution

- It maintains diversity of population0 0 0 1 1 1 0 0 0 0 1 1

1 1 1 0 1 0 1 1 0 0 1 0Mutation

Diversification

• Generate initial population covering entire range• Visit new places• Extract characteristics of each region• Cover as much as possible• Performing Global Search• All are done by Crossover operator

Intensification

• Should be started once search space is well scanned• Visit zones adjacent / nearby to already visited • Check the performance• Perform local search• This is done by Mutation operator

Algorithm• Step 1: Do coding, choose selection operator, crossover

and mutation probability (pc and pm). Choose population size (n), string length (l), max. no. of iterations (Nmax)

• Step 2: Evaluate each string of population

• Step 3: Perform Reproduction on population

• Step 4: Perform crossover on random pairs of strings

• Step 5: Perform mutation on each string

• Step 6: Evaluate strings of new population

• Step 7: Set N = N + 1 and go to step 3

• Terminate if N > Nmax or no further improvement on string performance

Advanced GA• Multi Point Crossover :

• Real Coded GA :- Real variables are directly used- Optimal point of any desired accuracy obtained

• Non – dominated Sorting :- To keep versatility of population- Give more chance to a poor performer to enhance its skills

• Pareto GA :- Population in a GA simulation is adaptively divided into separate

subpopulation, corresponding to each optimum point by use of sharing functions- Can get all the solutions of Pareto Optimal front in one shot

1 1 0 0 0 1 1 0 1 1 0 1 1 0 1 0

1 0 0 1 1 0 0 1 1 0 0 0 0 1 0 1

GA - Applications

• Reactor Design – Ammonia Synthesis

• Process Optimization

– Cumene Synthesis

- Phenol Production

• Scheduling – Refinery Operations

• Multiphase – Trickle Bed Reactor

• Polymerization Processes

– MMA Synthesis

- Polyethylene Plant

- Nylon Manufacture

• Water Distribution

Differential Evolution

• Introduced by Storn and Price in 1996• Algorithm works with a population of size N• Algorithm iterates as follows:

- Generate new vector by adding weighted difference of two vectors to third

- Mix new vector with target vector to yield trial vector

- Replace target vector with trial vector if latter is strictly superior

Differential Evolution

Differential Evolution

• F and CR are DE control parameters• F is a real-valued factor in the range

(0.0,1.0+]• Upper limit on F has been empirically

determined.• CR is a real-valued crossover factor in

range [0.0,1.0]• CR controls the probability that a trial

vector parameter will come from the randomly chosen noise vector

Importance of Parameters

• Optimal values are dependent both on objective function characteristics and on the population size, NP

• Practical advice on how to select control parameters NP, F and CR can be found in the literature

Crossover in DE

DE - Applications

• Multiprocessor synthesis• Power minimisation • Neural network learning. • Crystallographic characterization• Design of Shell-and-Tube Heat Exchangers• Heat transfer parameter estimation in a trickle

bed reactor• Gas Transmission Network• Water Pumping and Distribution Systems• Optimization of Ammonia Synthesis Reactor• Design and Operation of Thermal Cracker

DE - Advantages

• Powerful algorithm- multidimensional functions• Easy applicable to various problems.• Widely used• Literature and other materials available• Generally good accuracy for real world problems• Easy to implement as same parameter settings

work fine for a wide range of problems

• Drawback : Somewhat slow during initial iterations

Cumene Synthesis• Main reaction :

Benzene + Isopropyl Alcohol Cumene + Water (benzene alkylation)

• Secondary reactions :

Cumene + Isopropyl Alcohol p-Di-isopropyl Benzene + Water

(cumene alkylation)

p-Di-isopropyl Benzene m-Di-isopropyl Benzene (isomerization)

2 Isopropyl alcohol Di-isopropyl ether + Water (alcohol dehydration)

Cumene Synthesis

Catalyst• Beta is a crystalline alumino-silicate catalyst with

high silica content • Important characteristic is that it is the only large

pore zeolite with chiral pore intersections • It consists of 12-membered rings interconnected by

cages formed by intersecting channels • The linear channels have pore opening dimensions

of 5.7 7.5 Å• the tortuous channels with intersections of two linear

channels have approximate dimensions of 5.6 6.5 Å

• The catalyst has pore volume of 0.2 cm3/g. • Beta catalyst (1.5 mm extrudates with 20 % binder)

in its active protonated form with Si to Al ratio of 15 was obtained from M/s UCIL, India

Reactor• Vapor phase isopropylation of benzene was carried

out in a pilot plant scale stainless steel reactor • A preheater in its upstream and a condenser in the

down-stream • Material of construction: SS 316, • Internal diameter (ID): 25 mm • Wall thickness: 6 mm • Reactor length: 33 cm • Catalyst bed height: 10-15 cm • Heating coils are wound around the reactor to

provide proper heating and maintain temperature • Reactor is also jacketed with insulation to minimize

the heat loss

The Operation

• The liquid mixture of benzene and isopropyl alcohol was fed to the reactor by a positive displacement pump

• Hydrogen was used as the carrier gas • The condensed products collected were

analyzed with a Flame Ionization Detector (FID) using a “Xylene Master” capillary column fitted to a Shimadzu 15A Gas Chromatograph (GC)

Process Parametrs

• Important Operating Variables– reaction temperature (x1)

– pressure (x2)

– benzene to isopropyl alcohol mole ratio (x3)

– weight hourly space velocity (WHSV) (x4)

unit timeper fed alcohol isopropyl ofweight

unit timeper formed cumene ofweight 100 1

y

• Outputs are: Cumene yield and selectivity y1, y2

unit timeper produced aromatics totalofweight

unit timeper formed cumene ofweight 100 2

y

Expt. No.

Temperature ( 0C)

Pressure (atm.) Benz/IPA(mole ratio)

WHSV (hr-1)

Yield( wt %)

Selectivity( wt %)

1 110 1 8 3.3 0.07 77.03

2a 145 1 8 3.3 11.6 58.75

3 180 1 8 3.3 15.78 79.93

4 210 1 8 3.3 17.365 90.72

5 215 1 8 3.3 16.09 91.95

6 150 4 8 3.3 12.2 65.74

7 135 4 8 3.3 12.99 74.58

8a 110 4 8 3.3 0.71 80.82

9 100 4 8 3.3 0.19 75.02

10 110 1 10 3.3 0.55 67.74

11 110 1 8 3.3 0.24 54.85

12a 110 1 6 3.3 0.37 53.63

13 110 1 3 3.3 0.2 32.13

14 110 1 1 3.3 0.14 21.62

15 110 1 8 6.8 0.24 54.85

16 110 1 8 8 0.15 44.64

17 110 1 8 9.5 0.13 37.38

18 110 1 8 10.5 0.08 39.3

19a 110 1 8 12 0.09 39.13

20 110 1 8 13 0.07 39.1

21 105 1 8 6.8 0.3 70.38

22 110 1 8 6.8 0.24 54.85

23 115 1 8 6.8 0.35 48.25

24 130 1 8 6.8 4.61 76.68

25 185 1 8 6.8 9.2 59.23

26 210 1 6.5 3.3 20.04 91.8

27 155 1 6.5 3.3 16.93 77.4

28a 180 1 6.5 3.3 20.27 90.9

29 210 1 6.5 3.3 19.86 91.9

30 225 1 6.5 3.3 19.1 89.3

31 250 1 6.5 3.3 17.89 85.2

32 275 1 6.5 3.3 17.29 83.1

33 230 1 6.5 2.5 20.33 91.1

34 215 1 7 5 19.86 91.9

35a 215 10 7 5 19.54 92

36 215 18 7 5 18.68 89.1

37 215 25 7 5 17.74 86.8

38 195 25 6 5 18.92 85.6

39 210 25 6 5 22.1 93.7

40 230 25 6 5 22.02 93.8

41 250 25 6 5 21.35 90.7

42 280 25 6 5 20.48 86.2

Expt. No.

Temperature ( 0C)

Pressure (atm.) Benz/IPA(mole ratio)

WHSV (hr-1)

Yield( wt %)

Selectivity( wt %)

1 110 1 8 3.3 0.07 77.03

2a 145 1 8 3.3 11.6 58.75

3 180 1 8 3.3 15.78 79.93

4 210 1 8 3.3 17.365 90.72

5 215 1 8 3.3 16.09 91.95

6 150 4 8 3.3 12.2 65.74

7 135 4 8 3.3 12.99 74.58

8a 110 4 8 3.3 0.71 80.82

9 100 4 8 3.3 0.19 75.02

10 110 1 10 3.3 0.55 67.74

11 110 1 8 3.3 0.24 54.85

12a 110 1 6 3.3 0.37 53.63

13 110 1 3 3.3 0.2 32.13

14 110 1 1 3.3 0.14 21.62

15 110 1 8 6.8 0.24 54.85

16 110 1 8 8 0.15 44.64

17 110 1 8 9.5 0.13 37.38

18 110 1 8 10.5 0.08 39.3

20 110 1 8 13 0.07 39.1

21 105 1 8 6.8 0.3 70.38

22 110 1 8 6.8 0.24 54.85

23 115 1 8 6.8 0.35 48.25

24 130 1 8 6.8 4.61 76.68

25 185 1 8 6.8 9.2 59.23

26 210 1 6.5 3.3 20.04 91.8

27 155 1 6.5 3.3 16.93 77.4

28a 180 1 6.5 3.3 20.27 90.9

29 210 1 6.5 3.3 19.86 91.9

30 225 1 6.5 3.3 19.1 89.3

31 250 1 6.5 3.3 17.89 85.2

32 275 1 6.5 3.3 17.29 83.1

33 230 1 6.5 2.5 20.33 91.1

34 215 1 7 5 19.86 91.9

35a 215 10 7 5 19.54 92

36 215 18 7 5 18.68 89.1

37 215 25 7 5 17.74 86.8

38 195 25 6 5 18.92 85.6

39 210 25 6 5 22.1 93.7

40 230 25 6 5 22.02 93.8

41 250 25 6 5 21.35 90.7

42 280 25 6 5 20.48 86.2

Modeling of Output vs. Input

0

5

10

15

20

25

0 5 10 15 20 25 30 35 40 45

Expt. No.

Yie

ld (

wt

%)

Predicted

Experimental

0

20

40

60

80

100

0 5 10 15 20 25 30 35 40 45

Expt. No.

Sele

cti

viy

ty (

wt

%)

Predicted

Experimental

Optimization

• Best values of following GA-specific parameters were chosen heuristically:

- population size (Npop) = 25 - crossover probability (pcross) = 0.82 - mutation probability (pmut) = 0.05 - maximum number of generations (Ngen) =

100 • In order to obtain the best set of operating

conditions, GA runs were replicated several i.e. 50 times, using different random number generator seeds.

• The fitness function: popj Njwwyyywyw

..., 2, ,1;5.0ˆˆ;200100

ˆˆˆ21

212211

Optimized ResultsSoln.No.

ANN-GA

Optimized Inputs Maximized Outputs

Temp.(0 C)(x1

*)

Press.(atm.)(x2

*)

Benz/IPA(mol ratio)

(x3*)

WHSV (hr-1)(x4

*)Yield

(wt %)(y1

*)

Selectivity(wt %)

(y2*)

1 271.5 3.38 3.69 12.83 24.88 99.04

2 267.2 1.567 4.05 12.83 24.84 98.90

3 270.08 3.6 4.05 11.76 24.82 98.74

Experimental VerificationExp.No.

Experimental Conditions Yield (output 1) Selectivity (output 2)

Temp.(0C)

Pressure(atm)

Benz/IPA(mole ratio)

WHSV(hr-1)

GA-maximized

value (wt %)

Exptl.value

(wt %)

Error(%)

GA-maximized

value (wt %)

Exptl.value

(wt %)

Error(%)

1 271.5 3.4 3.7 12.8 24.88 24.69 0.77 99.04 98.98 0.06

2 267.2 1.6 4.0 12.8 24.84 23.79 4.41 98.90 98.70 0.20

3 270.0 3.6 4.0 11.8 24.82 24.58 0.98 98.74 98.65 0.09

Published in Chemical Engineering Journal, Vol. 97, No. 2 – 3, pg: 115 – 129 (2004)

Benefit of the Study

• The work extended from pilot plant level to commercial scale

• Implemented successfully by HPCL• Overall profit increased by almost 18 %• Some more research work with HPCL and

others regarding their multiphase operations

• Leads to optimization of Polypropylene Production unit of Reliance at their Hazira plant

Overall Conclusion

• Modeling and various approaches discussed

• ANN-based modeling introduced• Optimization and its necessity• Multi-objective optimization• Genetic Algorithm methodology• Differential Evolution – a novel

method• Cumene synthesis – case study

References• Rumelhart, D., Hinton, G., and Williams, R. “Learning Representations by Backpropagating

Errors”, Nature, 323, 533 - 536 (1986).• Deb, K. “Optimization for Engineering Design: Algorithms and Examples”, Prentice Hall of India,

New Delhi (2006)• Deb, K., “Multiobjective Optimization Using Evolutionary Algorithms”, Wiley, Chichester, UK

(2001)• Nandi, S. Ghosh, S. Tambe S and Kulkarni, B. D. “Artificial Neural Network Assisted Stochastic

Process Optimization Strategies”, AIChE J, Vol. 47, pp. 126-141 (2001).• Nandi, S. Mukherjee, P, Tambe, S. S. , Kumar, R. and Kulkarni, B. D. “Reaction Modeling and

Optimization Using Neural Networks and Genetic Algorithms: Case Study Involving TS-1 Catalyzed Hydroxylation of Benzene”, Ind. Engg. Chem. Res., Vol. 41, pp. 2159-2169 (2002).

• Nandi, S., Badhe, Y. , Lonari, J. B., Sridevi, U., Rao, B. S. Tambe, S. S., Kulkarni, B. D. “Hybrid Process Modeling and Optimization Strategies Integrating Neural Networks/Support Vector Regression and Genetic algorithms: Study of Benzene Isopropylation on HBeta Catalyst”, Chem. Engg. Jour. Vol. 97, pp. 115-129 (2004).

• Price, K.V. (1999). An Introduction to Differential Evolution. In: Corne, D., Dorigo,M. and Glover, F. (eds.) (1999). New Ideas in Optimization, pp. 79–108. McGraw-Hill, London. ISBN 007-709506-5.

• Storn, R. and Price, K.V. (1995). Differential evolution - a Simple and Efficient Adaptive Scheme for Global Optimization Over Continuous paces. Technical Report TR-95-012, ICSI, March 1995. Available via Internet: ftp://ftp.icsi.berkeley.edu/pub/techreports/1995/tr-95-012.ps.Z .

• Storn, R. and Price, K.V. (1997). Differential Evolution – a Simple and Efficient Heuristic for Global Optimization over Continuous Spaces. Journal of Global Optimization, 11(4): 341–359, December 1997. Kluwer Academic Publishers.