ARTICLE IN PRESS

0952-1976/$ - se

doi:10.1016/j.en

�CorrespondE-mail addr

konaramit@ya

Engineering Applications of Artificial Intelligence 20 (2007) 1086–1096

www.elsevier.com/locate/engappai

Brief paper

A swarm intelligence approach to the synthesis oftwo-dimensional IIR filters

Swagatam Das�, Amit Konar

Department of Electronics and Telecommunication Engineering, Jadavpur, University, Raja S.C. Mullick Road, Kolkata 700032, India

Received 21 September 2005; received in revised form 4 October 2006; accepted 3 February 2007

Available online 18 April 2007

Abstract

The concept of particle swarms, although initially introduced for simulating human social behaviors, has become very popular these

days as an efficient means for intelligent search and optimization. The particle swarm optimization (PSO), as it is called now, does not

require any gradient information of the function to be optimized, uses only primitive mathematical operators and is conceptually very

simple. This paper investigates a novel approach to the designing of two-dimensional zero phase infinite impulse response (IIR) digital

filters using the PSO algorithm. The design task is reformulated as a constrained minimization problem and is solved by a modified PSO

algorithm. Numerical results are presented. The paper also demonstrates the superiority of the proposed design method by comparing it

with two recently published filter design methods and two other state of the art optimization techniques.

r 2007 Elsevier Ltd. All rights reserved.

Keywords: IIR filters; Swarm intelligence; Particle swarm optimization; Differential evolution; Genetic algorithms

1. Introduction

In signal processing, the function of a filter is to removeunwanted parts of a signal, such as random noise, or toextract useful parts of the signal, such as the componentslying within a certain frequency range. There are two mainkinds of filter, analog and digital. They are quite differentin their physical makeup and in how they work. An analogfilter uses analog electronic circuits made up fromcomponents such as resistors, capacitors and op-amps toproduce the required filtering effect. Such filter circuits arewidely used in applications like noise reduction, videosignal enhancement, graphic equalizers in hi-fi systems, andin many other areas. A digital filter uses a digital processorto perform numerical calculations on sampled values of thesignal. The processor may be a general-purpose computersuch as a PC, or a specialized digital signal processor (DSP)chip.

e front matter r 2007 Elsevier Ltd. All rights reserved.

gappai.2007.02.004

ing author. Tel.: +91(033)2528 2717.

esses: swagatamdas19@yahoo.co.in (S. Das),

hoo.co.in (A. Konar).

Digital filters are broadly classified into two maincategories namely, finite impulse response (FIR) filters andinfinite impulse response (IIR) filters (Oppenheim et al.,1999; Proakis and Manolakis, 1996). An FIR filter is onewhose impulse response is of finite duration. The output ofsuch a filter is calculated solely from the current andprevious input values. This type of filter is hence said to benon-recursive. On the other hand, an IIR filter is one whoseimpulse response (theoretically) continues for ever in time.They are also termed as recursive filters. The current outputof such a filter depends upon previous output values.These, like the previous input values, are stored in theprocessor’s memory. The word recursive literally means‘‘running back’’, and refers to the fact that previouslycalculated output values go back into the calculation of thelatest output. The recursive (previous output) terms feedback energy into the filter input and keep it going.FIR filters are generally easier to implement, as they are

non-recursive and always stable (by definition). On theother hand, it is much more difficult to obtain linear phaseresponses and to control the overall frequency responseswith IIR filters. However, very sharp (narrow transition

ARTICLE IN PRESSS. Das, A. Konar / Engineering Applications of Artificial Intelligence 20 (2007) 1086–1096 1087

band) frequency responses can be easily realized with IIRfilters and this feature makes them suitable for a broadrange of applications. The key practical benefit of utilizingIIR filters is that they are much more efficient than FIRsystems, which require higher orders (i.e. more multi-plications) (Seppanen, 1999). Overall, when properlydesigned, IIR filters are capable of having more accurateresponses than those yielded by FIR filters (Proakis andManolakis, 1996). Since they are more challenging toimplement and allow for higher degrees of precision andpractical efficiency, we have selected IIR filter design as thefocus of our proposed methodology. This is consistent withour overall objective of improving the process by whichcomplicated filters are designed for efficient digitalimplementation. Moreover, our research concentrates onzero-phase IIR filters, which are the most commonly usednon-causal filters. The concept of zero-phase filtering hasbeen explained briefly in the next section.

Two-dimensional (2D) IIR filters find extensive applica-tions in the domain of denoizing of digital images,biomedical imaging and digital mammography, X-raysimage enhancement, seismic data processing, etc. (Kaczor-ek, 1985; Tzafestas, 1986; Lu and Antoniou, 1992). Multi-2D filter mathematics can be also used in grid methods forsolving partial differential equations, distributed control,and iterative learning control.

The most popular design methods for 2D IIR filters arebased either on an appropriate transformation of onedimensional (1D) filter (Tzafestas, 1986; Lu and Antoniou,1992) or on appropriate optimization techniques (Laaskoand Ovaska, 1994; Hsieh et al., 1997; Daniel and Willsky,1997). One of the major problems underlying the designtask is to satisfy the stability criterion for the filter transferfunction. Although researchers have attempted to tacklethe stability problem in a number of ways, most of theseefforts (Kaczorek, 1985; Tzafestas, 1986; Lu and Antoniou,1992; Laasko and Ovaska, 1994; Hsieh et al., 1997) resultedin a filter having a very small stability margin with hardlyany practical importance (Mladenov and Mastorakis,2001). The application of evolutionary computationtechniques to the design of digital IIR filters can be tracedto the work of Gorne and Schneider (1993). Chellapilaet al. (1996) employed an evolutionary programming tooptimize the coefficients of the transfer function of a 1DIIR filter. Schiner et al. used an evolutionary multi-objective optimization approach to the design of IIRprototype filters (Kalinli and Karaboga, 2005). Veryrecently, Kalinli et al. proposed an artificial immunesystem-based methodology for the design of IIR filters(Schnier et al., 2004).

‘Swarm Intelligence’ is the name given to a relatively newinterdisciplinary field of research, which has gained a widepopularity in recent times. Algorithms belonging to thisfield, draw inspiration from the collective intelligenceemerging from the behavior of a group of social insects(like bees, termites and wasps). These insects even with verylimited individual capability can jointly (cooperatively)

perform many complex tasks necessary for their survival.Particle swarm optimization (PSO) (Kennedy and Eber-hart, 1995) is a swarm-intelligence-based global optimiza-tion technique over continuous search spaces. Since itsadvent in 1995, PSO has attracted the attention of severalresearchers all over the world resulting into a huge numberof variants of the basic algorithm as well as manyparameter automation strategies. To the best of ourknowledge, PSO has not been applied to multi-dimensionalfilter design problem until date.The contribution of this paper is to present a consistent

engineering approach to formal specification-driven optimaldesign of 2D IIR filters with PSO. In this work, the designtask of 2D recursive filters is formulated as a constrainedoptimization problem. The stability criterion is presented asconstraints to the minimization problem. Thus, a generalizedfilter design framework has been illustrated, which is suitablefor application of any global optimization algorithm. Wemodified the PSO from its classical form (by incorporating amulti-elitist strategy) in order to improve its convergenceproperties over difficult fitness landscapes. The effectivenessof the modified version has been pointed out in our earlierwork reported in Das et al. (2006).We also employed the differential evolution (DE) (Storn

and Price, 1997) and a real coded genetic algorithm (GA)(Goldberg, 1989; Deb et al., 2002a) to the same problembased on this framework. Numerical results show that thePSO-based method yields a better approximation to thetransfer function of the filter as compared to the GA orDE-based methods and also beats the state of the art workspresented in Mladenov and Mastorakis (2001), Mastorakiset al. (2003) in a statistically significant manner. Comparedto the GA-based methods, the PSO algorithm used here iseasier to implement and requires fewer function evalua-tions to find an acceptable solution.The remainder of this paper is organized as follows.

Section 2 provides an overview of the filter designmethodology and its reformulation as a constrainedminimization problem. In Section 3, the PSO algorithm ispresented briefly and its application to the present problemis described in Section 4. Section 5 presents the resultsof applying the proposed method to a specific designproblem and also provides performance comparisons withother optimization techniques. The results are discussedand explained in Section 6 and finally in Section 7 thepaper is concluded.

2. Formulation of the design problem

In this work, the filter design is mainly considered from afrequency domain perspective. Frequency domain filteringconsists in first, taking the Fourier transform of the 2Dsignal (which may be the pixel intensity value in case of agray-scale image), then multiplying the frequency domainsignal by the transfer function of the filter and finallyinverse transforming the product in order to get the outputresponse of the filter. We illustrate the scheme in Fig. 1.

ARTICLE IN PRESS

Fourier

Transform

Transfer

function

H(ω1, ω2)

Inverse

Fourier

Transform

Pre-

processing

Post-

processing

f (x,y)

Input signal/

image

g(x, y)

Output signal/

image

F(ω1, ω2) H(ω1, ω2)F(ω1, ω2)

Fig. 1. Scheme of filtering with 2D digital filter.

S. Das, A. Konar / Engineering Applications of Artificial Intelligence 20 (2007) 1086–10961088

In this work, we are mainly interested in zero-phase IIRfilter design. A zero-phase filter is a special case of a linear-phase filter in which the phase slope is zero. The realimpulse response h(n) of a zero-phase filter is even [i.e. itsatisfies h(n) ¼ h(�n)]. Note that a zero-phase filter cannotbe causal. However, in many off-line applications, such aswhen filtering a sound file or an image file stored on acomputer disk, causality is not a requirement and zero-phase filters are usually preferred. Let the general proto-type 2D transfer function for the zero-phase IIR digitalfilter be

Hðz1; z2Þ ¼ H0

PNi¼0

PNj¼0pijz

izj

PNk¼1ð1þ qkz1 þ rkz2 þ skz1z2Þ

. (1)

The variables z1 and z2 can be interpreted as complex in-determinants in the discrete Laplace transform (z-trans-form). The relation ship of z1 and z2 with Fourier domainfrequency terms o1 and o2 are given by

z1 ¼ ejo1 ,

z2 ¼ ejo2 . ð2Þ

It is a general practice to take p00 ¼ 1(by normalizingpij’s with respect to the value of p00). Also, let us assumethat the user-specified amplitude response of the filter to bedesigned is Md, which is obviously a function of digitalfrequencies o1 and o2 (o1, o2A[0, p]). Now the maindesign problem is to determine the coefficients in thenumerator and denominator of Eq. (1) in such a fashionthat H(z1, z2) follows the desired response Md (o1, o2) asclosely as possible. Such an approximation of the desiredresponse can be achieved by minimizing

Jðpij ; qk; rk; sk;H0Þ ¼XN1

n1¼0

XN2

n2¼0

Mðo1;o2Þ�� ���Mdðo1;o2Þ� �b

,

(3)

where

Mðo1;o2Þ ¼ Hðz1; z2Þ��z1 ¼ ejo1

z2 ¼ ejo2

(4)

and

o1 ¼ ðp=N1Þn1; o2 ¼ ðp=N2Þn2

and b is an even positive integer (usually b ¼ 2 or 4).Eq. (3) can be restated as

J ¼XN1

n1¼0

XN2

n2¼0

Mpn1

N1;pn2

N2

� ����������Md

pn1

N1;pn2

N2

� �� �b

. (5)

Here, the prime objective is to reduce the differencebetween the desired and actual amplitude responses of thefilter at N1�N2 points. For bounded input boundedoutput (BIBO) stability the prime requirement is that the z-plane poles of the filter transfer function should lie withinthe unit circle. Since the denominator contains only firstdegree factors, we can assert the stability conditions,following (Kaczorek, 1985, Mastorakis et al., 2003; Luand Antoniou, 1992) as

qk þ rk

�� ��� 1osko1� qk � rk

�� ��, (6)

where k ¼ 1; 2; . . . ;N.Thus the design of a 2D recursive filter is equivalent to

the following constrained minimization problem:Minimize J

¼XN1

n1¼0

XN2

n2¼0

Mpn1

N1;pn2

N2

� ����������Md

pn1

N1;pn2

N2

� �� �b

(7a)

subject to the constraints

qk þ rk

�� ��� 1osk; k ¼ 1; 2; . . . ;N;

sko1� qk � rk

�� ��; k ¼ 1; 2; . . . ;N;(7b)

where N1, N2 and N are all positive integers. In Mladenovand Mastorakis (2001), the design problem has beentackled with neural networks and the work in Mastorakiset al. (2003) attempts to solve it using a binary coded GA.In the present paper, a much better solution has beenobtained using the modified PSO algorithm.

ARTICLE IN PRESSS. Das, A. Konar / Engineering Applications of Artificial Intelligence 20 (2007) 1086–1096 1089

3. The classical PSO and its modification

PSO (Kennedy and Eberhart, 1995; Kennedy andEberhart, 2001; Shi and Eberhart, 1998) is in principlesuch a multi-agent parallel search technique. Particles areconceptual entities, which fly through the multi-dimensionalsearch space. At any particular instant each particle has aposition and a velocity. The position vector of a particlewith respect to the origin of the search space represents atrial solution of the search problem. At the beginning apopulation of particles is initialized with random positionsmarked by vectors ~X i and random velocities ~V i. Thepopulation of such particles is called a ‘swarm’ S. Aneighborhood relation N is defined in the swarm. N

determines for any two particles Pi and Pj whether theyare neighbors or not. Thus for any particle P, a neighbor-hood can be assigned as N(P), containing all the neighborsof that particle. Different neighborhood topologies andtheir effect on the swarm performance have been discussedin Kennedy (1999). The PSO used in this work, implicitlyuses a so-called fully connected neighborhood topology (orgbest). Every particle is neighbor of every other particle.

Each particle P has two state variables:

(1)

Its current position ~xðtÞ (2) Its current velocity ~vðtÞ.

And also a small memory comprising,

(1)

Its previous best position ~pðtÞ i.e. personal bestexperience in terms of the objective function valuef ð~pðtÞÞ.

(2)

The best ~pðtÞ of all P 2 NðPÞ: ~gðtÞ i.e. the best positionfound so far in the neighborhood of the particle.

The PSO scheme has the following algorithmic para-meters:

(1)

Vmax or maximum velocity which restricts ~V iðtÞ withinthe interval ½�Vmax;Vmax�.

(2)

An inertial weight factor o. (3) Two uniformly distributed random numbers j1 and j2,

which respectively determine the influence of ~pðtÞ and~gðtÞ on the velocity update formula.

(4)

Two constant multiplier terms C1 and C2 known as ‘‘selfconfidence’’ and ‘‘swarm confidence’’, respectively.

Initially the settings for ~pðtÞ and ~gðtÞ are ~pð0Þ ¼ ~gð0Þ ¼~xð0Þ for all particles. Once the particles are initialized, theiterative optimization process begins where the positionsand velocities of all the particles are altered by the followingrecursive equations. The equations are presented for the dthdimension of the position and velocity of the ith particle:

Vidðtþ 1Þ ¼ oVidðtÞ þ C1j1 PdðtÞ � X idðtÞð Þ

þ C2j2 gdðtÞ � X idðtÞ

,

X idðtþ 1Þ ¼ X idðtÞ þ V idðtþ 1Þ. ð8Þ

The first term in the velocity updating formula representsthe inertial velocity of the particle. The second terminvolving ~pðtÞ represents the personal experience of eachparticle and is referred to as ‘‘cognitive part’’. The last termof the same relation is interpreted as the ‘‘social term’’which represents how an individual particle is influenced bythe other members of its society.In many occasions, the convergence of PSO is premature,

especially if the swarm uses a small inertia weight o orconstriction coefficient (Clerc and Kennedy, 2002). As theglobal best found early in the searching process may be apoor local minima, we propose a multi-elitist strategy forsearching the global best of the PSO. We call the new variantof PSO the MEPSO. The idea draws inspiration from theworks reported in Deb et al. (2002b). We define a growthrate b for each particle. When the fitness value of a particle oftth iteration is higher than that of a particle of (t�1)thiteration, the b will be increased. After the local best of allparticles are decided in each generation, we move the localbest, which has higher fitness value than the global best intothe candidate area. Then the global best will be replaced bythe local best with the highest growth rate b. Therefore, thefitness value of the new global best is always higher than theold global best. The pseudocode about MEPSO is as follows:

For t ¼ 1–tmax

For j ¼ 1 to N//swarm size is N

If (the fitness value of particlej in tth time-step4thatof particlej in (t�1)th time-step)

bj ¼ bj+1;End

Update local bestj .If (the fitness of local bestj4that of global best now)

Choose local bestj put into candidate area.End

EndCalculate b of every candidate, and record the candidate ofbmax .Update the global best to become the candidate ofbmax.Else

Update the global best to become the particle ofhighest fitness value.

End

End

4. Application of the PSO and other evolutionary algorithms

to the design problem

4.1. Further simplification of the problem

Without loss of generality let us assume N ¼ 2. Then H

(Z1, Z2) in Eq. (1) can be restated as

Hðz1; z2Þ

¼ H0p00 þ p01z2 þ p02z22 þ p10z1 þ p20z21 þ p12z1z

22 þ p21z21z2 þ p22z

21z22

ð1þ q1z1 þ r1z2 þ s1z1z2Þð1þ q2z1 þ r2z2 þ s2z1z2Þ.

ð9Þ

ARTICLE IN PRESS

Fig. 2. Progress towards the optima: Performance comparison of different

optimization algorithms on J2 for 50,000 FE.

S. Das, A. Konar / Engineering Applications of Artificial Intelligence 20 (2007) 1086–10961090

Now if we substitute Z1 and Z2 as in Eq. (2), then M (o1,o2) can be expressed as

Mðo1;o2Þ ¼ H0ðp00 þ p01f 01 þ p02f 02 þ p10f 10 þ p20f 20 þ p11f 11 þ p12f 12 þ p21f 21 þ p22f 22Þ

V

�

�jðp00 þ p01g01 þ p02g02 þ p10g10 þ p20g20 þ p11g11 þ p12g12 þ p21g21 þ p22g22Þ

V

�, ð10aÞ

with

f xy ¼ cos ðxo1 þ yo2Þ gxy ¼ sinðxo1 þ yo2Þ (10b)

and x, y ¼ 0,1,2

and

V ¼ ð1þ q1f 10 þ r1f 01 þ s1f 11Þ � jðq1g10 þ r1g01 þ s1g11Þ� �� ð1þ q2f 10 þ r2f 01 þ r2f 11Þ � jðq2g10 þ r2g01 þ s2g11Þ� �

.

ð10cÞ

From (10a) we may put M(o1, o2) in a compact form as

Mðo1;o2Þ ¼ H0NR � jNI

ðD1R � jD1I ÞðD2R � jD2I Þ, (11)

where

NR ¼ p00 þ p01f 01 þ p02f 02 þ p10f 10 þ p20f 20 þ p11f 11

þ p12f 12 þ p21f 21 þ p22f 22,

NI ¼ p00 þ p01g01 þ p02g02 þ p10g10 þ p20g20 þ p11g11

þ p12g12 þ p21g21 þ p22g22,

D1R ¼ 1þ q1f 10 þ r1f 01 þ s1f 11,

D1I ¼ q1g10 þ r1g01 þ s1g11,

D2R ¼ 1þ q2f 10 þ r2f 01 þ s2f 11,

D2I ¼ q2g10 þ r2g01 þ s2g11. ð12Þ

Hence, the actual magnitude may be written as

jMðo1;o2Þj ¼ H0

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiðN2

R þN21Þ

ðD21R þD2

1I ÞðD22R þD2

2I Þ

s. (13)

Now let us consider a specific example of the designproblem, where the user-specification for the desiredcircular symmetric low-pass filter response is given as

Mdðo1;o2Þ ¼ 1 if

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffio2

1 þ o22

qp0:04p

¼ 0:5 if 0:04ppffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffio2

1 þ o22

qp0:08p

¼ 0 otherwise: ð14Þ

Also from (5) the constraints may be put in acontinuously differentiable form as

� ð1þ skÞoðqk þ rkÞoð1þ skÞ

� ð1� skÞoðqk þ rkÞoð1� skÞ

ð1þ skÞ40

ð1� skÞ40. ð15Þ

Now in this problem we experiment with b ¼ 1, 2, 4 and8, N1 ¼ 100 and N2 ¼ 100. Finally, the constrained

minimization task becomesMinimize J

¼X100n1¼0

X100n2¼0

Mpn1

100;pn2

100

� ��� ����Mdpn1

100;pn2

100

� h i2(16a)

subjected to the constraints imposed by (15) with k ¼ 1, 2.We define our fitness function as

f ¼1

J þ eps, (16b)

so that maximization of f leads to minimization of J. eps isa small bias term having value 0.001.

4.2. Particle representation

In order to apply the PSO algorithm to the problemformulated in Eq. (16) we need to represent each trialsolution as a particle in a multi-dimensional search space.Since p00 is always set to 1 in Eq. (1), the dimensionality ofthe present problem is 14 and each particle has 14positional coordinates represented by the vector

X ¼ ðp01; p02; p10; p11; p12; p20; p21; p22; q1; q2; r1; r2; s1; s2;H0ÞT.

(17)

ARTICLE IN PRESSS. Das, A. Konar / Engineering Applications of Artificial Intelligence 20 (2007) 1086–1096 1091

All these 14-dimensional (14D) particles have 14components in their velocity vector.

4.3. Other competitive algorithms

In 2002, Deb et al. (2002a). proposed a generic parentcentric recombination scheme (PCX) and integrated it with asteady state, elite preserving, scalable and computationallyfast population alteration model of the GA, which theynamed the G3 (generalized generation gap) model. Theirresults indicate that the G3 model with PCX can outperformmany other existing GA models and the classical DE whentested on the standard benchmark functions. Below we brieflyoutline their algorithm. We use the G3 with PCX as our firstcompetitive algorithm. Classical DE (Storn and Price, 1997)has recently gained wide popularity as a fast and efficientoptimization algorithm over continuous search spaces. Wealso compare our proposed algorithm with DE over the zero-phase IIR filter design problem. The particular version of DEthat we choose is also known as the ‘DE/rand/1/bin’ since ituses a binary crossover. There are also other frequently usedDE operators, such as ‘DE/rand/1/bin’ and’DE/best/2/exp’,etc., which we did not investigate in this study.

4.4. Parameter setup for the algorithms

As suggested by Mladenov and Mastorakis (2001) andMastorakis et al. (2003), we select the initial value of theparameters of the vector in Eq. (17) randomly from theinterval (�3, 3). In case of PSO algorithm we vary the inertiaweight part linearly from 0.4 to 0.9. We find that this speedsup the convergence of the algorithm to some extent. Otherparameter settings and population sizes, used in all experi-ments reported in this paper, has been shown in Table 1.

4.5. Population initialization and population size

Each space-coordinate of a particle was initialized with arandom floating-point number whose absolute value waskept below 3.00. Following Eberhart and Shi (2000), themaximum allowable velocity Vmax for each particle waslimited to the upper value of the dynamic range of search,i.e., |Vmax| ¼ |Ximax| ¼ 3.00. Eberhart and Shi also showedthat the population size has hardly any effect on theperformance of the PSO method. It is quite common in

Table 1

Parameter settings for the competitor algorithms

G3 with PCX MEPSO DE

Parameter Value Parameter Value Parameter Value

Pop_size 100 Popsize 40 Popsize 10*dim

sZ 0.1 Inertia weight 0.4-0.9 CR 0.9

sz 0.1 C1, C2 1.494 R 0.8

m 3 Vmax 3.00

l 2

PSO research to limit the number of particles in the range20–60. van den Bergh and Engelbrecht (2001) have shownthat though there is a slight improvement of the optimalvalue with increasing swarm sizes, a larger swarm increasesthe number of function evaluations necessary to convergeto an error limit. We maintain a constant population of 40particles throughout the runs of the PSO algorithm. ForDE we have followed the usual 10 times rule i.e. thenumber of chromosomes in each generation is kept 10times the number of dimensions of the chromosomes.Hence, population size for DE is 14� 10 ¼ 140.

4.6. Handling the problem constraints

To handle the constraints we followed the method by Deb(2000) as follows: (a) any feasible solution is preferred to anyinfeasible solution; (b) between two feasible solutions, the onewith a better objective function value is preferred; (c) betweentwo infeasible solutions, the one having a smaller constraintviolation is preferred. To tackle the constraints presented inEq. (15) we start with a population of around 200 particleswith randomly initialized positional coordinates. Out ofthese, 40 particles were selected, space-coordinates of whichobey the constraints imposed by (15). If more than 40particles are initially found to obey the constraints, obviouslythe selection takes into account the initial fitness value ofthese particles. During the run of the program, the globallybest particle was sorted not only on the basis of its fitnessvalue in the swarm but also depending on whether or not itobeyed the constraints. That is, if a particle in course of itsmovement through the search space yields the lowest fitnessvalue found so far, its position will be memorized as theglobally best position by all other members in the swarm onlyif it satisfies the constraints. For evolutionary algorithms likeDE and GA, only those offsprings were promoted forreproduction or survival to the next generation which satisfiesthe constraints besides having greater fitness values.

4.7. Filter figures of merit used

The performances of the IIR filters designed here, areevaluated in terms of the following figures of merit:

(1)

Error of frequency response in pass band:

efpðoÞ ¼ HðejoÞ �HdðejoÞ

�� �� for o 2 ½0;op�.

GA used in Mastorakis et al. (2003)

Parameter Value

Maximum number of chromosomes per generation 250

No. of bits per gene 32

Mutation probability 0.05

Part of genetic materials interchanged during cross-over 12

Maximum number of children from each pair of parents 10

ARTICLE IN PRESS

Tab

Mea

the c

Filte

coef

p01p02p10p11p12p20p21p22q1q2r1r2s1s2H0

Tab

Mea

com

Valu

diffe

J1J2J4J8

Tab

Resu

Cost

J1J2J4J8

S. Das, A. Konar / Engineering Applications of Artificial Intelligence 20 (2007) 1086–10961092

Passband magnitude ripple:

(2)

epmrðoÞ ¼ HðejoÞ�� ��� 1 for o 2 ½0;op�.

(3)

Stopband attenuation:

AðoÞ ¼ 20 log10 HðejoÞ�� �� for o 2 ½os;p�.

(4)

Maximum magnitude of the poles,

where op and os are the passband and stopbandfrequencies, respectively. The transfer function of the filtersynthesized by a search algorithm will considerably deviatefrom the desired transfer function if the error of frequency

le 2

n value and standard deviations of the final results (filter coefficients) with

ompetitor algorithms)

r

ficient

MEPSO DE G3 with P

0.306170.0010 �0.242670.002 �0.301670.994970.0022 2.482770.0063 2.902370.393570.0068 �0.348470.0074 �0.34357�0.033870.003 �2.089870.001 �2.049070.648170.0067 0.032370.0054 0.038771.234570.0021 2.491570.0063 2.493270.503070.0037 0.161370.0005 0.197570.448170.0046 0.756370.0078 0.74937�1.023970.081 �0.911370.0009 �0.473870.034270.0001 ��0.025570.0366 �0.08437�0.960570.035 2.961370.00054 2.94937�0.037170.002 �0.034470.006 �0.037670.952370.0078 0.867470.0072 0.88747�0.905670.005 �0.807570.0026 �0.847670.0003470.001 0.001270.0073 0.07847

le 3

n value and standard deviations of the final results (J values) with exponent

petitor algorithms)

e of J for

rent exponents

MEPSO DE G3 wit

60.392370.0054 98.551370.0327 95.7113

9.000570.0323 11.907870.583 10.4252

0.503970.00054 2.961370.0344 0.5732

0.005870.0001 0.290370.0755 0.0178

le 4

lts of unpaired T-tests on the data of Table 3

function Std. error t 95% co

0.195 179.313 �35.44

0.221 6.3440 �1.85

0.030 3.1041 �0.15

0.0001 8.0000 �0.00

response in passband is considerably large. Hence, for agood design, the maximum passband error should be as low

as possible (Oppenheim et al., 1999; Proakis and Manola-kis, 1996). Similarly, the maximum passband ripple shouldbe low enough to ensure a distortionless production of theoutput signal. On the contrary, the minimum value ofpassband attenuation should be large in order to ensure theelimination of the undesired frequency components to ahigh degree.

5. Experiments and results

We run four population-based optimization algorithmsnamely MEPSO, DE, the G3 with PCX model and the

exponent p ¼ 2 after 50,000 FEs (mean of 20 independent runs of each of

CX GA in Mastorakis et al.

(2003)

NN-based method Mladenov

and Mastorakis (2001)

0.002 1.819370.0024 1.8922

0.0063 �1.103970.006 �1.2154

0.0074 0.072570.0032 0.0387

0.001 �2.520870.008 �2.5298

0.0054 0.427970.0022 0.3879

0.0063 0.600370.0024 0.6115

0.0005 �1.369370.005 �1.4619

0.0078 2.4502770.0082 2.5206

0.0009 �0.860370.007 �0.8707

0.0366 �0.890470.006 �0.8729

0.00054 �0.856670.003 �0.8705

0.006 �0.838970.006 �0.8732

0.0072 0.735170.0052 0.7756

0.0026 0.803470.0068 0.7799

0.0073 0.0009370.0049 0.0010

p ¼ 1, 2, 4, 8 after 50,000 FEs (mean of 20 independent runs of each of the

h PCX GA in Mastorakis et al.

(2003)

NN-based method

Mladenov and Mastorakis

(2001)

70.382 96.763570.8742 140.6201

70.989 10.034270.0663 14.4182

70.1324 0.634670.0154 1.2163

70.0396 0.009170.0014 0.0698

nf. intvl. Two-tailed P Significance

7 to �34.656 o0.0001 Extremely significant

16 to �0.9557 o0.0001 Extremely significant

18 to �0.0319 0.0036 Very significant

35 to �0.0021 o0.0001 Extremely significant

ARTICLE IN PRESS

rsin

term

softhefiguresofmeritmentioned

inSection5.2

(meanandstandard

deviations(inbracket)over

20independentrunswith

J2cost

function)

MEPSO

DE

G3withPCX

GAin

Mastorakiset

al.

(2003)

NN-basedmethod

Mladenovand

Mastorakis(2001)

ssband

0.005123(0.0000021)

0.029105(0.000982)

0.009462(0.0000028)

0.072612(0.00032)

0.062718

0.027643(0.0000282)

0.102827(0.0006298)

0.092812(0.0000462)

0.139281(0.00046)

0.198187

42.28172(0.02910)

35.20912(1.29182)

34.19829(2.01419)

29.19828(2.46292)

26.39782

0.846325(0.0002918)

0.891718(0.0006752)

0.918182(0.003262)

0.901819(0.00001)

0.918192

S. Das, A. Konar / Engineering Applications of Artificial Intelligence 20 (2007) 1086–1096 1093

binary encoded GA suggested in Mastorakis et al. (2003)on the design of a circular symmetric zero-phase low passfilter according to the user specification summarized inEq. (14). All the algorithms discussed here have beendeveloped from scratch in Visual C++ platform on aPentium IV, 2.2GHz PC, with 512 KB cache and 2 GBof main memory in Windows Server 2003 environment.The graphs and figures have been obtained usingMATLAB 6.5.

Two aspects of the algorithms have been investigated inthis work, firstly, their accuracy and then their speed ofconvergence. For comparing the speed of different algo-rithms the first thing we require is a fair time measurement.The number of iterations or generations cannot beaccepted as a time measure as the algorithms performdifferent amount of works in their inner loops and alsothey have different population sizes. We have used thenumber of fitness function evaluations as a measure oftime. The advantage of measuring complexity by countingthe function evaluations is that there is a strong relation-ship between this measure and the processor time as thefunction complexity increases.

To judge the accuracy of the algorithms, we firstly run allof them (except the neural network based one reported inMladenov and Mastorakis, 2001) for 50,000 FEs (FunctionEvaluations). Each algorithm is run independently (with adifferent seed for the random number generator in everyrun) for 20 times and the mean best J value as well as meanvalues of the parameters obtained along with the standarddeviations have been repored in Tables 2 and 3. Thenotation Jb has been used to denote two sets of experimentsperformed with the value of J obtained using exponentb ¼ 2 and 4. Table 4 summarizes the results of the unpairedt-test on the J value (standard error of difference of the twomeans, 95% confidence interval of this difference, the t

value, and the two-tailed P value) between the best andnext-to-best results in Table 2. For all cases in Table 3,sample size ¼ 20 and degrees of freedom ¼ 38. Next weselect four threshhold values of J for b ¼ 1, 2, 4 and 8(based on the data of Table 3 and report the averagenumber of FEs required by the optimization algorithms(over 20 independent runs) in order to reach the threshholdin Tables 5 and 6. This gives a measure of the relative speedof the algorithms over this problem.

Table

6

Perform

ance

comparisonofthederived

filte

Valueofthefigure

ofmeritsused

Maxim

um

erroroffrequency

response

inpa

Maxim

um

passbandmagnituderipple

Minim

um

stopbandattenuation(dB)

Maxim

um

magnitudeofthepoles

Table 5

Mean No. of FE required for reaching the cut off cost function value by

different algorithms (over 20 independent runs)

Jb cut-off

value

Mean no. of FEs required

MEPSO DE G3 with

PCX

GA in Mastorakis

et al. (2003)

J1 ¼ 70.00 26372.50 27736.25 33627.80 34637.05

J2 ¼ 11.00 32023.60 35237.95 39283.95 38291.45

J4 ¼ 0.80 24903.75 30483.80 40394.65 34243.85

J8 ¼ 0.10 30028.15 32948.55 38749.70 40056.48

ARTICLE IN PRESS

Fig. 3. Desired amplitude response |Md(o1, o2)| of the 2D filter.

Fig. 4. Amplitude response |M(o1, o2)| of the 2D filter using MEPSO

with b ¼ 2.

Fig. 5. Amplitude response |M(o1, o2)| of the 2D filter using DE with

b ¼ 2.

Fig. 6. Amplitude response |M(o1, o2)| of the 2D filter using G3 with

PCX with b ¼ 2.

Fig. 7. Amplitude response |M(o1, o2)| of the 2D filter using in the

method of Mastorakis et al. (2003) with b ¼ 2.

Fig. 8. Amplitude response |M(o1, o2)| of the 2D filter using in the

method of Mladenov and Mastorakis (2001) with b ¼ 2.

S. Das, A. Konar / Engineering Applications of Artificial Intelligence 20 (2007) 1086–10961094

ARTICLE IN PRESSS. Das, A. Konar / Engineering Applications of Artificial Intelligence 20 (2007) 1086–1096 1095

Figs. 3–8 show the ideal low-pass filter frequencyresponse and the frequency response of the filter designedusing the competitive algorithms used here. All thesefigures are drawn from the results obtained using miniiza-tion of J2 for 50,000 FE. Fig. 2 shows the progress of theoptimization algorithms towards the optima of the searchspace with no. of function evaluations.

Finally in Fig. 9, we try to catch a glimpse of theperformance of the designed low pass filters when appliedto remove Gaussian noise from the renowned 256� 256gray scale image of Lenna. We here study one genericapplication of the low pass 2D filters in the de-noising ofdigital images (Gonzales and Woods, 1992). Fast Fouriertransform (FFT) is performed on the original image usingthe MATLAB image processing toolbox. The filter transfer

Fig. 9. Result of filtering with the designed filter: (a) Original image ‘‘lenna’’. (b

Filtering with the MEPSO-based method. (d) Filtering with DE based method

as in Mastorakis et al. (2003).

function is multiplied with the image in frequency domainand then the product is subjected to inverse FFT (IFFT)for obtaining the filtered image in a presentable form.

6. Discussion

A closer look at Figs. 3–8 reveals that the MEPSOalgorithm used by us yields a better approximation of thedesired response as compared to works presented inMladenov and Mastorakis (2001) or Mastorakis et al.(2003) and yields a considerably good filter response with asmaller number of FEs as evident from Fig. 1. The ripplein the stop-band of Fig. 4 is much lesser as compared toFigs. 5–8. This leads to the filtering of a 2D filter withminimum distortion. The fact can be felt by taking a look

) Image corrupted with Gaussian noise of mean ¼ 0, variance ¼ 0.005. (c)

(e) filtering with G3 with PCX-based method (f) filtering with GA method

ARTICLE IN PRESSS. Das, A. Konar / Engineering Applications of Artificial Intelligence 20 (2007) 1086–10961096

at the de-noised image of Fig. 9(c) obtained with theMEPSO-based low-pass filter. This image preserves mostof the details of the original one although considerablysuppresses the Gaussian noise. As evident from the t-testresults on the data of Table 3, the PSO-based method beatsits nearest competitor in a statistically significant manner.The comparison also reflects the fact that the G3 with PCXmodel in Deb et al. (2002a), used by us, performs muchbetter than the GA used in Mastorakis et al. (2003) interms of accuracy. The possible reason may be that Binarycoded GA has problem due to discretization of the searchspace. The speed of reaching the optimum is comparablefor PSO and DE but poorer than the GAs as both PSO andDE uses very primitive mathematical operations. Espe-cially, the main body of PSO algorithm is surprisinglysimple (takes 4–5 lines to code in a standard languagelike C). The performance of DE is to some extent bleak interms of accuracy. We believe DE can be improved fordifficult optimization problems, like the present one, bysuitably tuning its scale factor and the crossover rate.

7. Conclusion

In this paper, a recently developed optimization algo-rithm, borrowed from the realm of swarm intelligence, hasbeen applied to the real-world problem of designing 2Dzero-phase recursive filters. The filter thus obtained has areasonably good stability margin (we have incorporatedthe stability criteria as constraints to the minimizationtask). Our method leads to a simpler filter since in practicewe have realized a factorable denominator. Compared tothe methods described in Mladenov and Mastorakis (2001)and Mastorakis et al. (2003), which, to our knowledge, arethe most recent and the best-known methods to date, thealgorithm used here yields a better design in considerablyless time. The MEPSO algorithm has also shown to meet orbeat two other very well-known evolutionary computingtechniques (DE and a real-coded GA) in a statisticallysignificant manner.

References

Chellapilla, K., Fogel, D.B., Rao, S.S., 1996. Optimizing IIR filters using

evolutionary programming. In: Proceedings of the Adaptive Distrib-

uted Parallel Computing Symposium, Dayton, OH, pp. 252–258.

Clerc, M., Kennedy, J., 2002. The particle swarm—explosion, stability,

and convergence in a multidimensional complex space. IEEE

Transactions on Evolutionary Computation 6 (1), 58–73.

Daniel, M., Willsky, A., 1997. Efficient implementations of 2-D non-causal

IIR filters. IEEE Transactions on Circuits and Systems II 44 (7), 549–563.

Das, S., Abraham, A., Konar, A., 2006. Spatial information based image

segmentation using a modified particle swarm optimization algorithm.

In: Sixth International Conference on Intelligent Systems Design and

Applications, ISDA’06. IEEE CS Press.

Deb, K., 2000. An efficient constraint handling method for genetic

algorithms. Computer Methods in Applied Mechanics and Engineer-

ing 186 (2–4), 311–338.

Deb, K., Anand, A., Joshi, D., 2002a. A computationally efficient

evolutionary algorithm for real-parameter optimization. Evolutionary

Computation 10 (4), 371–395.

Deb, K., Pratap, A., Agarwal, S., Meyarivan, T., 2002b. A fast and elitist

multiobjective genetic algorithm: NSGA-II. IEEE Transactions on

Evolutionary Computation 6 (2).

Eberhart, R.C., Shi, Y., 2000. Comparing inertia weights and con-

striction factors in particle swarm optimization. Proceedings of

IEEE International Congress on Evolutionary Computation 1,

84–88.

Goldberg, D.E., 1989. Genetic Algorithms in Search, Optimization, and

Machine Learning. Addison-Wesley, Reading, MA.

Gonzales, R.C., Woods, R.E., 1992. Digital Image Processing, third ed.

Addison-Wesley, Reading, MA.

Gorne, T., Schneider, M., 1993. Design of digital filters with evolutionary

algorithms. International Conference on Artificial Neural Networks

and Genetic Algorithms Ref. No. 155-1, pp. 1–7.

Hsieh, C.-H., Kuo, C.-M., Jou, Y.-D., Han, Y.-L., 1997. Design of two

dimensional FIR digital filters by a two-dimensional WLS technique.

IEEE Transactions on Circuits and Systems II 44 (5), 348–412.

Kaczorek, T., 1985. Two-Dimensional Linear Systems. Springer, Berlin,

Germany.

Kalinli, N., Karaboga, N., 2005. Artificial immune algorithm for IIR

filter design. Engineering Applications of Artificial Intelligence 18 (8),

919–929.

Kennedy, J., 1999. Small worlds and mega-minds: effects of neighborhood

topology on particle swarm performance. In: Proceedings of the

1999 Congress of Evolutionary Computation, vol. 3. IEEE Press,

New York, pp. 1931–1938.

Kennedy, J., Eberhart, R., 1995. Particle swarm optimization. In:

Proceedings of IEEE International Conference on Neural Networks,

pp. 1942–1948.

Kennedy, J., Eberhart, R.C., 2001. Swarm Intelligence. Academic Press,

New York, ISBN 1-55860-595-9.

Laasko, T., Ovaska, S., 1994. Design and implementation of efficient IIR

notch filters with quantization error feedback. IEEE Transactions on

Instrumentation and Measurement 43 (3), 449–456.

Lu, W.S., Antoniou, A., 1992. Two-Dimensional Digital Filters. Marcel

Dekker, New York.

Mastorakis, N., Gonos, F.I., Swamy, M.N.S., 2003. Design of two-

dimensional recursive filters using genetic algorithms. IEEE Transac-

tions on Circuits and Systems 50, 634–639.

Mladenov, V., Mastorakis, N., 2001. Design of two-dimensional recursive

filters by using neural networks. IEEE Transactions on Neural

Networks 12, 585–590.

Oppenheim, A.V., Schafer, R.W., Buck, J.R., 1999. Discrete-Time Signal

Processing. Prentice-Hall, Englewood Cliffs, NJ.

Proakis, J.G., Manolakis, D.G., 1996. Digital Signal Processing. Prentice-

Hall, Englewood Cliffs, NJ.

Schnier, T., Yao, X., Liu, P., 2004. Digital filter design using multiple

Pareto fonts. Soft Computing 8 (5), 332–343.

Seppanen, J., 1999. Audio Signal Processing Basics, Introductory

Lectures, Audio Research Group, Tampere University of Technology,

Web Resource /http://www.cs.tut.fi/sgn/arg/intro/basics.htmlS.

Shi, Y., Eberhart, R.C., 1998. A modified Particle Swarm Optimiser. IEEE

International Conference on Evolutionary Computation, Anchorage,

Alaska, May 4–9.

Storn, R., Price, K., 1997. Differential evolution—a simple and efficient

heuristic for global continuous spaces. Journal of Global Optimization

11 (4), 341–359.

Tzafestas, S.G. (Ed.), 1986. Multidimensional Systems, Techniques and

Applications. Marcel Dekker, New York.

van den Bergh, F., Engelbrecht, P.A., 2001. Effects of swarm size on

cooperative particle swarm optimizers. In: Proceedings of GECCO-

2001. San Francisco, CA, pp. 892–899.