Optimal Design of IIR Filters Using Linear Semi-Indefinite Programming Method

TAKAYUKI YAMAZAKI and KENJI SUYAMATokyo Denki University, Japan

SUMMARY

An optimal design method for stable IIR (InfiniteImpulse Response) filters under the min-max criterion isproposed. The design problem considered is the complexChebyshev approximation of a rational function includingthe stability constraint. We formulate this problem as aproblem of real linear semi-infinite programming problemusing the real rotation theorem. The problem is solved bythe three-phase method; one of the methods is used forsolving semi-infinite programming problems. The three-phase method includes three operations. In the first opera-tion, some active constraint candidates are selected by theiterative simplex method. Next, the second operation inte-grates some degenerate constraints. In the third operation,the approximate solution obtained up to the second opera-tion is adjusted so as to satisfy the optimality condition. Thefilters designed by the method are found to be more precisethan those designed by the conventional method. Severaldesign examples are presented to demonstrate the effective-ness of the proposed method. © 2011 Wiley Periodicals,Inc. Electron Comm Jpn, 94(6): 17–23, 2011; Publishedonline in Wiley Online Library (wileyonlinelibrary.com).DOI 10.1002/ecj.10330

Key words: digital filter; linear semi-indefiniteprogramming method; nonlinear programming problem;simplex method.

1. Introduction

A digital filter [1] is a basic discrete-time circuit usedin digital signal processing. Digital filters are widely usedin communications, measurement, control, and other fields.Digital filters are divided into those with an impulse re-sponse of finite length (FIR) and of infinite length (IIR).FIR filters guarantee stability, and perfect linear phasecharacteristics can be implemented. On the other hand, IIR

filters offer the same magnitude characteristics as FIRfilters but at a much lower order, offering a considerablecost reduction. However, stability is not guaranteed, andperfect linear phase characteristics cannot be implemented.Therefore, stability must be taken into account when de-signing IIR filters; furthermore, approximately linear phasecharacteristics must be assured for applications that requirewaveform preservation. This study deals with the fre-quency-domain design of IIR filters under such conditions.

Since an ideal low-pass filter is not causal, the prob-lem of digital filter design is generally an approximationproblem. Maximum error minimization (Chebyshev ap-proximation) is used as the approximation criterion in manycases. The frequency characteristic of an IIR filter is ex-pressed by a rational function, and its Chebyshev approxi-mation is generally a nonlinear problem. In addition,complex Chebyshev approximation is required when phasecharacteristics are considered.

For complex Chebyshev approximation, a methodthat combines the Remez algorithm with the eigenvalueproblem [2] and a method based on nonlinear programming[3] allow successful design, but both methods ignore stabil-ity. On the other hand, there are design methods with regardfor stability, such as a method using successive projectionbased on Rouché’s theorem [4], a method based on semi-definite programming [5], and a quadratic programmingmethod using real positiveness as a constraint to assurestability [6].

Real positiveness has also been used as a constraintfor linear programming problems, but every approach hassome advantages and disadvantages. For example, the de-nominator amplitude is not taken into account in an optimi-zation method that divides the complex Chebyshevapproximation problem into a real part and an imaginarypart [7]. In a method [8] based on the real rotation theorem[10], the complex approximation problem is formulated asan equivalent linear programming problem in the real do-main so as to apply linear programming techniques; how-ever, this method is limited to the design of low-pass filtersand three-band bandpass filters. Another design method [9]

© 2011 Wiley Periodicals, Inc.

Electronics and Communications in Japan, Vol. 94, No. 6, 2011Translated from Denki Gakkai Ronbunshi, Vol. 129-C, No. 1, January 2009, pp. 53–58

17

uses the simplex method to iteratively solve a linear pro-gramming problem formulated by the real rotation theo-rem; this method applies to filters of any type but highcomputational cost may be a problem.

The constraints of a linear programming problemformulated by the real rotation theorem involve continuousvariables, namely frequency and the rotation parameter, andhence lead to a linear semi-infinite programming problem(LSIP). In the LSIP, large-scale problems must be solved inorder to obtain accurate solutions by linear programming,resulting in a high computational cost of design based oniterative simplex methods such as that of Ref. [9]. Thus,solution optimality and computational cost are in a trade-offrelationship.

In this study, we propose a new method for designproblems formulated by means of the real rotation theorem;in particular, we combine the three-phase method [11] (analgorithm for LSIP solving) with the iterative simplexmethod. The simplex method is employed for the derivationof approximate solutions that are required in order to findoptimal solutions to the LSIP, and as a result, the solvedproblem can be reduced to a relatively small scale. Thisallows faster solving, and the solutions obtained have guar-anteed stability and optimality. Using numerical examples,we shall show that the proposed method can produce opti-mal solutions faster than conventional methods.

2. Complex Approximation Design of IIR Filter

In this section, we consider the problem of complexapproximation design problem of an IIR filter using themaximum error minimization criterion.

The input–output relation of an IIR filter can bedescribed as follows:

Here x(k) and y(k) denote, respectively, the input and outputsignals and an and bm are filter coefficients. Applying thez-transform to both sides of Eq. (1), the transfer functionH(z) = Y(z)/X(z) of the IIR filter is derived as follows:

Let H(ω) denote the frequency characteristic of H(z) withz = ejω, and let D(ω) denote the desired characteristic. Thenthe problem of IIR filter design by the maximum errorminimization criterion becomes the problem of complexChebyshev approximation of D(ω) by means of H(ω). With

the approximation band denoted by Ω, the design problemcan be expressed as follows:

where W(ω) is a weighting function.

3. Formulation of Semi-infinite ProgrammingProblem

In this section, we use the real rotation theorem toformulate design problem (3) as a semi-infinite program-ming problem.

H(ω) in Eq. (3) is a rational function; therefore, it isdifficult to handle in that form. Let us consider approximat-ing the desired characteristic D(ω)B(ω) by A(ω), that is,

This problem is equivalent to Eq. (3).Using the real rotation theorem, minimization prob-

lem (4) is transformed from the complex domain into thereal domain. The real rotation theorem states that the mag-nitude |w| of a complex number w = x + jy can be calculatedby the following equation:

Here Φ = r: 0 ≤ r < 2π. As shown in Fig. 1, Eq. (5) meansthat |w| coincides with the real part when w is rotated to thereal axis.

The following is obtained when Eq. (5) is applied toproblem (4):

(1)

(2)

(3)

(4)

Fig. 1. Real rotation theorem.

(5)

18

Denoting the maximum error by δ, Eq. (6) can be reformu-lated as the following constrained minimization problem:

Real positiveness [7] (a sufficient condition for stability) isimposed as constraint, that is,

Here Ω′ = ω′: 0 ≤ ω′ ≤ π, and ε is a small positive value.Since stability must be satisfied beyond the approximationband as well, Ω′ is different from Ω.

Arranging Eq. (7), Eq. (8) and writingD(ω) = d(ω)ejφ(ω), the filter design problem becomes thefollowing principal problem of semi-infinite programming:

The problem can be described in matrix representation asfollows:

Here T denotes the transpose, and

The dual problem of Eq. (10) is as follows:

Here y1(ω, r) and y2(ω′) are dual variables. The optimalsolution includes a total of N + M + 2 positive values, andthe rest of the elements are all zeros. Σ(ω,r)∈(Ω, Φ) andΣω′∈Ω′ mean that addition is performed only for positivey1(ω, r) and y2(ω′).

4. Proposed Method

In this section, we explain the proposed method, inwhich the iterative simplex method is applied to problem(11) after linearization, and then the three-phase method isemployed to find the best complex Chebyshev approximatesolution. The three-phase method consists of selection ofactive constraint candidates, integration of degenerate con-straints, and adjustment of the constraints.

4.1 Selection procedure

In this phase, active constraint candidates are selectedfor problem (11). For this purpose, ω is discretized into Selements on Ω, r is discretized into T elements on Φ, andω′ is discretized into U elements on Ω′; thus, problem (11)is reduced to the following problem with a finite number ofconstraints:

As regards problem (12), a1(ωi, rj) includes the nonlinear|B(ωi)| in the denominator, which makes calculation diffi-cult. Therefore, |B(ωi)| is first fixed at a constant value, Eq.

(6)

(8)

(7)

(9)

(11)

(10)

(12)

19

(12) is solved by the simplex method, and |B(ωi)| is updatedusing the resulting bm. Optimization of Eq. (12) by thesimplex method and updating of |B(ωi)| are repeated inalternation until bm converges, that is, until the following issatisfied:

Here bmk denotes the bm obtained at the k-th iteration, and e

is a small positive value.Problem (12) converges within 20 iterations in most

cases. The accuracy improves with larger S and T but thenumber of constraint increases, which requires larger mem-ory and increases the computational cost. However, theobjective of the selection procedure is to obtain approxi-mate solutions, and hence extreme accuracy is not needed.Therefore, the memory requirements and computationalcost can be reduced compared to mere iterations of thesimplex method [9].

4.2 Integration procedure

In this phase, the degenerate constraints among thosefound in the previous phase are integrated. If problem (12)is solved properly, N + M + 2 pairs of (ωi, rj) and ωk

g areobtained from the active constraints.

However, the selection procedure gives only approxi-mate solutions to Eq. (11), and therefore two constraintsclose to the optimal ω, r, ω′ may prove to be active at thesame time. Such simultaneously active constraints are inte-grated into one constraint. In this study, integration isperformed by taking the arithmetic mean when (ωi′, rj′) and(ωi′ + 1, rj′′) are active for adjacent constraints, as follows:

When (ωi′, rj′) and (ωi′, rj′′) of the same frequency are activesimultaneously, the same operation is performed by assum-ing ωi′ = ωi′+1. For ω′, integration is performed as follows:

4.3 Adjustment procedure

In this phase, the constraints are adjusted in terms ofsolution optimality so as to find the optimal solution. Letus assume that L pairs of ωl, rl and P values of ωm

g were

obtained in the integration phase. These are approximatesolutions to problem (11), and are assumed to be close tothe true active constraint ω, r, ω’. In the adjustment phase,ωl, rl, y1(ωl, rl), ωm

g , y2(ωmg ), and x are treated as initial

values, and the active constraints are adjusted so as to meetthe Karush–Kuhn–Tucker conditions [12], which are opti-mality conditions for solution to problem (11). Here x is thesolution to Eq. (10) found from the solutionsωi, rj, y1(ωi, rj), ωk, y2(ωk

g) to Eq. (11) using the comple-mentary slackness theorem [13].

The Karush–Kuhn–Tucker conditions are expressedby the following simultaneous nonlinear equations:

Newton’s method is used to solve these simultaneous non-linear equations, and convergence is achieved in severaliterations.

4.4 Design algorithm

The design algorithm of the proposed method isdescribed below.

(I) The denominator order M, the numerator order N,the number of frequency divisions S, the number of rotationparameter divisions T, the number of frequency divisionsU for stability condition, the lower stability limit ε, and theadmissible error e are specified. In addition, |B(ωi)| = 1 isset.

(II) Problem (12) is solved by the simplex method.(III) If Eq. (13) is not satisfied, then |B(ωi)| is updated

using bm found in step (II), and the algorithm returns to step(II).

(IV) The adjacency of ωi and ωk is checked using thesolutions obtained in step (II), and integration is performedusing Eqs. (14) to (18).

(V) Equations (19) to (24) are solved by Newton’smethod.

(13)

(14)

(15)

(16)

(19)

(20)

(21)

(22)

(23)

(24)

(17)

(18)

20

Here steps (I) to (III), (IV), and (V) respectively representselection, integration, and adjustment.

5. Numerical Examples

In order to verify the effectiveness of the proposedmethod, we present numerical examples. The settings are S= 200, T = 30, U = 200, ε = 10–4, e = 10–4. The desiredcharacteristic D(ω) is

Here τd is the passband group delay, ωp is the passbandcutoff frequency, and ωs is the stopband cutoff frequency.

5.1 Design example 1

In this example, we designed a filter with the samespecifications as the low-pass filter obtained by successiveprojection based on Rouché’s theorem [4]. The designconditions were M = 6, N = 12, τd = 9, ωp = 0.5π, ωs = 0.6π;the low-pass filter had a weight of 1 across the entire band.

The first phase was completed in 9 iterations of thesimplex method, and the third phase was completed in 7iterations of Newton’s method. The magnitude charac-teristic, the passband magnitude characteristic, and thepassband group delay characteristic are presented in Figs.2, 3, and 4, respectively. In the diagrams, the horizontal axisrepresents the frequency normalized by 2π; here 0.5 corre-sponds to half the sampling frequency. The design proce-dure took 3 s on a 3.20 GHz Pentium-4 PC with 1 GB ofmemory. The result was δ = 1.42 × 10–2 (stopband: –36.95dB, passband: 0.122 dB), which is better than the 1.71 ×10–2 (stopband: –35.34 dB, passband: 0.147 dB) of thecompared method. On the other hand, the maximum errorof the group delay was 0.8701,compared with 0.8619 of theexisting method.

Although the existing method also takes optimalityinto account, the results are somewhat different. This isbecause positive realness is adopted as the constraint toassure stability in the proposed method, while maximumpole radius is used for this purpose in the existing method.We may assume a trade-off between the maximum error δ(which defines the complex error) and the maximum errorof the group delay; thus, the proposed method may be moresuitable for minimization of δ, and the existing method ismore suitable for minimization of the group delay error,although this difference is only slight.

5.2 Design example 2

In this example, we designed a filter with the samespecifications as the low-pass filter obtained by the methodbased on nonlinear programming [3]. The design condi-tions were M = N = 4, τd = 5, ωp = 0.2π, ωs = 0.4π; thelow-pass filter had a weight of 1 across the entire band.

The first phase was completed in 11 iterations of thesimplex method, and the third phase was completed in 2iterations of Newton’s method. The magnitude charac-

(25)

Fig. 2. Magnitude response (design example 1).

Fig. 3. Passband magnitude response (design example 1).

Fig. 4. Passband group delay response (design example 1).

21

teristic, the passband magnitude characteristic, and groupdelay characteristic are presented in Figs. 5, 6, and 7,respectively. In the diagrams, the horizontal axis representsthe frequency normalized by 2π; here 0.5 corresponds tohalf the sampling frequency. The design procedure took 2s on a 3.20 GHz Pentium-4 PC with 1 GB of memory. Theresult was δ = 2.129 × 10–2 (stopband: –33.43 dB, passband:0.183 dB), which is better than the 4.198 × 10–2 (stopband:–27.54 dB, passband: 0.36 dB) of the existing method. Themaximum error of the group delay was 0.8940 comparedwith 1.34 of the existing method. In addition, the maximumpole radius was 0.8940, which confirmed stability.

Since stability is not considered in the comparedmethod [3], we ignored the stability constraint in this designexample; however, the same results were obtained when theconstraint was imposed. While the compared method isbasically approximate, one might expect better results, interms of both δ and group delay error, from the proposedmethod, which aims at optimality. In addition, the proposedmethod has the advantage that stability is guaranteed.

5.3 Discussion

In order to demonstrate that the proposed methodallows accurate design with smaller computational cost andmemory requirements than a simple simplex method, wedesigned a low-pass filter with a weight of 1 across theentire band, and with M = N = 8, τd = 5, ωp = 0.2π, ωs = 0.4π and ε = 10–4, e = 10–4. Figure 8 shows how the maximumerror varies with the number of constraints in the case ofsimple simplex method, and in the case of the proposedmethod. Here S = 10T, and the stability constraint is notapplied. Therefore, the total number of constraints is 10T2.

As can be seen from Fig. 8, when the simplex methodis applied in unaltered form, the solution does not convergeto the optimum even though the number of constraints isincreased; but convergence to the optimal solution isachieved at 4000 constraints in the proposed method. De-sign by the proposed method took 1.1 s on the same PC asin design examples 1 and 2; with simple simplex method,design with 105 constraints took 23.4 s. An optimal solutionwas not obtained, although 21.3 as much time was required.

Fig. 5. Magnitude response (design example 2).

Fig. 6. Passband magnitude response (design example 2).

Fig. 7. Passband group delay response (design example 2).

Fig. 8. Relationship between number of constraints andmaximum error.

22

Thus, the proposed method produced an optimal solutionmore than 20 times faster. Furthermore, more memory isneeded for design with more constraints, and therefore theproposed method is advantageous in terms of both compu-tational complexity and memory requirements.

6. Conclusions

We have proposed a method of complex Chebyshevapproximate design of IIR filters using the iterative simplexmethod combined with the three-phase method. The non-linear semi-infinite programming problem of complex ap-proximation design is formulated as a linear semi-infiniteprogramming problem using the real rotation theorem.Approximate solutions are found by the iterative simplexmethod, and then used as initial values for Newton’smethod in order to obtain the optimal solution. Comparedto mere iteration of the simplex method, the proposedmethod offers lower computational cost and memory re-quirements. In the future, we plan to examine the applica-tion of the proposed method to IIR filters with complexcoefficients.

REFERENCES

1. Takebe T. Design of digital filters. Tokai UniversityPress; 1992. (in Japanese)

2. Okuda M, Ikehara M, Takahashi S. Complex Cheby-shev approximation for IIR digital filters. IEICETrans Japan 1995;J78-A:651–658. (in Japanese)

3. Chen X, Parks TW. Design of IIR filters in the com-plex domain. IEEE Trans ASSP 1990;38:910–920.

4. Sugita T, Aikawa N. Complex Chebyshev approxi-mation to stable IIR filters design. Proc PACRIM,Vol. 2, p 563–566, 2001.

5. Lu WS. Design of stable minimax IIR digital filtersusing semidefinite programming. Proc ISCAS2000,Vol. 1, p 355–358.

6. Tseng CC. Design of stable IIR digital filter based onleast P-power error criterion. IEEE Trans CAS-I2004;51:1879–1888.

7. Chottera A, Jullien G. A linear programming ap-proach to recursive digital filter design with linearphase. IEEE Trans CAS 1982;29:139–149.

8. Yoshida T, Nishihara A, Fujii N. Magnitude andphase approximation of IIR digital filters using linearprogramming. IEICE Trans 1989;E72-E:1097–1103.

9. Tseng CC, Lee SL. Minimax design of stable IIRdigital filter with prescribed magnitude and phaseresponses. IEEE Trans CAS-I 2002;49:547–551.

10. Chen X, Parks TW. Design of FIR filters in thecomplex domain. IEEE Trans ASSP 1987;35:144–153.

11. Ito R, Suyama K, Hirabayashi R. Optimal design ofFIR filter with discrete coefficients based on integersemi-infinite linear programs. Proc ISCAS2001, pII-629–632.

12. Sakawa M. Mathematical programming. MorikitaShuppan; 2004.

13. Konno H. Linear programming. JUSE; 1987.

AUTHORS (from left to right)

Takayuki Yamazaki (nonmember) received a bachelor’s degree from Tokyo Denki University (electrical engineering) in2006, completed the M.E. program in 2008, and joined Hitachi Advanced Systems Corp. His student research interests weremathematical programming and digital filter design.

Kenji Suyama (member) completed the doctoral program at the University of Electro-Communications in 1998 and joinedthe faculty as a research associate. He became a research associate at Tokyo University of Science in 1999 and lecturer at TokyoDenki University in 2002, where he is now an associate professor. His research interests are microphone arrays, sound sourcelocalization, and digital filter design. He holds a D.Eng. degree, and is a member of IEICE, RISP, and SSPATJ.

23