Image restoration by an iterative regularizedpseudoinverse method

Junji Maeda and Kazumi Murata

In this paper we describe a digital method for restoring linearly degraded images in the presence of noise.The restoration procedure is an iterative regularized pseudoinverse (RPI) algorithm that is based on theprinciple of least squares. This method acquires the advantage of tolerance to noise by incorporating addi-tional constraints of non-negativity of the object and adaptive regularization. We also use a median filterin the proposed algorithm to suppress spiky noise. We present some results of computer simulations thatinclude both space-invariant and space-variant degradations. The effectiveness of the iterative RPI meth-od is demonstrated through computer simulations.

1. Introduction

Singular-value decomposition (SVD) and pseu-doinverse techniques have recently been used in digitalimage restoration. Hanson1 and Varah2 investigated1-D problems using a truncated SVD expansion. Thesame method was applied to 2-D noisy degraded imagesby Huang and Narendra.3 Andrews and Patterson4 5

demonstrated the restoration of space-variant degra-dations. Rushforth et al.6 and Severcan7 have usedthese techniques in the restoration of images withmissing high-frequency components. Although thesetechniques have provided good restoration in thenoise-free case, their performance in the presence ofnoise is highly unstable and inadequate. We haveproposed the iterative version of the above techniqueswhich we call the iterative regularized pseudoinverse(RPI) algorithm.8 We have demonstrated the useful-ness of the proposed method for restoring noisy ban-dlimited images (superresolution).

In this paper we investigate the use of the iterativeRPI algorithm in the restoration of linearly degradedimages in the presence of noise. This algorithm effec-tively uses additional constraints of non-negativity ofthe object and adaptive regularization to reduce theeffect of noise. In the present work we also propose toincorporate a median filter into the iterative RPI algo-rithm to suppress spiky noise. The combination of theabove additional constraints with median filtering

The authors are with Hokkaido University, Faculty of Engineering,Department of Applied Physics, Sapporo 060, Japan.

Received 23 November 1983.0003-6935/84/060857-05$02.00/0.(© 1984 Optical Society of America.

makes the proposed algorithm significantly noise tol-erant in comparison with other methods. After dis-cussing the restoration procedure, we present some re-sults of computer simulations.

II. Restoration Procedure

We consider the problem that the object has beenlinearly degraded by an imaging system and measuredwith some attendant noise. The imaging system has aspace-invariant point-spread function (SIPSF) or aspace-variant point-spread function (SVPSF). Weassume that the object of interest is known to be non-negative. At first we discuss the iterative RPI methodfor 1-D images. The sampled value g of the linearlydegraded image can be expressed by a matrix equa-tion:

g = Hf + n, (1)

where f and n are the sampled object and noise vectors,respectively, and H is the point-spread function matrix.We assume for convenience that the number of elementsin g, f, and n are N, and H has the dimension N X N.

Our aim is to estimate the unknown object or to solveEq. (1) for f. We seek the estimate f based on theprinciple of least squares, i.e., the objective function orsquared error:

W(t) = jig- ll (2)

is to be minimized. Since the matrix H is singular ornearly singular for the restoration problem, the fol-lowing pseudoinverse solution is realistic:

t = H+g, (3)

where H+ is the Moore-Penrose pseudoinverse of thematrix H.9 One definition of H+ is through the singu-lar-value decomposition of the matrix H:

15 March 1984 / Vol. 23, No. 6 / APPLIED OPTICS 857

3. 0

2 0

1 0

0 I 16 a-l

1 1 6 32 48 64i

Fig. 1. Logarithmic plots of the weighting factors Xi/(X? + a/X?) fora SIPSF (negative values are omitted for intuitive understanding).

H = UAVT, (4)

where U and V are N X N eigenvector matrices of HH Tand HTH, respectively, and A is an N X N diagonalmatrix whose elements A1, . . , AN are the singularvalues of H. Then the matrix H+ is represented as

H+ = VA+UT = -Vi UT (5)i=l 1

where J _ N is the rank of H, and ui and vi are thesingular vectors.

In general, Eq. (3) yields a poor restoration since theproblem becomes ill-conditioned. Rushforth et al. 6

presented the regularizing procedure that uses the fol-lowing RPI matrix in place of H+ in Eq. (5):

N iTf~+vA~uT=i X~ a/X~ ' (6)

where a is called the regularizing parameter. Thus therestored estimate is given by

= a/A2/X2 (uTg)vi. (7)

We call Eq. (7) the noniterative RPI estimate. Figure1 shows one example of the plots of logarithms of theweighting factors Xi/(XV + a/V) for various values of(e in the case of a SIPSF and N = 64. As we increase theparameter a, the high-order singular modes are grad-ually attenuated, and serious noise amplification canbe avoided.

The iterative RPI method has been established basedon the Gauss algorithm, which is the nonlinear least-squares technique.10 The restored estimate at the (k+ 1)st iteration by the iterative RPI algorithm is givenby

to, +I -1,ft H+ (g - H )

=h 2 + ['(g - Ht/)]vi, (8)

with the initial estimate

to = . (9)

The detailed derivation of the iterative RPI algorithm

is described in Ref. 8, which also includes the discussionabout the convergence of the proposed algorithm. Itis noted that the estimate at the first iteration in theiterative RPI algorithm, tf = ftg, is identical with thesolution of the noniterative RPI method in Eq. (7) fora = ao, where ao is the regularizing parameter in Ho.

In the implementation of the iterative RPI algorithm,the regularizing parameter a is increased from iterationto iteration, which we call the adaptive regularization,according to the following manner:

ah = c X ak-1 = C k X o, I(10)

where c is the constant that controls the degree of reg-ularization and we take c > 1. This adaptive techniquegradually attenuates the high-order singular modes andeffectively reduces the noise amplification. As to thenon-negativity constraint, we simply set the negativevalues of the restored estimate to zero at each cycle ofthe iteration. We also employ a median filter in theiterative RPI algorithm, which is discussed in Sec.III.

Next we consider the iterative RPI method for 2-Dimages. We restrict the problem to the case in whichthe point-spread function matrix is separable. In thiscase, the sampled value G of the measured degradedimage can be written in the matrix formll

G = AFBT + E, (11)

where F and E are the sampled object and noise ma-trices, respectively, and A and B are the degradationmatrices. If we treat the N X M matrices G and F,matrices A, B, and E have the dimensions N X M, M XM, and N X M, respectively. In the present work wetreat only the case A = B = H, and then Eq. (11) be-comes

G = HFHT+ E, (12)

where G, F, and E are all N X N matrices.Equation (12) is solved based on the principle of least

squares, i.e., the squared error IG- HFH T 2 is to beminimized. In this case, we can use the same RPI ma-trix Hl as the 1-D problem. Thus the restored estimatefor 2-D degraded images by the iterative RPI algorithmis given by

Fk+l = fF + Hl(G - HFkHT)(H/1)T,

with the initial estimate

Po = O.

(13)

(14)

Equations (13) and (14) represent the 2-D versions ofEqs. (8) and (9).

111. Incorporation of Median Filter

It is most important for the iterative RPI algorithmto set an appropriate f, by choosing the suitable valueof a 0. It is reasonable for the same signal-to-noise ratio(SNR) to choose a0 as small as possible, because a smallvalue of a0 corresponds to more information about thehigh-order singular modes. 8 However, the small valueof ao also leads to instability in the first estimate t1, i.e.,the smaller the parameter a0, the closer the weighting

858 APPLIED OPTICS / Vol. 23, No. 6 / 15 March 1984

(6)

0.5 -

: 10 20 30 40 50 60

Fig. 2. 1-D original object.

1 .0

0.

- . . .

0 20 30 4 5 6 0 -4 0 50 60

estimate of the iterative RPI algorithm. We have in-corporated median filtering into the iterative RPImethod by operating a median filter on a restored es-timate, after imposing the non-negativity constraint,at each cycle of the iteration. In computer simulationexamples in Sec. IV we used a median filter with awindow of length three for 1-D images. For 2-D images,a 3 X 3 plus-sign-shaped median filter was employed toavoid the deletion of the corners in the rectangularimages.

IV. Results of Computer Simulations and Discussion

We have applied the iterative RPI method to therestoration of 1-D and 2-D images. The restored esti-mate after the (k + 1)st iteration cycle is evaluated bythe restoring coefficient R, which is defined as'4

Fig. 3. Noisy degraded image by the SIPSF system at 30-dB SNR.

1 .0

0.5

0 A J,I 10 \_0 30

t.-

40 50 6d

Fig. 4. Restored image for Fig. 3 by usingalgorithm.

1.0.

0 .5.

0

the noniterative RPI

1I0 20 30 40 50 60

Fig. 5. Restored image for Fig. 3 by using the iterative RPI algorithmwithout median filtering.

R = 1- [11f +1hll X 100(%).t I f - g F

(15)

This R indicates how the restored image approaches theoriginal object in comparison with the degraded image.In our simulation, we terminate the iteration using ei-ther the restoring coefficient or the predeterminedmaximum number of iterations. Basically our iterativealgorithm is terminated when the restoring coefficientbecomes the largest. (This coefficient begins to de-crease thereafter owing to the ill-conditioned nature ofthe problem.) If the above condition is not fulfilleduntil the predetermined maximum iteration number (inour case 30 cycles), we terminate the iteration at thiscycle for computational convenience.

In the following simulations, we use N = 64 samplingpoints. This means that 64 X 64 sampling points areused in the 2-D simulation. Figure 2 shows the original1-D object. This object has been degraded by theSIPSF system of the uniform motion (9 pixels) andzero-mean white Gaussian noise has been added to thisblurred image. The resultant degraded image at a SNRof 30 dB is shown in Fig. 3. The SNR is defined by

variance of Hf

variance of n1.0

0

Fig. 6. Restored image for Fig. 3 by using the iterative RPI algorithmwith median filtering.

factors Xi/(X? + a/X) approach the values for inversefilter (see Fig. 1). As a result, the preferable value of aoinevitably accompanies a certain degree of noise am-plification, and thus spiky noise becomes noticeable inFl.

It is well known that a median filter is efficient inreducing the effect of spiky or impulse noise withoutblurring sharp edges.12"13 Consequently, we adopt amedian filter to suppress spiky noise in the restored

(16)

The restored image obtained by the noniterative RPImethod is shown in Fig. 4, which has R = 86.1%. Wecarried out the iterative RPI method at first withoutusing the median filter. The restored result obtainedwith a 0 = 2 X 10-5 and c = 1.3 is shown in Fig. 5, whichhas R = 95.7%. Although this figure illustrates betterrestoration than the result in Fig. 4, some spiky noisestill remains especially in the plateau region. We thenperformed the iterative RPI method with the medianfilter and obtained the result in Fig. 6. This figuredemonstrates more-significant improvement (R =98.4%) than the restored image in Fig. 5 and clearly in-dicates the effectiveness of incorporating the medianfilter into the iterative RPI algorithm.

We next treated the image in Fig. 7, which has beendegraded by the SVPSF system and has 30-dB SNR.In this figure the extent of the blur increases linearly aswe move from left to right. The restored images ob-tained by the noniterative RPI algorithm and the iter-

15 March 1984 / Vol. 23, No. 6 / APPLIED OPTICS 859

I a/ . , \1, , , , \1, , ,H,\

I .0-

0.5-

0 10: 20 30 40 50 6 0 -0 20 30 40 50 60

Fig. 7. Noisy degraded image by the SVPSF system at 30-dBSNR.

1.0

0 .5

with ao = 5 X 10-5 and c = 1.1. Although Fig. 11(a)demonstrates sufficient restoration in resolution (R =73.7%), the dappled patterns occur in the character re-gions mainly because of spiky noise. Figure 11(b) (R= 80.3%) obviously represents, in comparison with Fig.11(a), that the median filter successfully suppresses theeffect of spiky noise and provides significant improve-ment in appearance.

To study the performance of the proposed algorithmin restoring noisier images, the SNR has been changedto 15 dB, preserving the same SIPSF degradation.Figures 12(a) and (b) show the degraded image and therestored estimate obtained by the iterative RPI algo-rithm with the median filter (R = 61.6%), respectively.

0

Fi.g 8. Restored

I .0

0 .5

n

1 20 30 40 50 60

image for Fig. 7 by using the noniterative RPIalgorithm.

10 20 30 40 50 60

vow

Jo ILsMUFW _-

z @ _J

(a) (b)

Fig. 10. (a) 2-D original object; (b) noisy degraded image by theSIPSF system (K = 3, L = 13) at 30-dB SNR.

Fig. 9. Restored image for Fig. 7 by using the iterative RPI algorithmwith median filtering.

ative one with the median filter are shown in Fig. 8 (R= 89.4%) and Fig. 9 (R = 96.9%), respectively. Thesefigures indicate that the iterative RPI algorithm isuseful in restoring SVPSF degradations.

The next example is the 2-D restoration in the caseof the separable point-spread function. We used theGaussian point-spread function in this simulation. Themnth component of the matrix H in Eq. (12) is givenby 15 16

hmn = exp[-(m - n)2/2( 1

where o-m = IKI/V/ for the SIPSF, and Cm = Im -K 1 /'\- for the SVPSF. C is a normalization constantand K is a blurring factor which determines the extentof blur. For the SIPSF cases, the matrix H is bandedwith bandwidth L, and each column (or row) with fullbandwidth is summable to unity. For the SVPSF cases,each column in the matrix H is summable to unity.

Figure 10(a) shows a model object consisting of fourChinese characters which mean Applied Optics in En-glish. The degraded image obtained through theSIPSF system is shown in Fig. 10(b). We took K = 3and L = 13 in Eq. (17) and 30-dB SNR. The restoredresults obtained using the iterative RPI method withoutand with the median filter are shown in Figs. 11 (a) and(b), respectively. These restorations were performed

f~~~~~~~i - 0

(a)

: I ,: 000000 000000 tal000

He l k

0 _

(b)

Fig. 11. Restored images obtained by using the iterative RPI algo-rithm (a) without median filtering and (b) with median filtering.

(a) (b)

Fig. 12. (a) Noisy degraded image by the SIPSF system (K= 3, L=13) at 15-dB SNR; (b) restored image by the iterative RPI algorithm

with median filtering.

860 APPLIED OPTICS / Vol. 23, No. 6 / 15 March 1984

-

(17)

(a) (b)

Fig. 13. (a) Noisy degraded image by the SVPSF system (K = 0.2)at 30-dB SNR; (b) restored image by the iterative RPI algorithm with

median filtering.

(a) (b)

Fig. 14. (a) Noisy degraded image by the SVPSF system (K = 0.2)at 15-dB SNR; (b) restored image by the iterative RPI algorithm with

median filtering.

This result suggests the power of the proposed algo-rithm to restore degraded images in the presence of aconsiderable amount of noise.

We then restored the images degraded by the SVPSFsystem. We took K = 0.2 in Eq. (17). Figures 13(a)and 14(a) show the degraded images at SNRs of 30 and15 dB, respectively. It is noted that the SVPSF systemin this simulation has better focus at the center andincreasing blur toward the edges. The restored imagesby the iterative RPI algorithm with the median filter inthe cases of 30- and 15-dB SNRs are shown in Figs.13(b) (R = 86.1%) and 14(b) (R = 74.2%), respectively.These figures indicate the availability of the iterativeRPI algorithm in the restoration of noisy blurred imagesdegraded by the SVPSF systems as well as the SIPSFones.

As discussed in Sec. III, the value of a 0 plays an im-portant role in the implementation of the iterative RPImethod. Once the suitable value of ao is selected, wecan expect a good restoration by our constrained iter-ative algorithm. However, the optimum value of a0,among the several proper values, is determined empir-ically in the present work. Additional study is requiredabout the efficient determination of the optimum valueof ao so that the proposed method can be applied to thepractical problems. As to the values of c, our simulation

revealed that a suitable range of the value of c is-1.05-1.3. The optimum choice of c within this rangedepends on the value of a0, i.e., the larger value of c issuitable for the smaller value of a0, and the smaller c issuitable for the larger ao0.

The computing time for one iteration cycle in the it-erative RPI algorithm for our 2-D example is -0.6 secon the Hitac M-280H computer.

V. Conclusion

We have described a digital restoration procedure,the iterative RPI algorithm, to restore noisy degradedimages. This method is based on the principle of leastsquares and incorporates additional constraints ofnon-negativity of the object and adaptive regularizationtechnique to reduce the effect of noise. We also employa median filter to suppress spiky noise. As a result, theproposed method becomes significantly noise tol-erant.

Computer simulated examples using both SIPSF andSVPSF degradations in 1-D and 2-D images have beenpresented. The usefulness of the iterative RPI algo-rithm in restoring both types of degradation in thepresence of a considerable amount of noise has beenconfirmed through computer simulations. We havealso clearly demonstrated the effectiveness of incor-porating the median filter into the iterative RPI algo-rithm. The practical application of the proposedmethod has the remaining problem of efficiently de-termining the optimum value of a0 and is currentlyunder investigation.

References1. R. J. Hanson, SIAM (Soc. Ind. Appl. Math.) J. Numer. Anal. 8,

616 (1971).2. J. M. Varah, SIAM (Soc. Ind. Appl. Math.) J. Numer. Anal. 10,

257 (1973).3. T. S. Huang and P. M. Narendra, Appl. Opt. 14, 2213 (1975).4. H. C. Andrews and C. L. Patterson, Am. Math. Mon. 82, 1

(1975).5. H. C. Andrews and C. L. Patterson, IEEE Trans. Acoust. Speech

Signal Process. ASSP-24, 26 (1976).6. C. K. Rushforth, A. E. Crawford, and Y. Zhou, J. Opt. Soc. Am.

72, 204 (1982).7. M. Severcan, Appl. Opt. 21, 1073 (1982).8. J. Maeda and K. Murata, J. Opt. Soc. Am. Al, 28 (1984).9. A. Albert, Regression and Moore-Penrose Pseudoinverse (Aca-

demic, New York, 1972).10. J. Kowalik and M. R. Osborne, Method for Unconstrained Op-

timization Problems (American Elsevier, New York, 1968).11. H. C. Andrews and B. R. Hunt, Digital Image Restoration

(Prentice-Hall, Englewood Cliffs, N.J., 1977).12. J. W. Tukey, Exploratory Data Analysis (Addison-Wesley,

Reading, Mass., 1971).13. W. K. Pratt, Digital Image Processing (Wiley, New York,

1978).14. T. Honda, K. Kumagaya, and J. Tsujiuchi, Opt. Acta 24, 23

(1977).15. H. S. Hou and H. C. Andrews, IEEE Trans. Comput. C-26, 856

(1977).16. N. N. Abdelmalek, T. Kasvand, J. Olmstead, and M. M. Trem-

blay, Appl. Opt. 20, 4227 (1981).

15 March 1984 / Vol. 23, No. 6 / APPLIED OPTICS 861