26
Iterative Restoration Deblurring of SPOT Panchromatic Images George Lemeshewsky U. S. Geological Survey National Center, MS 521 Reston,VA 22092 Open-File Report 93-353 ABSTRACT This report describes the results of two preliminary tests. One was a test with a previously reported constrained iterative restoration technique (Schaefer and others, 1981, Kawata and Ichioka, 1980) to deblur 10-m-resolution panchromatic spectral band images from the SPOT-1, HRV 2 sensor system. 1 The other was an experiment to obtain a spatially compact estimate of the sensor's point spread function, or PSF. Without constraints the iterative process, in the limit, converges to inverse filter restoration. The problem of noise amplification characteristic of inverse filtering can be lessened by limiting the number of iterations while still achieving partial deblurring. Both fixed and variable constraints incorporating a prior knowledge about the undegraded image were used to reduce ringing artifacts. There are computational advantages if the PSF that models the blur degradation has small spatial extent. This was achieved experimentally, starting with a previously reported frequency domain model (Schowengerdt, R and Li, W. M., 1987), by a combination of frequency domain scaling, spatial domain filtering, and down-sampling. Initial results show that phase response of the desired small-size PSF only partially agrees with the original model. However, the derived modulation transfer function more closely matches the model than data from a conventional window filter-design technique with nearly twice the spatial extent Simulated image data with and without white Gaussian noise was used to illustrate the degree of convergence of inverse filter restoration. Tests of inverse filter restoration with SPOT imagery show that reasonable deblurring is possible with only minor noise amplification by limiting the number of iterations and using only simple space invariant constraints. 1 Any use of trade, product, or firm names is for descriptive purposes only and does not imply endorsement by the U.S. Government.

Iterative Restoration Deblurring of SPOT Panchromatic Images · Iterative Restoration Deblurring of SPOT Panchromatic Images ... Image restoration has been defined by Andrews

Embed Size (px)

Citation preview

Page 1: Iterative Restoration Deblurring of SPOT Panchromatic Images · Iterative Restoration Deblurring of SPOT Panchromatic Images ... Image restoration has been defined by Andrews

Iterative Restoration Deblurring of SPOT Panchromatic Images

George Lemeshewsky U. S. Geological Survey National Center, MS 521 Reston,VA 22092 Open-File Report 93-353

ABSTRACT

This report describes the results of two preliminary tests. One was a test with a previously reported constrained iterative restoration technique (Schaefer and others, 1981, Kawata and Ichioka, 1980) todeblur 10-m-resolution panchromatic spectral band images from the SPOT-1, HRV 2 sensor system. 1 The other was an experiment to obtain a spatially compact estimate of the sensor's point spread function, or PSF. Without constraints the iterative process, in the limit, converges to inverse filter restoration. The problem of noise amplification characteristic of inverse filtering can be lessened by limiting the number of iterations while still achieving partial deblurring. Both fixed and variable constraints incorporating a prior knowledge about the undegraded image were used to reduce ringing artifacts. There are computational advantages if the PSF that models the blur degradation has small spatial extent. This was achieved experimentally, starting with a previously reported frequency domain model (Schowengerdt, R and Li, W. M., 1987), by a combination of frequency domain scaling, spatial domain filtering, and down-sampling. Initial results show that phase response of the desired small-size PSF only partially agrees with the original model. However, the derived modulation transfer function more closely matches the model than data from a conventional window filter-design technique with nearly twice the spatial extent Simulated image data with and without white Gaussian noise was used to illustrate the degree of convergence of inverse filter restoration. Tests of inverse filter restoration with SPOT imagery show that reasonable deblurring is possible with only minor noise amplification by limiting the number of iterations and using only simple space invariant constraints.

1 Any use of trade, product, or firm names is for descriptive purposes only and does not imply endorsement by the U.S. Government.

Page 2: Iterative Restoration Deblurring of SPOT Panchromatic Images · Iterative Restoration Deblurring of SPOT Panchromatic Images ... Image restoration has been defined by Andrews

INTRODUCTION

Image restoration has been defined by Andrews (1974) as "... the reconstruction of an image toward an object (original) by inversion of some degradation phenomena," and by Tekalp and Trussell (1991) as "...the process of undoing imaging degradations based on a mathematical model of these degradations." Image degradations can be categorized as due to blur or to noise. Blur, for example, may be due to lens misfocus, relative motion between the object and the image sensor, or from atmospheric turbulence effects (Andrews, 1974). Noise degradations may be due to electronic circuit noise and/or signal quantization (Lim, 1990). Restoration is concerned with finding the best reconstruction or estimation of an undegraded image from a degraded, noisy image (Biemond and others, 1990). For example, Wiener filter image restoration is "best" in the sense of minimum mean square error (Andrews, 1974).

In contrast to restoration, Lim (1990) describes image enhancement as "... the processing of images to improve their appearance to human observers or to enhance other image processing systems' performance." An important distinction is that the objective of image enhancement is to make the processed image better in some sense than the unprocessed image. With restoration, the objective is to make the processed image resemble the original image as much as possible. For example an original undegraded image cannot be restored but it can be enhanced by increasing the sharpness by highpass filtering. Simple edge sharpening by high spatial frequency emphasis is commonly used for visual enhancement. Completely different objectives exist between enhancement and restoration.

This report describes preliminary experiments with constrained iterative restoration, as an inverse filter for blur removal in SPOT high-resolution visible (HRV) 10-m resolution panchromatic spectral wave band images (SPOT Image Corporation, Reston, Virginia © 1988). In the limit, as the number of iterations approaches infinity the process approaches inverse filter restoration. When limited to a finite number of iterations, the restoration is not the same as inverse filter restoration (Lim, 1990); however, the term restoration will be used in this report.

This report is organized into several sections. Section I describes the discrete space model for both degradation and restoration. A simple discrete system model, an approximation to the continuous model (Park, 1992), was used wherein the undegraded input image, degraded output image, and degradation are modeled as discrete. Herein the objective of restoration is a discrete estimate of the original continuous image. Restored images are then observed as continuous on an image display CRT. Display distortion was not included in the degradation model.

Section II describes the basic concept of inverse filter restoration and its inherent problem with noise amplification.

The iterative restoration technique for image deblurring is described in Section III for the special case of linear, shift-invariant blur degradation. In the limit, this process converges to inverse filter restoration. However, when limited to a finite number of iterations, noise amplification will be reduced. Spatially adaptive constraints, used to reduce ringing artifacts common to iterative restoration deblurring are also described.

Section IV summarizes the SPOT sensor system optical transfer function model (OTF) as given by Schowengerdt and Li (1987) in order to clarify subsequent discussion of the methods to calculate a small spatial extent PSF.

Page 3: Iterative Restoration Deblurring of SPOT Panchromatic Images · Iterative Restoration Deblurring of SPOT Panchromatic Images ... Image restoration has been defined by Andrews

Section V describes two distinct methods used to calculate a small spatial extent PSF, (i.e. the optical system impulse response) from the OTF: the common window method (Lim, 1990) and an experimental method that uses frequency domain scaling, spatial domain filtering, and subsampling. For the repeated convolutions required of iterative restoration, there are computational benefits when the modeled PSF is small.

Section VI contains experimental results of the PSF characterization, restoration examples for an elementary simulated image and examples of constrained iterative restoration applied to SPOT HRV, 10-m pixel, panchromatic sensor imagery. Specifically, satellite 1, HRV instrument 2, level 1A data were used. It is raw data, processed for detector normalization only. Observations on this preliminary work are in Section VII, Conclusions.

The motivation for experimenting with iterative restoration deblurring techniques is severalfold. No inverse degradation model or noise model is needed; only a model of the sensor PSF is required. Artifacts from excessive noise amplification are either constrained or else controlled by terminating the restoration prior to convergence and before noise becomes objectionable (Biemond and others, 1990). Prior to convergence, the partially restored image may be reasonably deblurred. Of course the latter is a subjective assessment, highly dependent upon image detail and noise levels.

In contrast to the repeated calculations for an iterative process, restoration could be performed by a single convolution of the degraded image with a restoration filter. An algorithm for the design of mean- square optimal values for small kernel restoration filters was recently described by Reichenbach and Park (1991).

I. DISCRETE SPACE MODEL

In the following description, g(i, j), f(i, j), and h(i, j) are two dimensional (2-D) discrete-space sequences, defined for all integer sample locations i, j, and g, f, h are continuous functions. When the degradation can be modeled or approximated as linear, shift-invariant, then

oo oo

g(U) = f(i,J)**h(i,j) = £ £ f(l,m)h(i-l,j-m) (1) 1 = oo m = oo

where g(i, j) is the degraded output image; f(i, j) is the undistorted, unknown input image; h(i, j) is the impulse response or point spread function, PSF; and ** denotes linear 2-D convolution. Since h(i, j) is typically of low pass character and blurs the image it is sometimes called the blurring function (Lim, 1990).

Typically, discrete image samples g(i, j) and f(i, j) at i, j are quantized values or pixels. However the simple system model of equation 1 does not include quantization effect. A more realistic imaging system model would include noise degradation to account for noise such as that from image sensor electronic components or from signal quantization. Including additive noise, n(i, j) with the degraded image g(i, j) of equation 1 gives

a,) = f(i,)**ha,) + nG,). (2)

Page 4: Iterative Restoration Deblurring of SPOT Panchromatic Images · Iterative Restoration Deblurring of SPOT Panchromatic Images ... Image restoration has been defined by Andrews

H. INVERSE FILTER RESTORATION

Inverse filter restoration to remove blur degradations is outlined next for the two conditions: with and without noise. Expressing equation 1, the ideal, no-noise case, in the frequency domain gives

G(o)!, o)2) = F(o)l7 002) H (o)^) (3)

where G, F, H denotes the Fourier transform (FT) of g, f, and h, respectively, and col, 0)2 are frequency variables. Inverse filter restoration deblurring of equation 3 is

F'(o)l7 o)2) =

={ F(o)1) 0)2) H(o)1) 02) } / H(o)1) 0)2) = F(o1 , 02) (4)

where F'(OI, 002) is the estimate of the original undegraded image F(o>1> 002). Ideally the restored, deblurred image F'(®i, 0)2) equals F(o)1 , 0*2).

With linear additive noise, inverse filter restoration for equation 2 is

, o>2) = { 0(0)!, G>2) + N(o>1 , 0)2) } / H(o)1 , 0)2)

=F(o)1} 0)2) + { N(o)!, 02) / H(o)!, o>2) } (5)

where N(o)j , o^) is the FT of noise n(i, j). Here the estimate for F(o)j , 002) includes the original noise N(o>1 , 0)2) term multiplied, or amplified, by 1 / H(o)l5 o^). Inverse filtering can be very sensitive to noise and give unacceptable restoration due to excessive noise amplification. Typically the OTF, H(O)I} G^) is small at higher frequencies, and hence I/ H(o)j , 0*2) will be large. Relatively low noise levels, when present at higher frequencies, will be excessively amplified. In fact, for severe motion- induced blur, H(<DI , 0)2) can equal zero. One elementary method to lessen the noise sensitivity of inverse filter restoration is to limit 1 / H(o)!, 0)2) to some threshold, T, when I/ 1 H(<D!, o^) I > T (Lim, 1990).

III. Constrained Iterative Restoration

The basic iterative restoration equation (Lim, 1990) for the linear, shift invariant blur model of equation 1 is

f'o(i,J) = Xg(i,j) (6a)

f WO. J) = f 'k(i, J) + X [ (g(i, j) - (h(i, j) ** f 'k(i, j)) (fib)

where X, positive, is a parameter that influences the rate of convergence; f 'k(i, j) is the estimate or approximation of f(i, j) at iteration k; f '0(i, j) is the initial estimate of the undegraded image; g(i, j) is the given degraded image; ** denotes 2-D linear convolution of equation 1, and h(i, j) is the PSF. It is important that the iterative approximation converges to a unique solution. In the Fourier frequency domain equation 6 can be expressed as (Lim, 1990)

Fk(G>1} G>2) =[0(0)!, 0)2)/H(0)1 , 0)2)] [1 - (1 - X H(0)!, 0)2))k+1 ] (7)

As k approaches infinity, F'k(o)1} o^) converges to G(O)I} o^) / H(o)l5 o^), i.e. the inverse filter restoration

Page 5: Iterative Restoration Deblurring of SPOT Panchromatic Images · Iterative Restoration Deblurring of SPOT Panchromatic Images ... Image restoration has been defined by Andrews

provided that (Schafer and others, 1981)

ll-XHCoh.ofc) I <1 (8)

If X is real and positive, then the condition for convergence is

Re [ H(o>i, 0)2) ]> 0 (9)

(i. e. the real part of complex H(o>l/ 0)2) is greater than zero). For example if 11(0)!, 0)2) is real and0 < H(<DI, 0)2) then the condition for convergence is satisfied when 0 < X < 2 (Schafer and others, 1981).

Since equation 7 converges to the inverse filter, if the degradation includes noise as in equation 2, there could be excessive noise sensitivity when k is large. One method to lessen noise sensitivity is to limit the number of iterations, thus preventing excessive noise amplification (Lim, 1990; Kawata and Ichioka,1980; Biemond and others, 1990). The partially deblurred image may be an acceptable visual enhancement.

Linear shift-invariant iterative restoration techniques often introduce ringing artifacts near abrupt intensity transitions (Schafer and others, 1981). However, under certain conditions, imposing a non- negative constraint will reduce such artifacts. In addition, this incorporates a priori knowledge that image intensities are known to be positive. Benefits of positivity and other constraints in the restoration of impulsive-like spectral signals with low or zero reference levels were described by Schafer and others (1981). Similarly, a positivity constraint to reduce ringing artifacts might be effective in low intensity regions of an image with abrupt intensity differences.

Schafer and others (1981) developed conditions for convergence when a positivity constraint was imposed on the basic iterative equation 6. Following their notation for one-dimensional sequences,

i) (lOa)

where q( i) = 8( i) - X h(i); 8 (i) is a unit impulse; f' (i), g( i), h( i) are discrete 1-D sequences for integer index i; and * is 1-D linear convolution. P, the positivity constraint operator is

when x_> 0otherwise. ' '

When f(i), the undegraded input is known a piori to be positive, equation 10 with positivity constraint will converge when Re [ H(o)) ] > 0, and ringing artifacts may be reduced. Equation 10 may be implemented as

because positive f 'k+1 ( i ) replaces f 'k( i ) on each successive iteration and positive f'k = P[ f \( i) ].

Another method to reduce ringing and excessive noise amplification, used by Kawata and Ichioka, (1980) was to constrain the solution f' k(i, j) of equation 6 such that f' k(i, j) is close to g(i, j), the degradedinput. The assumption was that the restored image, f'k(i, j), should not differ greatly from the blurred image, i.e.

O,)-dSf'ka)S(i, + d (13)

Page 6: Iterative Restoration Deblurring of SPOT Panchromatic Images · Iterative Restoration Deblurring of SPOT Panchromatic Images ... Image restoration has been defined by Andrews

where d is a constant. Although their iterative equations were in matrix form, to implement the constraint on f 'k(i, j) as in equation 13 they basically replaced acceleration parameter A, of equation 6 with a variableacceleration parameter rk(i, j) where

{K(l - l/d|f 'k (i, j) - g(i, j)|) when | Pk(i, j) - g(i, j) | < d0> J) - f when | f'k(i, j) - g(i, j) |> d (14)

Note that rk (i, j), now functioning as a constraint on restored image f'k (i, j), depends on iteration k and each individual pixel location i, j. When not equal to zero, rk(i, j) is roughly inversely proportional to the absolute value difference between the restored image and the degraded image. Substituting 1^(1, j) for A in equation 6 and, letting the initial guess for f '0(i, j) equal g(i, j) as suggested by Kawata and Ichioka (1980) yields the following variable-acceleration constrained-iterative restoration equation:

) = g(i,j) (15a)

f/k+l(i/ j) = f'k(i/ j) + rk (i/ j) ((g(i/ j) - h(i/ j) ** f'k(i/ j) ) (15b)

This equation with constraint P and an additional constraint described next was used in this work.

In addition to positivity constraint P of equation 11, the maximum value of f' k(i, j) at each k was bounded by means of constraint operator C, where

{ 255 when x > 255 (16)

x otherwise.

This was merely for convenience in order to visualize intermediate results, f ' k+i(i, j), on a 256-grey- leveldisplay. The significance of constraint C relative to the sensor system model and the conditions forconvergence were not considered, other than the fact that the data values of the given imageg(i, j) are between 0 and 255. The effects of C are believed to be minimal except when g(i, j) = 255.

The constrained iterative restoration, with variable acceleration parameter rk(i, j), used herein was

f'k+id j) = C[P[ f'k(i, j) + rk(i, j){( g(i, j) - h(i, j) ** f'k a j)}]] (17b)

where variable acceleration parameter rk(i, j) is that in equation 14, and constraint operators P, C are those of equations 11 and 16 respectively.

For testing, the choice for d in equation 13 leads to two distinct constraint conditions. The first, a nonvariable acceleration case occurs when d » 1; thus rk(i, j) = K. To simplify, let rk(i, j) = K. Thenequation 17 without constraint C is the 2-D equivalent of the positivity constrained 1-D restoration of equation 10. Because they converge the initial choice for f'o(i, j) in equation 17a can be arbitrary; thusf '0(i, j) equals g(i, j). Although constraint C was used with equation 17, as noted previously, its affect on convergence was not considered at this time.

Page 7: Iterative Restoration Deblurring of SPOT Panchromatic Images · Iterative Restoration Deblurring of SPOT Panchromatic Images ... Image restoration has been defined by Andrews

In contrast, the variable acceleration case occurs when d is such that rk(i, j) is not = K. Thus the choice for f o(i, j) is important. Kawata and Ichioka (1980) chose f o(i, j) equal to g(i, j). Also it was necessary in this study to modify equation 14 such that when k = 0, ik(i, j) equals an experimentally determined value less than K. Otherwise rk(i, j) equals K when k = 0 and typically the next estimate, f ' k+i(i, j), derived from f '0(i, j) will differ by more than d from g(i, j). Consequently on the next and all subsequent iterations, rk(i, j) becomes 0 and no further changes or updates are made to f 'k+i(i, j)-

IV. SPOT SENSOR MODEL

Using a given OTF model for the SPOT sensor, this report then describes and compares two distinct techniques to determine a small spatial extent PSF from the OTF model. Alternatively, the PSF might be derived in the spatial domain from edge features in the scene, using for example a variation of the edge gradient technique described by Reichenbach and others (1990).

The starting point for estimating the sensor's PSF having small spatial extent was the OTF model developed by Schowengerdt and Li (1987) because they describe in detail models for the individual components that comprise the OTF. They used a separable model, H(o>i, 02) = Ha(t0i) Hc(o)2) whereHa(t0i), Hc(t02) are the along and across track OTFs respectively, and co^, 0)2 are frequency variables. Then 1-D Fourier transforms may be applied individually to Ha(cO|) and H^o^-

In contrast to 1-D transforms, this report used more general 2-D Fourier transforms previously developed for another application. Discrete Fourier transform (DFT) processing is possible by first sampling the OTF, H(to1 , 0)2) to give the equivalent discrete OTF, H(u, v). Then the finite extent PSF, h(i,j) may be calculated as

h(i/ j) = IDFT[H(u/ v)] (18)

where IDFT is the 2-D, N-by-N sample inverse discrete FT, and u, v are discrete spatial frequency indices. First, the continuous form OTF is described in order to clarify subsequent discussion of the frequency and phase scaling methods used in this work to approximate a small PSF. Briefly, the continuous OTF, H(tOi,0)2) was:

(19)

where Ha(tOi), Hc(o)2) are the along and across track OTF, respectively. Then

H a(t»1 ) = H^O!) H2(co1 ) H3(to1) (20)

H c(o>2)= H4(to2) H5(o)2) H6(o)2) (21)

where HJ for i = 1, 6 are the individual subcomponent OTF's given in table 1. The reader should see Schowengerdt and Li (1987) for further details.

Page 8: Iterative Restoration Deblurring of SPOT Panchromatic Images · Iterative Restoration Deblurring of SPOT Panchromatic Images ... Image restoration has been defined by Andrews

Table 1. Source of blur and OTF model [ Tabular summary of the individual contributions to the sensor system blur and their OTF model ]

OTF

HI«D!)

H2«Di)

H3(G>i)

H4(0>2)

H5(0)2)

H6(0)2)

Source of Blur

Optics, along track

CCD integration, 1-D, along track

Motion, along track

Optics, across track

CCD integration, 1-D, across track

CCD charge transfer efficiency

OTF Model

(real, imaginary)

exp (-(SiO)!)2), 0

sine (S2<0i). 0

sine (83(1)1), 0

exp (-(s40)2)2), 0

sine (850)2), 0

( see equation 22 )

Parameters Si ... 85 fit the OTF model to known data. Note that the OTF model is complex. Chargecoupled device (CCD) integration refers to the blur due to integration, in one dimension, of the finite area of one element of the sensor's linear array. Because the optics OTF was modeled as Gaussian, with circular symmetry, sj = 84. Sinc(x) is the function sin(2xcx) /Zrcx.

The OTF, H^o^) attributed to CCD charge transfer efficiency, as given by Schowengerdt and Li (1987) was in terms of magnitude, IH^o^l and phase response $5(0)2). It is modeled in this report as

I = exp [ - ne( 1 - cos(s6 G^) ) ] = exp [ - S7( 1 - cos(s6 0)2) ) ](22a)

- sin(s5 0)2) = - s7 [ s60)2 - sin(s6 0)2)] (22b)

where ne is the CCD charge transfer inefficiency product and s6 and 87 are scale parameters; see their reference to Sequin and Thompsett (1975) for additional details.

Including the scale parameters, equations 20 and 21 become

Ha(sj, 82,83, (DI)

Hc(s4,85,85,87,0)2) = H4 (s4 0)2^5(850)2) H6(s6,87, o>j)

where HI ... H^ are the continuous model entries of table 1. In this study s^ . comparing plots of HI ... H^ to those in the report by Schowengerdt and Li, (1987).

(23)

(24)

. . 87 were found by

In summary, N discrete samples for Ha(u) and H^v) for spatial frequency indices u, v were calculated from equations 23 and 24 by letting o^ = 2rc(u/N) and 0)2 = 2rc(v/N), where -N/2 < u,v < N/2+1 and normalizing to 1, the sampling frequency or Nyquist rate (fs = o/Zrc) of one sample per period of the individual CCDsensing element. Both across- and along-track periods are assumed equal. Samples of the DFT were created by imposing conjugate symmetry on Ha(u), Hc(v) to give real h(i, j), and by observing for an evenfunction that the end point sample of the DFT has zero value imaginary component See Harris (1978) and

Page 9: Iterative Restoration Deblurring of SPOT Panchromatic Images · Iterative Restoration Deblurring of SPOT Panchromatic Images ... Image restoration has been defined by Andrews

Gonzalez and Wintz (1977) on DFT details. Thus the 2-D sensor model of the discrete OTF is

H(u/V) = Ha(u)Hc(v). (25)

V. SMALL EXTENT PSF

In general, a discrete estimate for the PSF can be calculated as the IDFT of H(u, v) using equation 18, however, there will be N by N samples. Although many of these samples are zero, it is desirable that the spatial size of the PSF be small in order to speed up the iterative computations as well as to reduce border effects due to the simple approximation (see section VI) made for the linear convolution.

Two methods to obtain a small spatial extent PSF size were tested. They were the common window method (Lim, 1990) and an experimental frequency domain-spatial domain scaling (FS) method. The former smoothly limits the PSF size to that of a predefined window. In contrast, preliminary experiments show the FS method leads directly a small size PSF. In both cases, the elementary criterion was that the OTF calculated from the small size PSF be close to the original discrete OTF model H(u, v). That is,

DFT[h'(i,j)] = H(u,v) (26)

where h'(i, j) is the reduced size PSF. With the window method, h'(i, j) was calculated in two steps. First N by N samples of h(i, j) were calculated as IDFT[ H(u, v) ]. Next, the extent was limited by multiplying h(i, j) by a window function w(i, j) much smaller than N by N to give reduced size h'(i, j). Since H(u, v) was separable, (i.e. H(u, v) = Ha(u) Hc(v)) a separable window function w(i, j) = w(i) w(j) is appropriate (Lim,1990). A Harm window, w(n) was applied where

{ cos2 ( P) when |n| < L

0 otherwise <27>and n= -N/2 ,..., -1,0,1,..., N/2. For integer L the extent of w(i, j) h(i, j) is 2L -1 by 2L -1. A typical windowed spatial extent was 9 by 9; N was 256.

A second, experimental frequency scaling method FS, developed herein to generate a small size PSF, relies in part on the scaling property of the Fourier transform: a linear contraction of the frequency axis results in a linear expansion of the spatial domain axis (Pratt, 1978). Hence if the ®i and 0)2 axes of H (o>l, o>2) are contracted by a factor of two, then IDFT [ H(u, v) ] yields a PSF stretched by a factor of 2 inthe spatial domain. Roughly, it is the PSF of a sensor with blur degradation twice that of the original model. The frequency scaling was implemented by letting si... 85 equal 2sj... 2s£ in the OTF models oftable 1. Also, experiments showed that closer agreement between the phase response of the DFT [ h'(i, j)] and the phase response of the original OTF model was possible by also linearly scaling the phase model. Thus 87 of equation 22b was replaced with 257. Further study of this scaling method is underway.

Next, the stretched PSF was first low-pass-filtered to minimize aliasing, and then down-or sub- sampled by a factor of two in two dimensions. This yielded a spatially compact PSF at the desired original scale. Because sub-sampling in effect stretches the frequency response by a factor of two (Vetterli, 1991), this PSF now represents H(u, v) at the original scale. Although the resultant PSF is very compact, its size was then truncated, to 5 by 5 samples, with negligible affect. This is illustrated in section VI, below. In effect, this truncation merely defines the PSF size.

Page 10: Iterative Restoration Deblurring of SPOT Panchromatic Images · Iterative Restoration Deblurring of SPOT Panchromatic Images ... Image restoration has been defined by Andrews

The low-pass spatial filtering for aliasing reduction was implemented with circular convolution and is thus consistent with the periodic properties of the discrete DFT, (Lim, 1990). The separable spatial filter weights were -0.05, 0.25, 0.60, 0.25, -0.05. The frequency attenuation, calculated from the DFT, was - 3.1 dB and -12.0 dB at 1/2 n and 3/4 rc, respectively, where 2 rc is the sampling frequency.

VI. EXPERIMENTS

All Fourier transform processing was with an N by N, (N = 256), DFT and IDFT implemented as separable in row and column with a 1-D fast Fourier transform (Gonzalez and Wintz, 1977).

Figure 1 shows the modulation transfer function (MTF), i. e.l H(u, v) I, and phase response plots of the SPOT-1 HRV 2 panchromatic sensors's OTF, H(u, v). It is the model used in this work, as explained in section IV, determined by adjusting scale parameters such that the OTF, as a function of scale and frequency, fits the data published by Schowengerdt and Li (1987). Figure 1 (a), (b), and (c) are, respectively, along-track MTF, I Ha(u, 0) I, across-track MTF, I Hc(0, v) I, and phase response as degrees ofphase lag. The frequency axis ranges from 0 to 1/2 the normalized Nyquist rate fs of one cycle per sensorelement sample period. The period is equal in both dimensions. MTF values from the published data at f = 1/4,1/2 are shown as and give a rough, relative measure of accuracy for the model used here.

Figure 2 shows results when the window method described in section V was used to limit the spatial extent of PSF, h(i, j). Figure 2 (a), (b), and (c) show, respectively, MTF along and across track, and phase response calculated from the DFT of the windowed h(i, j). Specifically, these data were calculated as DFT[ w(i, j) IDFT[ H(u, v) ] ]; w(i, j) was the Harm window, equation 27, and h(i, j) =IDFT[ H(u, v) ]. Data for PSF extents 5x5,7x7, and 9 x 9 are labeled 5,7,9, respectively. For comparison, the reference data of figure 1 (a), (b), (c) are also shown here and labeled "r".

In contrast to figure 2, figure 3 shows results for the frequency scaling method. The MTF and phase response plots, labeled "f', were derived from DFT [ hf(i, j) ] where small extent PSF hf(i, j) wascalculated by the frequency scaling, spatial filtering-subsampling method (FS) of section V without spatial truncation. Plots labeled "t" show results when small hf(i, j) was then truncated to extent 5 by 5. Plots "t"and "f' are indistinguishable. For reference, corresponding plots of figure 1 (a), (b), (c) are included in figure3.

Comparing figures 2 and 3, note that MTF for the FS method more closely matches the reference than that for the window method. The difference in the phase data of figure 3 is subject to further study.

The assumed minor effects of truncation are shown in table 2, which compares a measure of the spatial size of the PSF developed by the FS method without truncation against the nonwindowed calculation of the PSF from IDFT [ H(u, v) ]. The latter has the assumed worst-case spatial extent. In each case, PSF h(i, j) has N by N samples; hopefully only a few are nonzero. A measure of the compactness of nonzero samples of the PSF relative to its total size and as a function of a square region R surrounding its center element was defined as

10

Page 11: Iterative Restoration Deblurring of SPOT Panchromatic Images · Iterative Restoration Deblurring of SPOT Panchromatic Images ... Image restoration has been defined by Andrews

SR =N N

(28)

where h'(i, j) equals either: hf(i, j) via the FS method, without truncation, or h(i, j) = IDFT[ H(u, v) ],equation 18; N = 256 and R has dimensions r x r. Note that for the FS method, 0.9989 of the total sum I hf(i, j) I is within 5x5 region. Hence hf(i, j) was truncated to 5 x 5 samples.

Table 2. Comparison of fraction of lh(i, j)l in region R for frequency scaling method and IDFT method, (i.e. h(i, j) = IDFT [ H(u, v) ]).

R

1x1

3x3

5x5

7x7

9x9

11x11

SR,[FS method]

0.3380

0.9690

0.9989

1.0000

1.0000

1.0000

SR, [IDFT method]

0.3019

0.8450

0.8760

0.8964

0.9082

0.9168

Figure 4 shows the 2D MTF, I H(u, v) I as modeled in this work and given by equation 25. Spatial frequencies u,v range from -0.5 to 0.5 cycles per sensor element period.

Figure 5 illustrates the relative size of the PSFs. Figure 5 (b) is hf(i, j) via the FS method withouttruncation while figure 5 (a) is h(i, j) calculated as IDFT[ H(u, v) ]. The extent of the latter could be limited by windowing. The absolute value of the PSF amplitude is shown on a logarithmic scale ranging from 0 dB to -46 dB; black, mid-grey and white are 0, -23 dB, and -46 dB, respectively.

All remaining iterative restoration examples used equation 17 with acceleration parameter rk(i, j), equation 14. The 2-D convolution of equations 17 and 1, was approximated as

fifi _l i _ ml tn m^ (29)

(1, m) R

where R is the square, truncated to 5 x 5 sample spatial extent of PSF hf(i, j) calculated via the FS method and applied only to a subregion of the image to reduce boundary effects.

There were two test conditions for acceleration parameter rk(i, j): 1) fixed, equal to a constant value(s), or 2) variable as per equation 14. Table 3 summarizes test values for rk(i, j), A,, d when k = 0 and k>0.

11

Page 12: Iterative Restoration Deblurring of SPOT Panchromatic Images · Iterative Restoration Deblurring of SPOT Panchromatic Images ... Image restoration has been defined by Andrews

Table 3. Summary of test conditions for rk(i, j), X, and fk(i, j)

'fixed'

acceleration

'variable'acceleration

k

0>0

0>0

rk(i,j)

0.5= 1.0

0.5rk(iJ)

X

1.01.0

1.01.0

d

106106

10.010.0

fk(iJ)

g(ij)fk(iJ)

g(ij)fk(iJ)

When k = 0, fk+i (i, j) is the first deblurred estimate of the restored image. Also preliminary, 'fixed' acceleration tests showed that rk=0(i, j) = 0.5 improved the rate of convergence.

In Figures 6, 7, 8 the degraded image was generated by convolving a simulated image (figure 10 (a)) with the modeled blur PSF as per equation 28. Figure 6 shows fixed-acceleration restoration of a noiseless, blurred image after 5, 10, 15, and 25 iterations. Note that the restored profile plot after 25 iterations appears nearly the same as the original plot (a).

Figure 7 shows fixed-acceleration restoration of the blurred image plus zero-mean white Gaussian noise of variance 2.0 (Kay, 1988). Noise amplification typical to inverse filter restoration is evident in plot (e) for 15 iterations. Plot (c), 5 iterations, shows perhaps reasonable deblurring with only moderate noise amplification.

Figure 8 shows variable-acceleration results for the same image plus noise of figure 7 (b). Compared to figure 7, figure 8 has less noise amplification, but less deblurring. Comparing figures 7 (c) and 8 (c) for 5 iterations, fixed-acceleration data show more deblurring with comparable noise.

Figure 9 shows RMS error vs iterations for the profiles shown in figures 6,7, and 8. Two errors are shown: error 1 for {g(i, j) - h(i, j)**fk(i, j)}, (plots 1,2, 3), measured on the 1-D profile shown; and error 2for { g(i, j) - fk(i, j) }, (plots 4, 5), measured in a constant intensity, 7x7 pixel region of the 2-D image andcentered on the left most part of the profile. Error 1 is a rough measure of restoration error, error 2 is a measure of the noise amplification. Not shown is error 2 for blur without noise; it is zero. Note that for fixed- acceleration, with noise, after about 7 iterations, error 1 (of plot 2) is relatively low and error 2 (of plot 5) might not be excessive. This might be an acceptable deblurring enhancement.

Figure 10 shows image restoration examples after 25 iterations, corresponding to figures 6, 7, 8. Deblurring of the one-pixel-wide line, for fixed acceleration is evident in both (d) and (e).

For the following examples, the SPOT 10-m-resolution panchromatic image intensities were first linearly transformed such that the minimum value (24) and the mean value plus 6 standard deviations (180) become 0 and 255 respectively. Because much of the sample image now has low intensity with abrupt increases in intensity, the positivity constraint of equation 11 might be effective in reducing ringing artifacts. Since few intensity values are 255, constraint C, equation 16, will have little effect.

Figure 11 shows several restoration examples for a 128 x 128 sample of a SPOT-1, HRV 2, level 1 A, panchromatic image for fixed and variable acceleration.

12

Page 13: Iterative Restoration Deblurring of SPOT Panchromatic Images · Iterative Restoration Deblurring of SPOT Panchromatic Images ... Image restoration has been defined by Andrews

Figure 12 gives RMS error vs iterations for the SPOT image of figure 10 (a). Plots 2, 3,4, 5 were for the same restoration conditions as in figure 9. Error 1, (plots 2, 3) was measured on the entire image, less a 10 pixel border. Error 2, (plots 4,5) was measured in a 7x7 pixel area of relatively constant intensity, of mean 17.85, standard deviation 1.79. Note that after about 5 to 7 iterations, error 1, plot 2 is nearly constant; this might give reasonable deblurring with moderate noise amplification.

Figure 13 shows two row intensity profiles for an isolated bright pixel in the image of figure 11: before and after 8 iterations, fixed-acceleration restoration. Deblurring is clearly evident.

Figure 14 shows, for fixed-acceleration, rk(i, j), images related to the errors of figure 12, foriterations k = 0, 2, 4, 6. The upper left is the restored result after one iteration. Columns 2, 3 are respectively rk(i, j) { g(i, j) - h(i, j)**fk(i, j)}+ 128 and { fk+1 (i, j) - g(i, j)} + 128; their intensity scales are exaggerated for display. Column 2 is roughly the spatial detail added to the previous restored image fj^i, j) to give fk+i(i» j)- Column 3 shows both restored details and noise amplification.

Figure 15, like figure 14, is a sequence of restored images, however for variable-acceleration. Column 4 shows rk(i, j) on a scale of 0 to 255. Column 2 is the spatial detail, constrained by rk(i, j), which is added to fk(i, j) to give fk+i(i, j). Comparing column 2 of figures 14 and 15, less detail is restored for variable acceleration.

Figures 16 and 17 show before and after restoration by fixed acceleration (8 iterations), respectively, for a 256 x 256 sample SPOT image.

VIL CONCLUSION

The iterative restoration process requires an estimate of the blur degradation point spread function (PSF). There are computational advantages if the PSF has small spatial extent. Starting with a given optical transfer function (OTF) model, an experimental technique was used, which required frequency scaling the OTF model, spatial filtering the inverse Fourier transform of the scaled OTF, and subsampling to give the desired small-size PSF. The corresponding modulation transfer function better matched the original model than results based on a window technique. Although encouraging, the calculated phase response only closely agrees with the original model and is the subject of further study.

Previously reported iterative restoration algorithms with either fixed constraints or variable acceleration parameter constraints were applied to SPOT level 1A, 10-meter-resolution panchromatic band images. They produce, in the limit for fixed constraints, inverse filter restoration. Although variable acceleration reduced noise amplification, there was less deblurring than iterative inverse filtering. With simple fixed constraints, limiting the iterative process to a few iterations prevented excessive noise amplification and gave reasonable deblurring for the moderate blur degradation modeled here.

13

Page 14: Iterative Restoration Deblurring of SPOT Panchromatic Images · Iterative Restoration Deblurring of SPOT Panchromatic Images ... Image restoration has been defined by Andrews

REFERENCES

Andrews, H. C., 1974, Digital Image Restoration - A Survey: Computer, v. 7, no. 5, p. 36-45

Biemond, J., Lagendijk, R. L., and Mersereau, R. M., 1990, Iterative Methods for Image Deblurring: Proc. of the IEEE, v. 78, no. 5, p. 856-882.

Gonzalez, H. C., and Wintz, P., 1977, Image Transforms, chap. 3 of Digital Image Processing: Boston, Addison-Wesley, p. 36-114.

Harris, F. J., 1978, On The Use of Windows for Harmonic Analysis with the Discrete Fourier Transform: Proc. of the IEEE, v. 66, no. 1, p. 51-83.

Kawata, S., and Ichioka, Y., 1980, Iterative Image Restoration for Linearly Degraded Images, I. Basis: J. Opt. Soc. Am., v. 70, no. 7, p. 762-768.

Kay, S. M., 1988, Computer Program to Generate Real White Gaussian Noise, Appendix 5A of Modern Spectral Estimation: Englewood Cliffs, New Jersey, Prentice Hall, p. 145-147.

Lim, J. S., 1990, Two-Dimensional Signal and Image Processing: Englewood Cliffs, N. J., Prentice Hall, 694 p.

Park, S. K., 1992, Image Gathering, Interpolation And Restoration - A Fidelity Analysis: Visual Information Processing, SPIE, v. 1705, p. 134-144.

Pratt, W. K., 1978, Continuous Image Characterization, chap. 1 qf Digital Image Processing: New York, Wiley, p. 1-49.

Reichenbach, S. E., Park, S. K., and Narsayanswamy, R., 1990, Characterizing Digital Image Acquisition Devices: Opt. Eng., v. 30, no. 2, p. 170-177.

Reichenbach, S. E., and Park, S. K., 1991, Small Convolution Kernels for High Fidelity Image Restoration: IEEE Trans. Signal Processing, v. 39, p. 2263-2274.

Schafer, R. W, Mersereau, R. M., and Richards, M. A., 1981, Constrained Iterative Restoration Algorithms: Proc. of the IEEE, v. 69, no. 4, p. 432-450.

Schowengerdt, R., and Li, W. M., 1987, Research in Image Restoration and Resampling for the SPOT Remote Sensing System: U. S. Geological Survey, Coop Agreement 14-08-000l-A-0330, 60 p.

Sequin, C. H., and Tompsett, M. F., 1975, Transfer Inefficiency, chap. 4 of Charge Transfer Devices, New York: Academic Press, p. 70-108.

SPOT Image Corporation, 1988, SPOT User's Handbook: Reston, Vir., v. 2, chap. 1.

Tekalp, M. A., and Trussell, J. H., 1991, Comparative Study of Some Statistical and Set-Theoretic Methods for Image Restoration: Computer Vision, Graphics and Image Processing, v. 53, no. 2, p. 108-120.

Vetterli, M., 1991, Multirate Filter Banks for Subband Coding, chapt. 2 of Woods, J. W, ed., Subband Image Coding: Massachusetts, Kluwer Acad. Pub., p. 43-100.

14

Page 15: Iterative Restoration Deblurring of SPOT Panchromatic Images · Iterative Restoration Deblurring of SPOT Panchromatic Images ... Image restoration has been defined by Andrews

MTF, Across Track 0.0

-30.0 -

-60.0

0.1 .

0.5

Phase, Across Track, degrees

0.1frequency

(a)frequency

(b)frequency

(c)

Figure 1. Modulation transfer function (MTF) and phase response for SPOT-1, HRV 2 sensor. These data were developed from the transfer function, (TF) model of equations 23 and 24 after first determining scale parame­ ters such that individual TF graphs agreed with reference data given by Schowengerdt and Li (1987). MTF val­ ues from the published data at frequencies 0.25,0.5 are shown as and give a rough, relative measure of accuracy for the model used here. Plots (a) and (b) show I H(u,v) I, or MTF, calculated from equation 25 for normalized spatial frequencies (cycles per sensor element) u, v in the along- and across-satellite-track direc­ tions, respectively. Plot (c) shows the phase response, calculated from equation 25, as degrees lag in the across- track direction attributed to charge transfer efficiency of the linear array CCD sensor. The phase response in the along-track direction is zero. These data were the desired reference, or design objective, for a small extent spa­ tial filter that characterizes the sensor's blur.

0.5

0.1-

MTF, Across Track 0.0

-30.0 .

-60.0Phase, Across Track, degrees

0.1 0.1 0.5 0.1frequency

(a)frequency

(b)frequency

(c)

Figure 2. Results for window method filter design. Plots (a), (b), (c) show I H(u,v) I and phase response deter­ mined from the discrete Fourier transform (DFT) of the point spread function, h(i, j), whose spatial extent was limited with a Harm, or Cos2(-) window function as described in section V. Specifically this data was cal­ culated as DFT[ w(i, j) IDFT[ H(u, v) ] ] where IDFT denotes inverse DFT, w(i, j) was the finite extent win­ dow function, H(u, v) was that of equation 25 as shown in figure 1, and h(i, j) was IDFT[ H(u, v) ]. Plots for window sizes 9 x 9,7 x 7 and 5 x 5 are labeled 9,7, and 5, respectively. Also, the reference plots of figure 1 (a), (b), (c) are shown here as r.

Page 16: Iterative Restoration Deblurring of SPOT Panchromatic Images · Iterative Restoration Deblurring of SPOT Panchromatic Images ... Image restoration has been defined by Andrews

1.0 MTF, Across Track

0.5 -

0.1 -

-30-

-60-

Phase, Across Track, degrees

.5frequency

(a)

.1frequency

(b)

.5 .1frequency

(c)

.5

Figure 3. Results for frequency scaling filter design. Plots (a), (b), (c) show I H(u, v) I, or MTF, and phase response determined from the DFT of the small spatial extent point spread function h'(i, j); i. e. DFT[ h'(i, j) ]. In this case h'(i, j) was calculated by the frequency scaling, spatial filter­ ing and sub-sampling technique of Section V. Specifically (a), (b), and (c) each contain three plots labeled r, f, t for the conditions: "r" reference data, of figure 1; "f' MTF and phase response results for small h'(i, j) found via the frequency scaling technique; and "t" MTF and phase response results where the spatial extent of small h'(i, j) of (f) was then truncated to size 5x5. The plots for f and t are indistinguishable. Comparison between figure 2 (a), (b) and figure 3 (a), (b) shows that a 9 x 9 size spatial filter developed by the window method has comparable MTF to the desired small, 5x5, size filter developed by frequency scaling method.

0.5Figure 4. MTF, I H(u, v) I, of SPOT -1, HRV 2 panchromatic sensor system calcu­ lated from equation 25. Profile plots of this data were given in figure 1 for u,v =u,0 and u,v = 0,v. The along and across satellite track MTF's correspond to u, 0 and 0,v axes respectively. Both u and v range from -0.5 to 0.5 cycles per sensor element.

(a) across track (b)

Figure 5. Point spread function comparison. Both (a), (b) show 19 x 19 samples of the IPSFI on a scale of 0.0 dB (black) to -46.0 dB (white), (a) is the PSF found as h(i, j) =inverse DFT[ H(u,v) ] for H(u, v), of equation 25; I H(u, v) I is shown in figure 4. A window func­ tion, equation 27, would then limit its spatial extent (b) is the PSF found via the frequency scaling technique of section V, without trunca­ tion. Note in (b) the smaller size PSF; all sample values beyond the central 5 x 5 are < -46 dB relative to the center value.

Page 17: Iterative Restoration Deblurring of SPOT Panchromatic Images · Iterative Restoration Deblurring of SPOT Panchromatic Images ... Image restoration has been defined by Andrews

250.00-

200. 00-

150.00-

100 00-

50. 00-

0.00-

(a) Original

0.00 10.00 20.00 30.00 40.00 50.00 £0.00 70.00 80.00 90.00 100.00 110.00

Pixel Location250.001

(b) Blurred

0.00 10.00 20.00 30 00 40 00 50.00 60.00 70.00 80.00 90.00 100 00 110.00

250.00-

200. 00-

150. 00-

100. 00-

SO. 00-

0.00-

, (c) Restored: 5M

!fr«

0. QO 10.00 20 00

250.00-

200. 00-

150.00-

100.00-

SO. 00-

0 DO-

i i i i 30.00 40 00 50.00 60.00 70.00 80.00 90.00 100.00 110.00

, (d) Restored: 10

0.00 10.00 20.00 30.00 40.00 SO. 00 60.00 70.00 80.00 90.00 100.00 110.00

250. 00-

200 00-

150 00-

100.00-

50. 00-

0.00-j

(e) Restored: 15

0.00 10.00 20.00 30.00 40.00 50.00 60.00 70.00 80.00 90.00 100.00 110.00

250 oo-, , , , , , , , . , (f) Restored: 25200. 00-

1SO.OO-

100. 00-

£0.00-

0.00-0.00 10.00 20.00 30.00 40.00 50.00 60.00 70.00 80.00 90.00 100.00 110.00

Figure 6. Restoration of simulated, noiseless image, (a) Row of the original image, (b) Degraded image made by convolving the original image with the 5 x 5 PSF found via the frequency scaling method of section V. (c), (d), (e), and (f) Restorations of (b) after 5,10,15,25 iterations, respectively. Here the variable acceleration parameter rk(i, j), equation 14, was held constant at: 0.5 for k = 0 or 1.0 fork>0.

Page 18: Iterative Restoration Deblurring of SPOT Panchromatic Images · Iterative Restoration Deblurring of SPOT Panchromatic Images ... Image restoration has been defined by Andrews

2SC. CO-

200 DO-

150. 00-

100. CO-

50. 00-

(a) Original

c.oo-0.00 10.00

Pixel Location

250. DO-

200.00-

1SO.OO-

100.00-

50. DO-

20.00 30.00 40.00 50.00 60.00 70.00 80.00 90.00 100.00 110.00

0.00

(b) Blurred plus noise

250.00-

200. 00-

150. 00-

100.00-

50.00-

0.00-

250. DO-

200. 00-

150. DO-

100. 00-

50.00-

0.00 10.00 20.00 30.00 40.00 50.00 60.00 70.00 80.00 90.00

, (c) Restored: 5100.00 110.00

0.00 10.00 20.00 30.00 40.00 50 00 60.00 70.00 80.00 90.00 100.00

(d) Restored: 10

0.00-

110.00

250. DO-

200. 00-

150.00-

100.00-

£0 00-

0.00-

0.00 10.00 20.00 30.00 40.00 50.00 60.00 70.00 80.00 90.00 100.00

(e) Restored: 15110.00

0.00 10.00 20.00 30.00 40.00 50.00 60.00 70.00 80. 00 90.00 100. 00 110.00

Figure 7. Restorations of simulated image plus noise. Plot (a) shows a row profile of the original image; it is the same as figure 6 (a). Plot (b) shows a blurred image profile, as in figure 6 (b), plus zero-mean white Gaussian noise, with variance = 2.0. Plots (c), (d), and (e) show restorations of (b) after 5,10, and 15 iterations, respectively. Acceleration parameter rk(i, j) was held fixed to the same conditions as in figure 6. This example, approaching inverse filter restoration when k » 0, shows moderate noise amplification. As with figure 6, these profile example are for across-track blur.

Page 19: Iterative Restoration Deblurring of SPOT Panchromatic Images · Iterative Restoration Deblurring of SPOT Panchromatic Images ... Image restoration has been defined by Andrews

250. DO-

200. 00-

150.00-

100.00-

50. 00-

0.00-

(a) Original

0.00 10.00

rtxel Location250.00-

200. 00-

150.00-

100. 00-

50. 00-

0.00-

20.00 30.00 40.00 50.00 60.00 70.00 80.00 90.00 100.00 110.00

, (b) Blurred plus noise

250. DO-

200. 00-

150.00-

100.00-

50. 00-

0.00-

0.00 10.00 20.00 30.00 40.00 50.00 £0.00 70.00 80.00 90.00 100.00 110.00

, (c) Restored: 5

0.00 10.00 20.00 30.00 40.00 50.00

250.00-

200.00-

150. 00-

100. 00-

50.00

0.00-

60.00 70.00 80.00 90.00 100.00 110.00

, (d) Restored: 10

0.00 10.00 20.00 30.00 40.00 50.00 60.00 70.00 80.00 90.00 100.00 110.00

250.00-

200. DO-

150. 00-

100.00-

50.00-

0.00-

, (e) Restored: 15

0.00 10.00 20.00 30.00 40.00 50.00 60.00 70.00 80.00 90.00 100.00 110.00

Figure 8. Restoration of simulated image plus noise using variable acceleration. Plots (a) and (b) are the same as fig. 7. (a), (b). In this example rk(i, j) was 0.5 when k = 0; otherwise rk(i, j) was as given by equation 14 for k > 0. That is r^i, j) was a function of I degraded -restored I image. More deblur- ring is evident in figure 7, the fixed acceleration condition, than here.

Page 20: Iterative Restoration Deblurring of SPOT Panchromatic Images · Iterative Restoration Deblurring of SPOT Panchromatic Images ... Image restoration has been defined by Andrews

Figure 9. RMS error per iteration k for simulated data.

Plots 1,2 and 3, calculated at the middle row of the images for figure 10, give a measure of restoration error. Plots 4,5 calculated in the left-most constant intensity area give a rough measure of the increased noise. Test conditions were:

Plot

12

3

4

5

RMS Error

g(i,j)-h(i,j)**fk(i,j)»

»

g(U) - fk+l(i J)it

Noise

no

yes

yes

yes

yes

Tk(iJ)

fixed

fixed

variable

variable

fixed

Iterations, k

The degradation model, equation 1, was g(i, j) = h(i, j)**f(i, j) where f(i, j), g(i, j) are the original and blurred images, respectively, h(i, j) was the PSF via section V, and ** denotes 2-D convolution.

(a) Original (b) Blurred (c) Blurred + Noise

(d) Restored b; fixed accel. (e) Restored c; fixed accel. (f) Restored c; variable accel.

Figure 10. Restoration of simulated images after 25 iterations, (a) Simulated original, (b) Blurred original, (c) Image (b) plus zero-mean white Gaussian noise of variance 2.0. Image (b) was simulated by convolving (a) with the SPOT PSF determined via the FS method of section V. Profiles in the across-track direction (horizon­ tal) were shown in figures 6,7, 8. Images are 128 x 128 pixels.

Page 21: Iterative Restoration Deblurring of SPOT Panchromatic Images · Iterative Restoration Deblurring of SPOT Panchromatic Images ... Image restoration has been defined by Andrews

(c) Restored: 8 it.; fixed accel

(b) Restored: 5 iterations (it.); fixed acceleration

(d) Restored: 25 it.; fixed accel.

(e) Restored: 8 it.; variable accel.; d=10.0 (f) Restored: 25 it.; variable accel.; d=10.0

Figure 11. Restoration examples with SPOT HRV 10-m pixel image (© 1988 CNES, licensed by SPOT Image Corpora­ tion, Reston, Virginia).

Page 22: Iterative Restoration Deblurring of SPOT Panchromatic Images · Iterative Restoration Deblurring of SPOT Panchromatic Images ... Image restoration has been defined by Andrews

11.00

10. 00-

9.00-

B.OO-

7.00-

i 6.00-I

I

| 5. 00\

4.00-

3.00-

2.00-

1.00-

0.00

0.00 5.00 10.00 15.00

Iterations, k i i

20.00 25.00

Figure 12. RMS error per iteration, k, for figure 11 images.

For the SPOT image of figure 11 (a), plots 2 and 3 give a measure of restoration error while 4 and 5 give a rough measure of the increased noise. Error plots 2 and 3 were found for the entire image, less a 10-pixel-wide border. Error plots 4 and 5 were calculated in a relatively constant- intensity 7x7 pixel area with mean and std. dev. = 17.85, 1.79 respectively. Test conditions were:

Plot

2

3

4

5

RMS Error

g(i,j)-h(i,j)**fk(i,j)«

g(i.J)- fkH-l(i.J)«

Tk(iJ)

fixed

variable

variable

fixed

Plot numbers 2,3,4, and 5 also correspond to those of fig­ ure 9. The acceleration parameter rk(i, j) of equation 14, with d= 10.0, was either fixed or variable.

260. DO-

240. 00

0.00

Pixel location, j 90 100

Figure 13. Intensity profiles for the raw SPOT image of figure 10 (a), and the fixed-acceleration restored image(8 iterations) of figure 10 (c). The profiles were from row i = 64, columns j = 78,..., 102 relative to upper left cornerof the image.

Page 23: Iterative Restoration Deblurring of SPOT Panchromatic Images · Iterative Restoration Deblurring of SPOT Panchromatic Images ... Image restoration has been defined by Andrews

i, j)-h(i, j)**fk(i, j)l +128 +128

k = 0

k = 2

k = 4

k = 6

Figure 14. Image restoration sequence for fixed acceleration (parameter %(i, j) = 1.0). Column 1 shows the restored image fk+i(i, j) resulting from the calculation at iteration k = 0, 2, 4, or 6. Col­ umns 2, and 3 show, respectively, the difference images ^(i, j) [ g(i, j) - h(i, j)**fjc(i, j) ] +128 and [fk+l(i» J) ~ gO» J)l + 128. Column 2 is the spatial information added to the the previous iteration result, ffc( i, j), to give ffc+ i(i, j). Note that as k increases, finer spatial details appear in the images of column 2. The column 3 images show both increased spatial detail and noise.

Page 24: Iterative Restoration Deblurring of SPOT Panchromatic Images · Iterative Restoration Deblurring of SPOT Panchromatic Images ... Image restoration has been defined by Andrews

k = 0

k = 2

k = 4

k = 6

Figure 15. Image restoration sequence for acceleration parameter rk(i, j) variable. Column 1 shows the restored image fk+i(i, j) resulting from the calculation at iteration k = 0, 2,4, or 6. Column 4 shows variable acceleration parameter rk(i, j) (equation 14 with threshold d = 10.0) on a scale of 0 to 255. Column 2 shows the product rk(i, j) times difference image [g(i, j) - h(i, j)**fk(i, j)] + 128. It is roughly the spatial information, constrained by the parameter rk(i, j), which is added to the the previous iteration result, fk( i, j), to give fk+ i(i, j). Column 3 shows [ffc+ i(i, j) - g(i, j)] + 128. For k > 0, the spatial detail in column 2 imagery tends to remain relatively low and constant. The col­ umn 3 images show only minor increases in both spatial detail and noise.

Page 25: Iterative Restoration Deblurring of SPOT Panchromatic Images · Iterative Restoration Deblurring of SPOT Panchromatic Images ... Image restoration has been defined by Andrews

Figure 16. Raw SPOT panchromatic image sample, 256 x 256 pixels (© 1988 CNES, licensed by SPOT Image Corporation, Reston, Virginia).

Page 26: Iterative Restoration Deblurring of SPOT Panchromatic Images · Iterative Restoration Deblurring of SPOT Panchromatic Images ... Image restoration has been defined by Andrews

Figure 17. Example of iterative restoration applied to the SPOT image shown in figure 16. Condi­ tions were 8 iterations and fixed acceleration (© 1988, licensed by SPOT Image Corporation, Reston, Virginia.)