ARTICLE IN PRESS

0165-1684/$ - se

doi:10.1016/j.sig

�Correspondfax: +9133 577

E-mail addr

Signal Processing 86 (2006) 924–936

www.elsevier.com/locate/sigpro

A simple and efficient algorithm for multifocus imagefusion using morphological wavelets

Ishita Dea, Bhabatosh Chandab,�

aDepartment of Computer Science, Barrackpore Rastraguru Surendranath College, Kolkata 700120, IndiabElectronics and Communication Sciences Unit, Indian Statistical Institute, Kolkata 700108, India

Received 11 October 2004; received in revised form 16 July 2005

Available online 25 August 2005

Abstract

This paper presents a simple yet efficient algorithm for multifocus image fusion, using a multiresolution signal

decomposition scheme. The decomposition scheme is based on a nonlinear wavelet constructed with morphological

operations. The analysis operators are constructed by morphological dilation combined with quadratic downsampling

and the synthesis operators are constructed by morphological erosion combined with quadratic upsampling. A

performance measure based on image gradients is used to evaluate the results. The proposed scheme has some

interesting computational advantages as well.

r 2005 Elsevier B.V. All rights reserved.

Keywords: Multifocus image fusion; Multiresolution signal decomposition; Morphological wavelets; Image gradient

1. Introduction

A wide variety of data acquisition devices areavailable at present, and hence image fusion hasbecome an important subarea of image processing.There are sensors which cannot generate images ofall objects at various distances (from the sensor)with equal clarity (e.g. camera with finite depth offield, light optical microscope, etc.). Thus severalimages of a scene are captured, with focus on

e front matter r 2005 Elsevier B.V. All rights reserve

pro.2005.06.015

ing author. Tel.: +9133 577 8085;

6680.

ess: chanda@isical.ac.in (B. Chanda).

different parts of it. The acquired images arecomplementary in many ways and a single one ofthem is not sufficient in terms of their respectiveinformation content. However, viewing a series ofsuch images separately and individually is not veryuseful and convenient. The advantages of multi-

focus data can be fully exploited by integrating thesharply focused regions seen in the differentimages. Before integrating, one must bring theconstituent images to a common coordinatesystem. Widely used techniques perform this taskby using some common geometrical referencescalled ground control points (GCPs). This processis called registration. After registration the images

d.

ARTICLE IN PRESS

I. De, B. Chanda / Signal Processing 86 (2006) 924–936 925

are combined to form a single image through ajudicious selection of regions from differentimages. This process is known as multifocus image

fusion. Both the reliability of redundant informa-tion and the quality of complementary informa-tion present in the constituent images areimproved in a fused image. So it gives a betterview for human and/or machine perception. Afused data can also render itself more successfullyfor any subsequent processing like object recogni-tion, feature extraction, segmentation, etc.

There are a number of techniques for multifocusimage fusion. Simple techniques in which thefusion operation is performed directly on thesource images (e.g. weighted average method),often have serious side effects like reduction in thecontrast of the fused image. Other approachesinclude, image fusion using controllable camera[1], probabilistic methods [2], image gradientmethod with majority filtering [3], multiscalemethods [4] and multiresolution approaches[5–8]. Methods described in [1] depend on con-trolled camera motion and do not work forarbitrary set of images. Probabilistic techniquesinvolve huge computation using floating pointarithmetic and thus requires a lot of time andmemory-space. Image gradient method with ma-jority filtering has the drawback that the defocusedzone of one image is enhanced at the expense offocused zone of others.

An image often contains physically relevantfeatures at many different scales or resolutions.Multiscale and multiresolution approaches pro-vide a means to exploit this fact. This is one of thereasons why these techniques have become sopopular. Multiscale methods involve processingand storing of scaled data at various levels whichare of the same size as that of the original images.This results in a huge amount of memory and timerequirement [4]. Multiresolution techniques ofimage fusion using pyramid or wavelet transformproduce good result in less computation time usingless memory. The basic idea in these techniques isto decompose the source images at first byapplying the pyramid or wavelet transform, thenthe fusion operation on the transformed images isperformed and finally the fused image is recon-structed by inverse transform. One major advan-

tage of multiresolution transform is that spatial aswell as frequency domain localization of an imageis obtained simultaneously. Another advantage isthat it can provide information on the sharpcontrast changes, and human visual system isespecially sensitive to these changes. Both pyramidand wavelet transforms are used as multiresolutionfilters. Wavelet transform can be considered as aspecial case of pyramid transform but it has morecomplete theoretical support [9]. Various methodsof image fusion using multiresolution techniqueshave been suggested before [5–8]. A survey onthese works may be found in [8]. Burt andLolezynski [5] suggested a method in which theimages are first decomposed into a gradientpyramid. Activity measure of each pixel iscomputed then by finding out the variance of a3� 3 or 5� 5 window centered around that pixel.Depending on this measure, either the larger valueor the average value is chosen. Finally thereconstruction is done. Li et al. [6] used similarmethod except the fact that wavelet transforms areused for decomposition and consistency verifica-tion is done along with area-based activitymeasure and maximum selection. Their methodreduces the artifacts such as blocking effects whichare common in image fusion using multiresolution.Methods described in [5,6] are complex and time-consuming. Moreover it was not mentionedwhether the method can be applied to more thantwo multifocus images. In the method suggestedby Yang et al. [7], an impulse function is defined atfirst to describe the quality of an object in amultifocus image. Then sharply focussed regionsare extracted by analyzing the wavelet coefficientsof two primary images and two blurred images. Tofuse two images, this method compares the waveletcoefficients of four images and thus involvesdouble computation. Moreover, all its computa-tions involve floating point arithmetic.Wavelet transform is a linear tool in its original

form [9]. But nonlinear extensions of discretewavelet transform are possible by various methodslike lifting scheme [10] or morphological operators

[11,12]. The problem with linear wavelets like Haarwavelet is that during signal decomposition oranalysis the range of the original data is notpreserved [12]. Secondly, linear wavelets act as

ARTICLE IN PRESS

I. De, B. Chanda / Signal Processing 86 (2006) 924–936926

low-pass filters and thus smooth-out the edges.This results in reduction in the contrast in fusedimages. The nonlinear wavelet introduced byHeijmans et al. [12] overcomes this drawback byusing morphological operators. But it involvesdivision operation and thus either requires floatingpoint arithmetic or introduces truncation error byusing integer arithmetic.

We introduce in this paper a nonlinear morpho-

logical wavelet transform which preserves the rangein the scaled images and involves integer arith-metic only. We use this transform to present afusion algorithm to fuse a set of two-dimensionalgray-scale multifocus images of some scene. Themethod is simple, computationally efficient andproduces good results. Integrated-chip implemen-tations of image processing algorithms are goingto become more common in near future. Ourmethod will be useful in this respect. The resultsobtained by it has been compared with thoseobtained by using Haar wavelet and the morpho-logical wavelet suggested by Heijmans and Gout-sias [12] using several sets of data. The paper isorganized as follows. Section 2 gives the basictheory (without proof) of multiresolution analysisusing wavelets and a brief discussion on morpho-logical operators. This section also introduces theproposed wavelet transform based on theseoperators. Section 3 describes the image-fusionalgorithm using the new morphological wavelet.Experimental results and discussion are given inSection 4 and the concluding remarks are pre-sented in Section 5.

2. Basic theory and a new morphological wavelet

A brief overview of multiresolution signaldecomposition theory using wavelets is given first,followed by the discussion on morphologicaloperators, and finally a new wavelet transformbased on morphological operators is presented.

2.1. Multiresolution analysis

The theory of multiresolution signal decomposi-tion scheme using wavelets can be applied to awide variety of signals. We are restricted here to

two-dimensional gray-scale image signals only. Atwo-dimensional gray-scale image signal X is afunction which map (a subset of) discrete two-dimensional space Z2 to a finite set of non-negative integers G ¼ fg1; g2; . . . ; gng called the setof gray values. Let us consider a set V 0 of suchimage signals. A multiresolution signal decom-position scheme on V0 uses two types of operators,namely, signal analysis and signal synthesis opera-tors. Signal analysis operators c"j : Vj ! Vjþ1,map the signal space Vj at level j, to a coarsersignal space V jþ1 and the detail analysis operatorso"j : Vj !W jþ1, map Vj to a coarser detail spaceW jþ1. All V 0js and W 0

js have the same structure asV0. The operators c

"

j and o"j are called the scaling

function and the wavelet function, respectively.Signal analysis operation proceeds by mapping asignal to a level higher in the pyramid structure,thereby reducing information. Details are stored ateach level to restore this information loss. So, forX j 2 Vj we have,

c"j ðXjÞ ¼ X jþ1; X jþ1 2 V jþ1, (1)

o"j ðXjÞ ¼ Y jþ1; Y jþ1 2W jþ1. (2)

Signal synthesis or reconstruction is done bysynthesis operator C#j : Vjþ1 �W jþ1! Vj , whichmap a signal to a level lower in the pyramid. Toensure loss-less or perfect reconstruction, thefollowing condition must be satisfied.

C#j ðc"

j ðXjÞ;o"j ðX

jÞÞ ¼ X j ; X j 2 Vj. (3)

There are two more conditions, namely,

c"j ðC#

j ðXjþ1;Y jþ1ÞÞ ¼ X jþ1, (4)

o"j ðC#

j ðXjþ1;Y jþ1ÞÞ ¼ Y jþ1, (5)

where X jþ1 2 V jþ1 and Y jþ1 2W jþ1. They ensurethat the decomposition is non-redundant in thesense that repeated applications of these schemesproduce the same result. A special case calleduncoupled wavelet decomposition occurs whenthere exists a binary operation _þ on V j andoperators c#j : Vjþ1! V j and o#j : W jþ1 ! V j

ARTICLE IN PRESS

I. De, B. Chanda / Signal Processing 86 (2006) 924–936 927

such that

C#j ðXjþ1;Y jþ1Þ ¼ c#j ðX

jþ1Þ _þo#j ðYjþ1Þ,

X jþ1 2 Vjþ1; Y jþ1 2W jþ1. ð6Þ

Then perfect reconstruction and non-redundancyconditions become

c#j c"

j ðXjÞ _þo#j o

"

j ðXjÞ ¼ X j ; X j 2 Vj, (7)

c"j ðc#

j ðXjþ1Þ _þo#j ðY

jþ1ÞÞ ¼ X jþ1,

X jþ1 2 Vjþ1; Y jþ1 2W jþ1, ð8Þ

o"j ðc#

j ðXjþ1Þ _þo#j ðY

jþ1ÞÞ ¼ Y jþ1,

X jþ1 2 Vjþ1; Y jþ1 2W jþ1. ð9Þ

If an one-dimensional wavelet decompositionscheme can be applied to two and higher dimen-sions, by applying it to other dimensions sequen-tially, then this decomposition is called separable.

Fig. 1. Wavelet transform on a 2� 2 block of X.

2.2. Morphological operators

A brief overview of the morphological operatorsis given now. Let us consider a signal X 2 V0 (seeSection 2.1). So X is a function from domain D toG where D is a subset of Z2 and G is the set ofgray-values. Let A � Z2 be a structuring element.Then the morphological operators, dilation dAðX Þ

and erosion �AðX Þ of X by A are defined as

dAðX Þðr; cÞ ¼ maxðr1;c1Þ2A; ðr�r1; c�c1Þ2D

X ðr� r1; c� c1Þ,

�AðX Þðr; cÞ ¼ minðr1;c1Þ2A; ðrþr1; cþc1Þ2D

X ðrþ r1; cþ c1Þ.

So dilation (erosion) simply replace the value ateach point of X by the maximum (minimum) valuein a neighborhood defined by the structuring

element A. Other morphological operators can beconstructed by combining d and �. For exampleopening aAðX Þ and closing fAðX Þ of X by A aredefined as

aAðX Þðr; cÞ ¼ dAð�AðX ÞÞðr; cÞ,

fAðX Þðr; cÞ ¼ �AðdAðX ÞÞðr; cÞ.

2.3. A new morphological wavelet

Heijmans and Goutsias introduced a morpholo-

gical variant of the linear Haar wavelet by usingthe morphological operation dilation (erosion) [12].It is an one-dimensional scheme and the multi-dimensional implementation can be obtained byapplying it to other dimensions sequentially.However, a non-separable two-dimensional ver-sion of the morphological Haar wavelet transformhas also been defined in [12], which will be used inour experiments. We, now propose a non-separable

two-dimensional uncoupled morphological wavelet

decomposition scheme, which will be used for ourimage-fusion algorithm. Unique analysis operatorsðc";o"Þ are used at all levels of the multiresolutionscheme. Similarly, unique synthesis operatorsðc#;o#Þ are used at all levels. These operatorsare explained for the lowermost levels 0 and 1.Let us consider the signal space V 0 of Section

2.1. It is our original signal space. Then V 1 andW 1 are the signal and detail spaces at level 1having the same structure as V 0. Consider animage signal X 2 V 0. Then X is a mapping of (asubset of) Z2 to the set of gray-values G and it canbe represented by an M �N matrix, whereM;N 2 Z. Let us assume that M and N both areeven. Then X can be divided into consecutive anddisjoint 2� 2 submatrices or blocks, which aretotal MN=4 in number. Four positions of such ablock B may be denoted by ðr; cÞ, ðr; cþ 1Þ, ðrþ1; cÞ and ðrþ 1; cþ 1Þ (see Fig. 1). Using quadraticdownsampling, the analysis operators c" : V0!

V1 and o" : V 0!W 1 are defined as

c"ðX ÞðBÞ ¼M ¼ maxfX ðr; cÞ;X ðr; cþ 1Þ,

X ðrþ 1; cÞ;X ðrþ 1; cþ 1Þg,

ð10Þ

o"ðX ÞðBÞ ¼ ðyv; yh; ydÞ, (11)

ARTICLE IN PRESS

T1

T1

T0

T2 T3

Wavelet

Transform

Tm

Case 1: Transform when T0 is maximum

Case 2: Transform when T2 is maximum

0T

2T 3T

Wavelet

Transform

m 1

0

T

T mT

T Tm

m 1T T

3mT T

3mT T

m 2T T

Fig. 2. Example for proposed wavelet transform.

I. De, B. Chanda / Signal Processing 86 (2006) 924–936928

where yv; yh; yd represent the vertical, horizontal

and diagonal detail signals, respectively, defined as

yv ¼M � X ðr; cþ 1Þ if M � X ðr; cþ 1Þ40;

X ðr; cþ 1Þ �M otherwise;

((12)

yh ¼M � X ðrþ 1; cÞ if M � X ðrþ 1; cÞ40;

X ðrþ 1; cÞ �M otherwise;

((13)

yd ¼M � X ðrþ 1; cþ 1Þ if M � X ðrþ 1; cþ 1Þ40;

X ðrþ 1; cþ 1Þ �M otherwise:

((14)

The second condition in the last three equations isrequired to maintain the information on positionof the maximum value M as evidenced in thesuccessive example. Scaled signal and detail valuesobtained above belong to X 1 and Y 1, respectively,and they can be stored conveniently in similarpositions of another matrix.

The original signal at level 0 is reconstructed bythe synthesis operation. Using quadratic down-sampling, synthesized signals bX are given bybX ðr; cÞ ¼ bX ðr; cþ 1Þ ¼ bX ðrþ 1; cÞ

¼ bX ðrþ 1; cþ 1Þ ¼M ð15Þ

and synthesized details bY are given bybY ðr; cÞ ¼ minðyv; yh; yd; 0Þ, (16)

bY ðr; cþ 1Þ ¼ minð�yv; 0Þ, (17)

bY ðrþ 1; cÞ ¼ minð�yh; 0Þ, (18)

bY ðrþ 1; cþ 1Þ ¼ minð�yd; 0Þ, (19)

where M ¼ X 1ðr; cÞ is the scaled signal at ðr; cÞ andyv; yh; yd are vertical, horizontal and diagonaldetails, respectively. This is an uncoupled decom-position scheme and the binary operation _þ is theordinary addition of numbers. Hence the recon-structed signal X 0 at any point ðu; vÞ 2 fðr; cÞ; ðr; cþ1Þ; ðrþ 1; cÞ; ðrþ 1; cþ 1Þg is given by

X 0ðu; vÞ ¼ bX ðu; vÞ þ bY ðu; vÞ. (20)

Example. Let us consider the 2� 2 block B of X

with X ðr; cÞ ¼ T0;X ðr; cþ 1Þ ¼ T1;X ðrþ 1; cÞ ¼T2 and X ðrþ 1; cþ 1Þ ¼ T3. Let Tm ¼ maxfT0;T1;T2;T3g. Then c"ðX ÞðBÞ ¼ Tm and the detailsare given by o"ðX ÞðBÞ ¼ ðTv;Th;TdÞ where

Tv ¼Tm � T1 if Tm � T140;

T0 � Tm otherwise;

(

Th ¼Tm � T2 if Tm � T240;

T0 � Tm otherwise;

(

Td ¼Tm � T3 if Tm � T340;

T0 � Tm otherwise:

(

Now Tm may occur at any of the four positions ofthe block 2� 2 submatrix. The situations of Tm

occurring at ðr; cÞ and ðrþ 1; cÞ are illustrated inthe Fig. 2. In the first case Tm occurs at positionðr; cÞ and all the detail values are positive. In thesecond case Tm occurs at position ðrþ 1; cÞ and theinformation is preserved by placing the negativevalue T0 � Tm as the horizontal detail.The analysis operator-pair ðc"j ;o

"

j Þ can be usedrecursively to decompose a signal upto a desiredlevel kX1. Similarly the synthesis operator-pairðc#j ;o

#

j Þ can be used recursively to reconstruct asignal from any level to the lowest level 0. It is easyto see that the analysis and synthesis operators

ARTICLE IN PRESS

(a) (b) (c)

Fig. 3. (a) Original signal X, (b) scaled signal X 1 and details

Y 1 ¼ fy1v; y1h; y

1dg at level 1, (c) scaled signal X 2 and details Y 2 ¼

fy2v; y2h; y

2dg at level 2.

I. De, B. Chanda / Signal Processing 86 (2006) 924–936 929

satisfy the perfect reconstruction and non-redun-dancy conditions 7–9 given in Section 2.1. Theoperators c" and o" involve simple arithmeticoperations and one interesting point to note is thatthe integer values are mapped to integer valuesonly. Another point to note is that, if all values ofX belong to the range ½0;R�, then analyzed signal-values will belong to the range ½0;R� and analyzeddetail-values will belong to the range ½�R;R�,irrespective of the number of times the operatorsare applied (see Fig. 3).

3. Image fusion

We now present the image fusion algorithmproposed by us using the morphological wavelettransform given in Section 2.3. Consider n two-dimensional multifocus images X 1;X 2; . . . ;X n.These images must be registered and of same size.The proposed analysis operators c" and o", areapplied on the n individual images k timesrecursively. If X i; i ¼ 1; 2; . . . ; k are M �N

images, the analysis operators can be applied atmost kmax times where kmax ¼ minðblog2 Mc;blog2 NcÞ. After completion of the analysis opera-tion, a set of n scaled image signals at the topmostlevel k are obtained. They are denoted byX k

i ; i ¼ 1; 2; . . . ; n. A set of detail signals Yji, i ¼

1; 2; . . . ; n are also obtained at each level j, j ¼ 1–k.As mentioned in the last section, if the range ofimage X i is ½0;R�, then the range of scaled imageX k

i is ½0;R� and the range of details Yji , j ¼

1; 2; . . . ; k are ½�R;R�. While comparing X ki ; i ¼

1; 2; . . . ; n positionwise, greater absolute valuecorresponds to a brighter pixel and while compar-ing Y

ji , j ¼ 1; 2; . . . ; k; i ¼ 1; 2; . . . ; n positionwise,

greater absolute value corresponds to sharpcontrast changes such as edge, line and regionboundaries. Based on this observation, scaledimages X k

i ; i ¼ 1; 2; . . . ; n are combined by com-paring the values at each position ðr; cÞ andchoosing the one with greatest absolute value.Similar operation is applied on correspondingdetails at each level. Thus a single fused image atlevel k and a detail at each level j, j ¼ 1; . . . ; k areobtained. Then the reconstruction phase begins.The image at level k � 1 is reconstructed byapplying the synthesis operators c# and o# (asproposed by us in the previous section) followedby addition. Synthesis operators are applied k

times recursively to obtain the image at level 0.The algorithm can be summarized as below.

3.1. The algorithm

1. Analysis step: Apply the analysis operatorsc" and o"; k times recursively, on image X i; i ¼1; . . . ; n and get Xi ¼ fX

ki ;Y

1i ;Y

2i ; . . . ;Y

ki g, where

X ki is the scaled image at level k and Y

ji, j ¼

1; . . . ; k are the details at levels 1; 2; . . . ; k, respec-tively.2. Fusion step: Compare fXi; i ¼ 1; 2; . . . ; ng

and combine them into X ¼ fX k;Y 1;Y 2; . . . ;Y kg,where X k and Y j are given by X kðr; cÞ ¼maxfjX k

1ðr; cÞj; jXk2ðr; cÞj; . . . ; jX

knðr; cÞjg and Y jðr; cÞ

¼ maxfjYj1ðr; cÞj; jY

j2ðr; cÞj; . . . ; jY

jnðr; cÞjg, respec-

tively.3. Synthesis step: Reconstruct the fused image

X j at level j, j ¼ k � 1; . . . ; 0, by applying thesynthesis operators c# and o#, respectively, onX jþ1 and Y jþ1 following by addition, i.e.

X jðr; cÞ ¼ c#ðX jþ1ðr; cÞÞ þ o#ðY jþ1ðr; cÞÞ.

3.2. Example

The algorithm is illustrated by using 2� 2sample data A and B taken from the multifocusimages X 1 and X 2, respectively.Let

A ¼a0 a1

a2 a3

" #and B ¼

b0 b1

b2 b3

" #,

ARTICLE IN PRESS

I. De, B. Chanda / Signal Processing 86 (2006) 924–936930

where ai and bi; i ¼ 0; 1; 2; 3 are non-negativeintegers. Applying the analysis operators c" ando" once, A becomes,

A1 ¼a10 a1

1

a12 a1

3

" #,

where

a10 ¼ amax ¼ maxfai; i ¼ 0; 1; 2; 3g and

a1i ¼

amax � ai if amax4ai;

�ðamax � a0Þ otherwise:

(Here a1

0 is the scaled signal-data and a1i ; i ¼ 1; 2; 3

are the detail-data at level 1.Similarly, after the analysis operation, B be-

comes,

B1 ¼b10 b1

1

b12 b1

3

" #A1 and B1 are fused in C1, by the fusion step,where

C1 ¼c10 c11

c12 c13

" #and

c1i ¼a1

i if ja1i jXjb

1i j;

b1i otherwise;

for i ¼ 0; 1; 2; 3:

8<:The fused data C at level 0 is obtained by applyingthe synthesis operators c# and o# followed byaddition. Therefore

C ¼c0 c1

c2 c3

" #,

where

c0 ¼ c10 þminð0; c11; c12; c

13Þ and

ci ¼c10 if c1i o0;

c10 � c1i otherwise;for i ¼ 1; 2; 3:

(We now, claim that, ci is always less or equal to R,where R is the greatest value of ai andbi; i ¼ 0; 1; 2; 3. This happens because ci is obtainedby subtracting a non-negative value fromc10 ¼ maxða1

0; b10Þ. However the lower bound of ci

may not remain within the lower bounds of A and

B. The method can be applied to the completeimages X 1 and X 2 by taking as many 2� 2samples as required.

4. Experimental results and discussion

The proposed method for image fusion has beenimplemented in C language on Unix environmentand has been tested on a number of multifocusimages. We have compared our method withsimilar methods using Haar wavelet and the two-dimensional morphological Haar wavelet intro-duced by Heijmans and Goutsias [12]. Theexperimental results are shown in Figs. 4–7. Ineach figure, the original multifocus images aregiven first, followed by the fused images obtainedby the proposed method and two other methodsmentioned above. The fusion is done by decom-posing the constituent images up to the third level,in all the cases. Time taken by these algorithms aremore or less same and not significant.

4.1. Performance analysis

Careful manual inspection of Figs. 4–7 revealsthat the results obtained by the proposed waveletare better than that of Haar wavelet and arecomparable to that of Heijmans and Goutsias’method [12]. However this is a subjective measureof quality and may not be universally acceptable.Hence a quantitative measure is also given.Gradient or derivative operators are useful tools

to measure the variation of intensity with respectto immediate neighboring points or pixels of animage [13]. It is observed that a pixel possesseshigh gradient value when it is sharply focused. Anobjective criterion based on this knowledge issuggested to measure the quality of the results. Thegradient operator suggested by Roberts [14] is usedhere. Thus magnitude of gradient Gðr; cÞ at a pointðr; cÞ of image X is obtained by

Gðr; cÞ ¼ 12fjX ðr; cÞ � X ðrþ 1; cþ 1Þj

þ jX ðr; cþ 1Þ � X ðrþ 1; cÞjg.

It may be noted that Gðr; cÞ is obtained from X

through a non-linear transformation. So, we

ARTICLE IN PRESS

(a) (b)

(c) (d)

(e) (f)

Fig. 4. Original multifocus images and the fused images by different algorithms: (a) Near focused image; (b) middle focused image; (c)

far focused image; (d) fused image with proposed algorithm; (e) fused image by Haar wavelet; (f) fused image by Heijmans and

Goutsias’ method.

I. De, B. Chanda / Signal Processing 86 (2006) 924–936 931

ARTICLE IN PRESS

(a) (b)

(c) (d)

(e) (f)

Fig. 5. Original multifocus images and the fused images by different algorithms: (a) Left focused image; (b) center focused image; (c)

right focused image; (d) fused image with proposed algorithm; (e) fused image by Haar wavelet; (f) fused image by Heijmans and

Goutsias’ method.

I. De, B. Chanda / Signal Processing 86 (2006) 924–936932

cannot restore X ðr; cÞ from Gðr; cÞ. However,Gðr; cÞ for the image with all parts properlyfocused may be obtained from various partially

focused images as follows. For a set of n multi-focus images X i; i ¼ 1; . . . ; n, the gradient imagesGi; i ¼ 1; . . . ; n are obtained first. Then, Gi; i ¼

ARTICLE IN PRESS

(a) Focus on background (b) Focus on foreground

(c) Fused image with proposed algorithm (d) Fused image by Haar wavelet

(e) Fused image by Heijmans and Goutsias method

Fig. 6. Original multifocus images and the fused images by different algorithms: (a) Focus on background; (b) focus on foreground; (c)

fused image with proposed algorithm; (d) fused image by Haar wavelet; (e) fused image by Heijmans and Goutsias’ method.

I. De, B. Chanda / Signal Processing 86 (2006) 924–936 933

1; . . . ; n are combined into G by taking themaximum gradient value at each position, i.e.

Gðr; cÞ ¼ maxfG1ðr; cÞ;G2ðr; cÞ; . . . ;Gnðr; cÞg

for all ðr; cÞ.

Thus only the sharply focused regions from theconstituent images have their contribution inthe maximum gradient image G. Let G0 denotethe gradient image obtained from the recon-structed image X 0. It is referred to as the gradient

ARTICLE IN PRESS

Table 1

Similarity between maximum gradient and fused gradient

images

Figure Proposed

Algo

Haar

wavelet

Heijmans and

Goutsius’ method

Fig. 4 0:861 0:824 0:853Fig. 5 0:823 0:835 0:823Fig. 6 0:929 0:926 0:930Fig. 7 0:840 0:744 0:835

(a) (b)

(c) (d)

(e)

Fig. 7. Original multifocus images and the fused images by different algorithms: (a) Focus on background; (b) focus on foreground; (c)

fused image with proposed algorithm; (d) fused image by Haar wavelet; (e) fused image by Heijmans and Goutsias’ method.

I. De, B. Chanda / Signal Processing 86 (2006) 924–936934

of fused image. Then, more similar G and G0 are,better is the fusion algorithm. The similarity S

between two images is calculated as

SðG;G0Þ ¼ 1�

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPðGðr; cÞ � G0ðr; cÞÞ2

qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPðGðr; cÞÞ2

qþ

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPðG0ðr; cÞÞ2

q .

Hence, for an ideal fused image S approaches thevalue 1. Similarity between maximum gradient andfused gradient images are listed in Table 1. Thistable also conforms with the quality measuredthrough manual inspection.

The results obtained by our method are quitegood besides the fact that artifact such as block-

ing effects are noticed in the fused images insome cases (see Fig. 4). But this is a commonphenomena in pixel-based image fusion using

ARTICLE IN PRESS

I. De, B. Chanda / Signal Processing 86 (2006) 924–936 935

multiresolution approach and happens due to thefact that error introduced at the topmost level isamplified during reconstruction [6]. In our case,these effects are obvious at the regions where thedata is out of focus in all the source images. Theyare present in the fused images obtained by theother two methods as well.

Apart from the quality of the results, theproposed algorithm has some computational ad-vantages as well. Unlike two other waveletsexperimented with, our method ensures that integerpixel values are mapped to integer values onlyduring both analysis and synthesis. This is an usefulproperty for lossless data compression [15]. Sec-ondly, irrespective of the number of times theanalysis operators are applied, the range of thevalues in the scaled images will be same as that ofthe original multifocus images, say ½0;R�, and therange of the detail values will be ½�R;R�. Hencememory-space required during decomposition isfixed. Thirdly, simple arithmetic operations likeaddition, subtraction and comparison are the onlyoperations used in the method. Other two methodsinvolve division operation and thus they eitherrequires floating point arithmetic or introducestruncation error by using integer arithmetic.Fourthly, due to the non-linear nature of theproposed method, important geometric information(e.g. edges) is well-preserved at lower resolutions.Finally, the method is very fast due to its simplicity.For a set of n, M �N images, it takes only Oðn�

M �NÞ computational time. The simplicity of themethod and the use of integer arithmetic makes itsuitable for chip-level implementation.

Besides this, the non-linear wavelet proposed byus possesses the following invariance properties.Both analysis and synthesis operators are transla-tion invariant in the spatial domain. In thefrequency domain, they are grey shift (multiplica-tion) invariant. That means adding (multiplying) acertain value to all pixel values in the original datawill result in adding (multiplying) that value to thescaled signal data during analysis [12]. Also, detailswill not change in case of addition and will getmultiplied by that value in case of multiplication.The wavelets possessing these invariance properties,offer better option for image fusion than thosewhich do not possess them [8].

5. Conclusion

In this paper, we have presented a non-linearwavelet constructed by morphological operatorsand also presented a multifocus image fusionalgorithm based on that wavelet. The results arequite impressive considering the fact that thecomputational cost is very low. The use of simplearithmetic operations makes the method suitablefor hardware implementation. However the meth-od suffers from the problem that sometimesartifacts are formed in the fused image. But thisis a common problem for all other methodsexperimented with in this paper. Our method isdefinitely better than the Haar wavelet method andis at par with Heijmans and Goutsias waveletmethod in this respect. These effects are obvious atthe regions where the data is out of focus in all thesource images. So if the intersection of thedefocused regions from all the source images arereduced, then this effect will be minimized.

Acknowledgements

We gratefully acknowledge the sources of dataused in our experiments. The images of indoorscene [4] and garden scene [7] are obtained fromthe websites

http://www.geocities.com/~j1hagan/lessons/

depthofield.htm andhttp://www.photozone.de/4Technique/compose/

dof.htm.The original multifocus images in both sets were

unregistered and the registration process was doneby Mr. Buddhajyoti Chattopadhyay. The regis-tered multifocus images in Figs. 5 and 6 wereobtained from the site

http://www.ece.lehigh.edu/SPCRL/IF/multifocus.

htm.

References

[1] W. Seales, S. Dutta, Everywhere-in-focus image fusion

using controllable cameras, Proceedings of SPIE 2905,

1996, pp. 227–234.

ARTICLE IN PRESS

I. De, B. Chanda / Signal Processing 86 (2006) 924–936936

[2] I. Bloch, Information combination operators for data

fusion: a review with classification, IEEE Trans. SMC:

Part A 26 (1996) 52–67.

[3] H.A. Eltoukhy, S. Kavusi, A computationally efficient

algorithm for multi-focus image reconstruction, Proceed-

ings of SPIE Electronic Imaging, June 2003.

[4] S. Mukhopadhyay, B. Chanda, Fusion of 2d gray scale

images using multiscale morphology, Pattern Recognition

34 (2001) 1939–1949.

[5] P.J. Burt, R.J. Lolezynski, Enhanced image capture

through fusion, in: Proceedings of the Fourth Interna-

tional Conference on Computer Vision, Berlin, Germany,

1993, pp. 173–182.

[6] H. Li, B. Manjunath, S. Mitra, Multisensor image fusion

using the wavelet transform, Graph. Models Image

Process. 57 (3) (1995) 235–245.

[7] X. Yang, W. Yang, J. Pei, Different focus points images

fusion based on wavelet decomposition, Proceedings of

Third International Conference on Information Fusion,

vol. 1, 2000, pp. 3–8.

[8] Z. Zhang, R.S. Blum, Image fusion for a digital camera

application, Conference Record of the Thirty-Second

Asilomar Conference on Signals, Systems and Computers,

vol. 1, 1998, pp. 603–607.

[9] S. Mallat, A Wavelet Tour of Signal Processing, Academic

Press, London, 1998.

[10] W. Sweldens, The lifting scheme: a new philosophy in

biorthogonal wavelet constructions, in: A. Laine, M.

Unser, (Eds.), Wavelet Applications in Signal and Image

Processing, vol. III, Proceedings of SPIE, vol. 2569, 1995,

pp. 68–79.

[11] J. Goutsias, H.J. Heijmans, Nonlinear multiresolution

signal decomposition schemes, Part 1: morphological

pyramids, IEEE Trans. Image Process. 9 (November

2000) 1862–1876.

[12] H.J. Heijmans, J. Goutsias, Multiresolution signal decom-

position schemes, Part 2: morphological wavelets, IEEE

Trans. Image Process. 9 (November 2000) 1897–1913.

[13] B. Chanda, D.D. Majumdar, Digital Image Processing

and Analysis, Prentice-Hall of India, New Delhi-110001,

2000.

[14] L.G. Roberts, Machine perception of three dimensional

solids, in: J.T. Tippet (Ed.), Optical and Electro-optical

Information Processing, MIT Press, Cambridge, MA,

1965.

[15] A.R. Calderbank, I. Daubechies, W. Sweldens, B.L. Yeo,

Wavelet transforms that map integers to integers, Appl.

Comput. Harmonic Anal. 5 (1998) 332–369.