Reconstruction of an Object from its Fourier Modulus: Development of the Combination Algorithm Composed of the Hybrid Input–Output Algorithm and Its Converging Part

Reconstruction of an object from its Fourier modulus:development of the combination algorithm composed ofthe hybrid input–output algorithm and its converging part

Hiroaki Takajo, Tohru Takahashi, Katsuhiko Itoh, and Toshiro Fujisaki

The hybrid input–output algorithm �HIO� used for phase retrieval is in many cases combined with theerror-reduction algorithm �ER� to attempt to stabilize the HIO. However, in our previous paper �J. Opt.Soc. Am. A 16, 2163 �1999��, it was demonstrated that this combination makes it more likely that theresultant algorithm will fall into a periodic state before reaching a solution because the values of the inputobject outside the support, which is imposed as the object-domain constraint, are set to be zero in theintervals in which the ER is implemented. This paper deals with this problem inherent in the combi-nation algorithm. The converging part of the HIO �CPHIO�, which is an algorithm we previouslydeveloped �J. Opt. Soc. Am. A 15, 2849 �1998��, can be thought of as an extension of the ER for the casein which the input object can have nonzero values outside the support. Keeping this in mind, thealgorithm is then constructed by combining the HIO with the CPHIO instead of with the ER. Thecomputer simulation results that demonstrate the effectiveness of the proposed algorithm are given.© 2002 Optical Society of America

OCIS codes: 100.5070, 100.3010, 100.2000, 070.2590.

1. Introduction

In many areas of science and engineering, such aselectron microscopy, astronomy, crystallography, andwave-front sensing, the phase-retrieval problem isencountered. This problem can be stated as follows:Given the Fourier modulus of an object and theknowledge of its support and�or nonnegativity, re-construct the object or, equivalently, reconstruct theFourier phase of the object.

The iterative Fourier-transform algorithm, devel-oped by Fienup, is the most effective technique forsolving the phase-retrieval problem.1,2 Various ver-sions of the iterative Fourier-transform algorithm ex-ist. The most basic version is a descendant of theGerchberg–Saxton algorithm,1,3 which is called theerror-reduction algorithm �ER�. Even though thehybrid input–output algorithm �HIO� has been rec-

H. Takajo, K. Itoh, and T. Fujisaki are with the Department ofElectronic Engineering, Kyushu Institute of Technology, 1-1Sensui-cho, Tobata, Kitakyushu, 804-8550, Japan. T. Takahashiis with the Department of Electrical Engineering, Oita NationalCollege of Technology, Maki, Oita, 870-01, Japan. E-mail addressfor H. Takajo is [email protected].

Received 6 February 2002; revised manuscript received 3 June2002.

0003-6935�02�296143-11$15.00�0© 2002 Optical Society of America

ognized empirically to be one of the most successful ofthese versions,4,5 the behavior of this algorithm is notyet sufficiently understood. This is because thestrong nonlinearity of the operation in the HIO doesnot allow for theoretical analysis of the behavior ofthe HIO itself and is a major disruption to the studyof the stagnation problem6–8 present in the iterativeFourier-transform algorithm. To deal with this sit-uation, in our previous paper,9 a special case wasstudied. That is, theoretical arguments were devel-oped, although on a rather intuitive level, on theconvergence property of the HIO with an infinitesi-mally small feedback parameter, which was calledthe infinitesimal HIO algorithm and was abbreviatedas IHIO. As a result, it was demonstrated that �1�,in the HIO and thus in the IHIO, the state can occurin which, although the input object changes at eachiteration, the output object does not �this state andthe output object in this state were called the output-stagnating state and the output-stagnation object,respectively� and that �2� the IHIO has two abilities:the ability to locate an output-stagnation object andthe ability to emerge from an output-stagnation ob-ject if it is not a solution. A solution is the object thatsatisfies both the Fourier-domain and the object-domain constraints and, thus, the object that satisfiesthe Fourier-domain constraint and for which theobject-domain error takes the value zero. Because

10 October 2002 � Vol. 41, No. 29 � APPLIED OPTICS 6143

an output-stagnation object is the object that satisfiesthe Fourier-domain constraint and for which theobject-domain error has a locally minimum value,9 asolution is contained in the set of the output-stagnation objects. Therefore the two abilities men-tioned in the above �2� are extremely important infinding a solution.

Further study was done, in Ref. 10, on the behaviorof the HIO, and it was demonstrated that the HIOhave an additional aspect that is also important infinding a solution. That is, denoting the term “thesupport imposed as the object-domain constraint” assimply “the support,”11 the following can be said: inthe input object outside the support, the output-object values outside the support that have been pro-duced so far accumulate, and so it is conjectured thatthe probability is extremely low that the HIO will fallinto a periodic state before reaching a solution. Inmany cases, the HIO has been combined with the ERin order to attempt to stabilize the HIO.1,12 How-ever, in Ref. 10, it was also demonstrated that thiscombination algorithm �HIO�ER� has a serious prob-lem. That is, in the HIO�ER, because the values ofthe input object outside the support are set to be zeroin the intervals in which the ER is implemented, theHIO�ER is more likely to fall into a periodic statebefore reaching a solution than the HIO alone.

The purpose of this paper is to deal with this prob-lem inherent in the HIO�ER. Concretely, we wishto stabilize the HIO without losing the abilities andthe aspect of the HIO that are important for findinga solution. In the investigation of the behavior ofthe IHIO, we demonstrated that the HIO can be sep-arated into two parts:9,13 one part generates the al-gorithm that is called the converging part of the HIOand is abbreviated as CPHIO and the other partforms the emerging term. Furthermore, it was clar-ified that the CPHIO can converge to an output-stagnation object of the HIO, even if the value of thefeedback parameter is not infinitesimally small, ifthat value is not too large, whereas the emergingterm causes the HIO to emerge from an output-stagnation object other than a solution.

To deal with the problem inherent in the HIO�ER,we note that because, as was described in the above,the CPHIO can converge to one of the output-stagnation objects of the HIO unless the value of thefeedback parameter is too large and because, as wasproven in Appendix B in Ref. 9, the set of the conver-gence output-objects of the ER is included in the setof the output-stagnation objects of the HIO, theCPHIO can be thought of as an extension of the ERfor the case in which the input object can have non-zero values outside the support, even though theFourier-domain and the object-domain errors do notalways decrease monotonically. Therefore, in thispaper, we propose to combine the HIO with theCPHIO instead of with the ER. The effectiveness ofthis combination algorithm, which we denote hereaf-ter as the HIO�CPHIO, is demonstrated through theresults of computer simulations.

This paper is organized as follows: Section 2 is a

preliminary section in which we briefly review theER. Section 3 consists of three subsections, Subsec-tion 3.A, 3.B, and 3.C. First, in Subsection 3.A, weonce again take a look at how the CPHIO and theemerging term can be derived from the HIO as indi-cated in Refs. 9 and 13, and explain the convergenceproperty of the CPHIO. In this subsection, we studyalso how the HIO can emerge from an output-stagnation object other than a solution owing to theexistence of the emerging term. Next, in Subsection3.B, we look again at the problem inherent in theHIO�ER and, finally, in Subsection 3.C, we constructthe proposed algorithm, the HIO�CPHIO. The com-puter simulation results, which demonstrate the ef-fectiveness of the HIO�CPHIO, are given in Section4, and in Section 5 the conclusion is presented.

2. Brief Review of the Error-Reduction Algorithm

In preparation for subsequent discussions, in thissection we give a brief review of the ER.

Let us assume that the object to be reconstructed,or the original object, f �x, y�, is of real and nonnega-tive value and is sampled on an M � N array. Thenf �x, y� and its discrete Fourier transform �DFT�, F�u,v�, are related to each other through the following:

F�u, v� � �F�u, v��exp�i��u, v��

� �x�M�21

M�2

�y�N�21

N�2

f � x, y�

� exp�i2��ux�M � vy�N��, (1)

f � x, y� � �MN�1 �u�M�21

M�2

�v�N�21

N�2

� F�u, v�exp�i2��ux�M � vy�N��, (2)

where M and N are even, and the variables x and urange from M�2 1 to M�2, and the variables y andv range from N�2 1 to N�2. To avoid aliasing inthe computation of �F�u, v��2, f �x, y� is assumed tohave nonzero values only in the region of the sizeM�2 � N�2.

The mth iteration of the iterative Fourier-transform algorithm consists of the following foursteps: �1� An input object gm�x, y� is Fourier trans-formed, yielding Gm�u, v� � �Gm�u, v��exp�i�m�u, v��;�2� a new Fourier-domain function is formed by use ofthe known Fourier modulus �F�u, v�� with the com-puted phase: G m�u, v� � �F�u, v��exp�i�m�u, v��; �3�G m�u, v� is inverse Fourier transformed to yield g m�x,y�, which is called the output object; �4� the object-domain constraint is imposed to generate the �m 1�th input object gm1�x, y�.

In the case of the ER, the �m 1�th input object isformed by

gm1� x, y� � �g m� x, y� � x, y� � D0 � x, y��D , (3)

where D is the set of points �x, y� at which g m�x, y�satisfies the object-domain constraints.

6144 APPLIED OPTICS � Vol. 41, No. 29 � 10 October 2002

For monitoring the progress of the algorithm, theFourier-domain and�or the object-domain errors,which are defined as

�Fm2 � �MN�1 �

u�

v��Gm�u, v�� F�u, v��2, (4)

�om2 � �

� x,y��D

� g m� x, y��2, (5)

respectively, are utilized. By using Parseval’s theo-rem, Eq. �4� can be rewritten as

�Fm2 � �

x�

y� gm� x, y� � g m� x, y��2. (6)

Now, let us think of �Fm2 as a function of the MN

parameters gm�x, y� and denote them as �F2 and g�x,

y� �with the subscript m deleted�, respectively. Thatis, �F

2 is given by

�F2 � �

x�

y� g� x, y� � g � x, y��2. (7)

�Of course, g �x, y� is the output object when g�x, y� isapplied to the iterative Fourier-transform algorithmas an input object.� Then, according to Fienup,1 thederivatives of �F

2 with respect to the MN parametersg�x, y� are provided by

��F2�

�g� x, y�� 2� g� x, y� � g � x, y��. (8)

Therefore, because from Eq. �8�

��F2�

�g� x, y��

g�gm

� 2� gm� x, y� � g m� x, y�� (9)

is obtained, it is obvious that Eq. �3� can be rewrittenas

gm1� x, y�

� �gm� x, y� �12

��F2�

�g�x, y��

g�gm

�x, y� � D

0 �x, y��D.

(10)

The necessary and sufficient condition for the itera-tive Fourier-transform algorithm to exist in the con-verging state is

gm1� x, y� � gm� x, y�. (11)

Consequently, from Eqs. �10� and �11�, the ER con-verges to the state in which

��F2�

�g� x, y��

g�gm

� 0 � x, y� � D (12)

is satisfied or, equivalently, to the state in which

gm� x, y� � g m� x, y� � x, y� � D (13)

is satisfied.In general, a real-valued object pictured on an M �

N array is represented as a point in MN-dimensionalspace. The set of objects that satisfy the object-domain constraints forms a subspace of the MN-dimensional space, which we call the object-domain-constraint space �ODC space�. For example, whenonly the support constraint is imposed as the object-domain constraint, the ODC space is the Ms-dimensional space embedded in the MN-dimensionalspace, where Ms is the number of sampling pointsthat exist in the support. If, in addition to the sup-port constraint, nonnegativity is imposed, the ODCspace forms the subspace of that Ms-dimensionalspace, in which no point can have coordinates of neg-ative values. From Eqs. �10� and �13� it can be easilyunderstood that the input object of the ER moves inthe ODC space in the direction in which the Fourier-domain error �F

2 decreases and converges to the ob-ject that satisfies Eq. �13�. The necessary andsufficient condition for the input object of the ER to bea solution is

gm� x, y� � g m� x, y�. (14)

Therefore, the existence of the objects that satisfy Eq.�13� but not Eq. �14� is the major cause of the stag-nation of the ER.8 At the points corresponding tosuch objects, �F

2 has locally minimum values.Also, Fienup proved that the ER has the following

error-reduction property:1

�Fm2 � �om

2 � �F,m12 � �o,m1

2. (15)

This relation tells us that in the ER not only theFourier-domain error but also the object-domain er-ror decrease monotonically and, therefore, the ERconverges to the state in which not only the Fourier-domain error but also the object-domain error takethe local minimum values, although the term thatmakes the object-domain error decrease is not in-cluded in Eq. �10�. This stems from the fact that theinput object of the ER is confined in the ODC space.

Here a note on the method of projection onto convexsets is appropriate. The ER is the same as themethod of projection onto convex sets for the phase-retrieval problem, and the error-reduction property,Eq. �15�, is equivalent to the property that Levi andStark called set-distance reduction in Refs. 14 and 15.Because the set of the objects that satisfy the Fourier-domain constraint is nonconvex, the ER does not en-joy the strong convergence properties of projectiononto the convex set. So, the ER is usually much lesseffective than the HIO.1

Finally, we would like to comment on the unique-ness problem. This problem refers to the question ofwhether f �x, y� can be defined uniquely by its Fouriertransform. There are the omnipresent ambiguitiesthat f �x, y�, any translation of the object f �x x0, y y0�, and the twin object f �x x0, y y0� all havethe same Fourier modulus. These ambiguitieschange only the object’s position or orientation, notits appearance. If they are the only ambiguities,then we refer to the object as being unique. Bruckand Sodin demonstrated that the presence of


ambiguities—of course, ambiguities other than theomnipresent ones—is equivalent to the factorabilityof the z transform of f �x, y�.16 More detailed anal-ysis on the uniqueness problem was given byHayes.17 Because the polynomials of two �or more�complex variables are rarely factorable, it is widelybelieved that in the majority of two-dimensionalcases the solution is unique.

3. Construction of the Proposed Algorithm

A. Converging Part of the Hybrid Input-Output Algorithmand the Emerging Term

In the investigation of the behavior of the IHIO, wedemonstrated that the HIO can be separated into twoparts:9,13 the CPHIO and the emerging term. Toconstruct the HIO�CPHIO, in this subsection weshow again how these parts are derived from theHIO, and explain the convergence property of theCPHIO. Moreover, we describe how the HIOemerges from an output-stagnation object other thana solution owing to the existence of the emergingterm.

In the case of the HIO, the rule for producing the�m 1�th input object is replaced by

gm1� x, y� � �g m� x, y� � x, y� � Dgm� x, y� � �g m� x, y� � x, y��D ,

(16)

where � is a constant feedback parameter. As wasshown in Refs. 9 and 13, when we take the DFT ofboth sides of Eq. �16�, Eq. �16� can be rewritten as

Gm1�u, v� � Gm�u, v� � DFT��12

��F2�

�g� x, y��g�gm

�D�� i�

MN4�F�u, v��2 � ��o

2�

��u, v��G �G m

�� G m�u, v�

� �MN

4�F�u, v�� o2�

��G �u, v��G �G m

�� G m�u, v�, (17)

where ��1�2��F2��g�x, y��g�gm

�D, which appears inthe second term on the right-hand side of Eq. �17�, isdefined by

�12

��F2�


�D

� �12

��F2�

�g� x, y��

g�gm

� x, y� � D

0 � x, y��D

(18)

and is called the D part of �1�2��F2��g�x, y��g�gm

.The CPHIO is defined by the equation

Gm1�u, v� � Gm�u, v� � DFT��12

��F2�


�D�� i�

MN4�F�u, v��2 � ��o

2�

��u, v��G �G m

�G m�u, v�,

(19)which is obtained from Eq. �17� by the omission of thefinal term on its right-hand side, and the emergingterm is specified by this omitted term. Further-more, in Ref. 9, it was clarified that Eq. �19� can bewritten as

Gm1�u, v� � GmND�u, v� � G m

D�u, v�

� i�1

�F�u, v��2Im�G m�u, v�

� �G mND�u, v��*�G m�u, v�, (20)

where GmND�u, v� represents the DFT of gm

ND�x, y�,which is defined by

gmND� x, y� � �0 � x, y� � D

gm� x, y� � x, y��D , (21)

and is called the ND part of gm�x, y�. G mD�u, v� and

G mND�u, v� are the DFTs of the D and the ND parts,

respectively, of g m�x, y�, and the asterisk denotescomplex conjugation. Note that Eq. �20� allows us tocalculate the �m 1�th input object, gm1�x, y�, fromgm�x, y� and g m�x, y�, which are the quantities pro-duced at the mth iteration.

From Eqs. �11� and �19�, it is seen that the CPHIOis fixed at the output-stagnating state of the HIO thatis specified by

��F2�

�g� x, y��

g�gm

� 2� gm� x, y� � g m� x, y��

� 0, � x, y� � D, (22)

��o2�

��u, v��

G �G m

� 4

MNIm�G m�u, v��G m

ND�u, v��*�

� 0. (23)

As was demonstrated in Ref. 9 and as is discussedbelow, if the HIO has fallen into this state, the situ-ation occurs in which, although the input objectchanges at each iteration, the output object does not.The output-stagnation object is the output object inthis state. From Eqs. �22� and �23�, it is apparentthat, in the output-stagnating state of the HIO, boththe Fourier-domain and the object-domain errorstake the local minimum values.

Look at Eq. �19� again, which defines the CPHIO.The second term on the right-hand side of this equa-tion moves the input object of the CPHIO in the di-rection in which the Fourier-domain error decreases,whereas the third term moves it in the direction in


which the object-domain error decreases. As is eas-ily recognized from Eqs. �19� and �20�, the input objectof the CPHIO is not confined in the ODC space unlikethe input object of the ER. Therefore the movementof the input object of the CPHIO in the direction inwhich the Fourier-domain error decreases does notalways result in the decrease in the object-domainerror. That is, a conflict can occur between the ef-fects of the second and the third terms on the right-hand side of Eq. �19�. However, as is suggested byEq. �19� and as was argued in Ref. 9, the CPHIO canconverge to the output-stagnating state of theHIO—in other words, the output object of the CPHIOcan converge to an output-stagnation object of theHIO—unless the value of � is so large that the con-flict between the effects of the second and the thirdterms on the right-hand side of Eq. �19� becomessignificant.

Now, let us discuss the function of the emergingterm. In Refs. 9 and 13, it was demonstrated thatwhen the HIO has arrived at an output-stagnationobject, say, g s�x, y�, at the mth iteration, the emergingterm reduces to �G s

NS�u, v� and that, if the output-stagnating state continues until the �m k�th itera-tion,

Gmk�u, v� � Gm�u, v� � k�G sND�u, v� (24)

holds, where G sNS�u, v� represents the DFT of the ND

part of g s�x, y�, which we denote as g sND�x, y�. Also,

it was clarified that there exists at least one pair offrequencies, say, �u0, v0� and �u0, v0�, at which thephase of G s

ND�u, v� is equal to the phase of G s�u, v�and thus to the phase of Gm�u, v�. �Note that theFourier phase of a real-valued object is an odd func-tion of the frequency �u, v�.� From these facts andfrom the fact that there always exists an integer k0that satisfies

�Gm�u0, v0�� k0��G sND�u0, v0�� (25)

and thus satisfies

�Gm�u0, v0�� k0��G sND�u0, v0��, (26)

it is easily recognized that owing to the existence ofthe emerging term the HIO can always emerge froman output-stagnation object if the output-stagnationobject is not a solution even if the HIO has beentemporarily trapped by it. More concretely, in theoutput-stagnating state, although the output objectof the HIO does not change at all, the input objectchanges by �g s

ND�x, y� at each iteration and, as aresult, a change by � cannot fail to occur in the Fou-rier phase of the input object at some pair of frequen-cies if the iteration is continued.

Here, it is appropriate to point out the following:In the above we showed that, owing to the existenceof the emerging term, the HIO can always emergefrom an output-stagnation object other than a solu-tion. However, note that, as was discussed in theconclusion section in Ref. 9, if the HIO—more pre-cisely, the input object of the HIO—has not gonebeyond the boundary of the territory18 of an output-

stagnation object when the HIO has emerged fromthat output-stagnation object, the probability is highthat the HIO will turn around and go back to thatoutput-stagnation object. That is, the emergence ofthe HIO from an output-stagnation object does notnecessarily mean the emergence of the HIO from itsterritory. If the HIO cannot emerge from the terri-tory of an output-stagnation object other than a so-lution, it is, of course, impossible for the HIO to reacha solution. In the next subsection, keeping this factin mind, we will study the problem inherent in theHIO�ER.

B. Problem Inherent in the Combination AlgorithmComposed of the Hybrid Input–Output Algorithm and theError-Reduction Algorithm

As was proven in Appendix B in Ref. 9, the set of theconvergence output objects of the ER is included inthe set of the output-stagnation objects of the HIO.This means that, if an object is a convergence outputobject of the ER, it is also an output-stagnation objectof the HIO. Although the manner whereby the HIOemerges from an output-stagnation object other thana solution was studied in Subsection 3.A, in this sub-section we will reexamine it to understand clearly thedifference between the performance of the HIO andthat of the HIO�ER, assuming that the output-stagnation object in which we are interested is theconvergence output object of the ER and paying at-tention to the aspect of the HIO that was referred toin Section 1 and derived again in Appendix A.

Let us denote a convergence output object of the ERother than a solution as g s,ER�x, y� and its D and NDparts as g s,ER

D�x, y� and g s,ERND�x, y�, respectively.

Now recall that, as was studied in Subsection 3.A, assoon as the HIO arrives at g s,ER�x, y�, it falls into theoutput-stagnating state. In this state, although theoutput object of the HIO stays at g s,ER�x, y�, the inputobject—more precisely, the ND part of the inputobject—changes by �g s,ER

ND�x, y� at each iterationand, as a result, the HIO emerges from g s,ER�x, y� atthe time when a change by � has occurred in theFourier phase of the input object at some pair offrequencies, say �u0, v0� and �u0, v0� �in otherwords, at the time when the input object has gonebeyond the objects whose DFT, G�u, v�, take the valuezero at �u0, v0� and �u0, v0��. Also, note that, inthis state, as is apparent from Eq. �16�, the D part ofthe input object of the HIO is fixed at the pointg s,ER

D�x, y�, and thus the input object moves in thespace that passes through the point g s,ER

D�x, y� and isparallel to the complement of the ODC space withrespect to the MN-dimensional space. In Fig. 1 weshow schematically how the HIO behaves in thisoutput-stagnating state and how it emerges fromthat state. In Fig. 1 the point g s,ER

D�x, y� is repre-sented as point P, and the space which passesthrough this point and is parallel to the complementof the ODC space is assumed to be of two dimensionsand to coincide with the plane of the paper. There-fore, it can easily be recognized that, in this output-stagnating state, the input object of the HIO moves


along a path that lies in the plane of the paper and isparallel to the vector gs,ER

ND�x, y�, which originatesfrom point P and terminates at the point g s,ER�x, y�,represented as point Q in Fig. 1. Let us call thispath the emerging path. At the time when the inputobject has gone beyond the objects with �G�u0, v0�� G�u0, v0�� 0, the HIO—more precisely, the out-put object of the HIO—emerges from point Q, moreconcretely, moves to some point that is different frompoint Q and that exists in the space formed by the setof objects satisfying the Fourier-domain constraint�see also Fig. 2 in Ref. 9.� However, as was pointedout in the last paragraph in Subsection 3.A, if theinput object has not gone beyond the boundary of theterritory of g s,ER�x, y� at that time, the probability ishigh that the output object of the HIO will turnaround and go back to point Q. However, because ofthe aspect of the HIO that was studied in Appendix A,the probability is conjectured to be extremely low thatthe emerging path along which the input object willmove after the output object has gone back to point Qwill coincide with the emerging path along which theinput object moved previously. Consequently, evenwhen the HIO turns around and goes back to g s,ER�x,y� many times, one should not too readily concludethat the HIO cannot ever emerge from the territory ofg s,ER�x, y�.

Now, as is seen from Eq. �3�, in the HIO�ER, thevalues of the ND part of the input object are set to bezero in each of the intervals in which the ER is im-plemented. Therefore, it is easily recognized that, inthis algorithm, the aspect of the HIO studied in Ap-

pendix A is not effective in many cases. Especially,it should be noted that, unlike the HIO, when theHIO�ER goes back to g s,ER�x, y� more than once, it islikely to move along the emerging path originatingfrom point P more than once. Therefore theHIO�ER is more likely to fall into a periodic statebefore reaching a solution than is the HIO alone.

C. Proposed Algorithm

As was studied in the above subsection, the HIO�ERis more likely to fall into a periodic state before reach-ing a solution than is the HIO alone. To deal withthis problem inherent in the HIO�ER, we note thatthe CPHIO can converge to one of the output-stagnation objects of the HIO if the value of � is nottoo large, and note that the set of the convergenceoutput objects of the ER is included in the set of theoutput-stagnation objects of the HIO. �Needless tosay, this means that the ER also converges to anoutput-stagnation object of the HIO.� Therefore theCPHIO can be thought of as an extension of the ERfor the case in which the input object is not confinedin the ODC space. So, in this paper, we construct,instead of the HIO�ER, the HIO�CPHIO. More con-cretely, the HIO�CPHIO consists of K cycles of theHIO and the CPHIO, where one cycle consists of KHiterations of the HIO, followed by KC iterations of theCPHIO, where K, KH, and KC are the prescribednumbers.

4. Results of Computer Simulations

Now, let us show the results of the computer simu-lations, which were carried out to examine the per-formance of the proposed algorithm. The originalobject f �x, y� is assumed to be a 32 � 32 object witha circular support, which is pictured on a 64 � 64array and is displayed in Fig. 2 by use of a printerwith 256 gray levels. In our experience, reconstruc-tion of an object with a circular support is much moredifficult than that of an object with a non-circularsupport. As the object-domain constraint, the 32 �32 square support was imposed along with nonnega-tivity and, as the initial input object, an array ofuniformly distributed random numbers was em-ployed. Two types of the HIO�CPHIOs were con-structed: one was the HIO�CPHIO for which onecycle consisted of 70 iterations of the HIO with � �0.8 followed by 230 iterations of the CPHIO with � �0.2 and the other was the type for which the number

Fig. 1. Schematic representation of how the HIO behaves in theoutput-stagnating state in which the output object of the HIO staysat a convergence output object of the ER, denoted as g s,ER and howthe HIO emerges from it. P and Q represent the points g s,ER

D andg s,ER, respectively. In this figure, the space passing through pointP and parallel to the complement of the ODC space is assumed tobe of two dimensions and to coincide with the plane of the paper.In this output-stagnating state, although the output object of theHIO stays at point Q, the input object moves along the emergingpath, which lies in the plane of the paper and is parallel to thevector g s,ER

ND. In this figure, three examples of the emergingpath are indicated.

Fig. 2. Original object.


of iterations of the CPHIO in one cycle was increasedto 930. In what follows, we denote the former as theHIO�CPHIO �70�230� and the latter as the HIO�CPHIO �70�930�. For comparison, the HIO�ER forwhich one cycle consisted of 70 iterations of the HIOwith � � 0.8 followed by 30 iterations of the ER wasconstructed, along with the HIO alone with � � 0.2.Because the movement of the input object of theCPHIO in the direction in which the Fourier-domainerror decreases does not always result in a decreasein the object-domain error, the convergence speed ofthe CPHIO is much lower than that of the ER. Thuswe assigned much larger values to the number ofiterations of the CPHIO in one cycle for the two typesof the HIO�CPHIOs, in comparison with the valuegiven to the number of iterations of the ER in onecycle of the HIO�ER.

Now let us look at the case in which the Fouriermodulus was not contaminated by noise. �In theuncontaminated case, we imposed, as the Fourier-domain constraint, the Fourier modulus �F�u, v�� thatwas computed from the original object f �x, y� itself.Therefore f �x, y� was the object that satisfied both ofthe object-domain and the Fourier-domain con-straints, which means, of course, that f �x, y� was asolution. Consequently, the existence of a solutionwas guaranteed in our simulations for the uncontam-inated case.� In Fig. 3�a�, the relation of the normal-ized object-domain error defined by

Eom � � �om2

�MN�1 �u

�v

�F�u, v��2�1�2

(27)

versus the number of iterations m obtained by theexecution of 3 � 104 iterations of the HIO�ER �that is,by the execution of 300 cycles of the HIO�ER� isshown �but only for the first 1.5 � 104 iterations�,whereas in Fig. 3�b�, those relations obtained by run-ning two types of the HIO�CPHIOs and the HIOalone are given. From these figures, the following isseen: In the case of the HIO�ER, Eom fell into aperiodic state19 and, as a result, Eom did not decrease

to less than 1.7 � 102; whereas, in both cases of theHIO�CPHIOs, Eom decreased effectively to zero, al-though the decreasing speed in the HIO�CPHIO �70�930� is much lower than that in the HIO�CPHIO�70�230�. It is also seen that, in the HIO alone, Eomdeceased effectively to zero. More concretely, in thecases of the HIO�CPHIO �70�230� and the HIO�CPHIO �70�930�, Eom decreased to less than 105

within 9 � 103 and 2.5 � 104 iterations, respectively.While in the case of the HIO alone, Eom decreased toless than 105 within 8.7 � 103 iterations. Notethat, in both cases of the HIO�CPHIOs, in the inter-vals in which the CPHIO was implemented, Eom de-creased, overall, monotonically as in the intervals inwhich the ER was executed in the HIO�ER. In theHIO�ER, because Eom did not converge to any value,we chose, as the estimate of f �x, y�, the output objectfor which the value of Eom was smallest from amongthe output objects produced by 3 � 104 iterations ofthe HIO�ER. We call this estimate the recon-structed object by 3 � 104 iterations the HIO�ER, andindicate it in Fig. 4�a�. This figure, along with Fig.3�a�, tells us that the HIO�ER could not locate f �x, y�.Also, in Fig. 4�b�, we show the output object producedat the 9 � 103th iteration of the HIO�CPHIO �70�230�. It can be seen that the appearance of thisobject is indistinguishable from that of f �x, y�. �Boththe HIO�CPHIO �70�930� and the HIO alone couldalso produce output objects that had appearances in-distinguishable from that of f �x, y�, although thoseoutput objects have not been indicated in this paper.�In addition to Eom, we introduced the normalized trueerror defined by

Etm � �� x,y��S

� g̃ m� x, y� � f � x, y��2

�x

�y

� f � x, y��2 1�2

, (28)

where S is the square support imposed as the object-domain constraint, and g̃ m�x, y� represents g m�x, y� orthe object obtained when it is rotated by �. Al-though this error can be calculated only when f �x, y�

Fig. 3. Relations of Eom versus the number of iterations m in the uncontaminated case. �a�, Relation obtained by the HIO�ER; �b�relations obtained by the HIO�CPHIO �70�230�, by the HIO�CPHIO �70�930�, by the HIO alone.


is known, in our experience it is extremely useful inthe evaluation of the visual quality of the recon-structed object. For the objects shown in Fig. 4�a�,Eom � 1.74 � 102; Etm � 3.48 � 101, whereas, forthe object in Fig. 4�b�, Eom � 8.63 � 106; Etm �9.62 � 106.

As is apparent from Eqs. �16� and �20�, the numberof times the DFT and the inverse DFT operationstaken together have to be executed in one iteration ofthe HIO is two, whereas, that number in one iterationof the CPHIO is four. Therefore we can say that,although the numbers of iterations of the HIO aloneand the HIO�CPHIO �70�230� that were required forEom to decrease to less than 105 were comparablewith each other, the computational burden of the HIOalone was less severe than that of the HIO�CPHIO�70�230�. When we take into account this fact, themost effective method in the uncontaminated case isconsidered to be the HIO alone.

Now, to show the superiority of the HIO�CPHIO tothe HIO alone, let us proceed to the case in which theFourier modulus is contaminated by noise. Assum-ing that the quantity directly obtained through themeasurement is the Fourier intensity of f �x, y� ratherthan its Fourier modulus, we added the Gaussianrandom variables to the intensity and defined therate of noise to the Fourier modulus as follows:

NR � ��u�

v��Fn�u, v�� F�u, v��2

�u

�v

�F�u, v��2 �1�2

� 100%,

(29)

where �Fn�u, v�� represents the Fourier modulus con-taminated by noise.

Fig. 5 plots the relations of Etm versus NR for thereconstructed objects by 100 cycles of the HIO�CPHIO �70�230� and by 300 cycles of the HIO�CPHIO �70�930�. �As in the HIO�ER in theuncontaminated case, the output object for which theobject-domain error took the smallest value fromamong the output objects produced by, for example,100 cycles of the HIO�CPHIO �70�230� is called thereconstructed objects by 100 cycles of the HIO�CPHIO �70�230�.� For comparison, in this figure,plots for the reconstructed objects by 3 � 104 itera-tions of the HIO alone are also indicated. From Fig.5, it is seen that, when NR � 2%, there was littledifference among the values of Etm for the recon-structed objects by these three algorithms; however,when NR � 2%, especially when NR � 5%, Etm for thereconstructed objects by both HIO�CPHIOs took con-siderably lower values than Etm for the reconstructedobjects by the HIO alone did. It was the HIO�CPHIO �70�930� that worked best for decreasing thevalues of Etm. Figures 6�a� and 6�b� show the recon-structed objects by the HIO�CPHIO �70�930� and bythe HIO alone, respectively, in the case in whichNR � 8%. Also, in Figs. 7�a� and 7�b�, the recon-structed objects in the case in which NR � 15% aregiven. From these figures it is recognized that, inboth the cases in which NR � 8% and NR � 15%, thevisual quality of the reconstructed objects by theHIO�CPHIO �70�930� is much better than that of thereconstructed objects by the HIO alone. For the re-constructed objects indicated in Figs. 6�a� and 6�b�,Eom � 3.71 � 102; Etm � 1.84 � 101, and Eom �7.10 � 102; Etm � 2.28 � 101, respectively,whereas, for the reconstructed objects given in Figs.7�a� and 7�b�, Eom � 5.62 � 102; Etm � 3.01 � 101,and Eom � 1.03 � 101; Etm � 4.00 � 101, respec-tively.

In the contaminated case, we have a situation inwhich a solution does not exist. To deal with this

Fig. 4. Reconstructed objects in the uncontaminated case. �a�Reconstructed object by 3 � 104 iterations of the HIO�ER, forwhich Eom � 1.74 � 102 and Etm � 3.48 � 101. �b� Recon-structed object at the 9 � 103th iteration of the HIO�CPHIO �70�230�, for which Eom � 8.63 � 106 and Etm � 9.62 � 106.

Fig. 5. Relations of Etm versus NR: �open circle� for the recon-structed objects by 100 cycles of the HIO�CPHIO �70�230�; �filledcircle� for the reconstructed objects by 300 cycles of the HIO�CPHIO �70�930�; �open triangle� for the reconstructed objects by3 � 104 iterations of the HIO alone.


situation, in Ref. 13, we introduced a quasi-solutionthat is significant. Because the quasi-solution is amember of the set of the output-stagnation objects ofthe HIO and because the HIO and, thus, the HIO�

CPHIO cannot be fixed at any object when the solu-tion does not exist, it is important, especially in thecontaminated case, for the HIO�CPHIO to have theability to visit many territories of the output-stagnation objects and the ability to come close to theoutput-stagnation objects in those territories. Supe-riority of 300 cycles of the HIO�CPHIO �70�930� to100 cycles of the HIO�CPHIO �70�230� suggests that,in the HIO�CPHIO, those abilities can be strength-ened by increasing the number of cycles and the num-ber of iterations of the CPHIO in each cycle.

Here, note that, in Ref. 13, the separated HIO al-gorithm, which was abbreviated as SHIO, was con-structed to attempt to find the quasi-solution. As issuggested by its name, in the SHIO, the CPHIO andthe emerging term are used separately. Because theSHIO emerges from an output-stagnation object lo-cated by the CPHIO by changing the phase of Gm�u,v� at only one pair of frequencies by � by use of theemerging term, the probability is conjectured to behigh that, via the emergence, the SHIO goes into theterritory of a neighboring output-stagnation object.Consequently, the SHIO appears to have a forte forsearching thoroughly a comparatively small area inthe MN-dimensional space, looking for the output-stagnation objects. Therefore, the SHIO can hardlybe expected to search a large area in the MN-dimensional space with rapid steps. To deal withthis situation, the SHIO was preceded by the HIOitself and the resultant algorithm was denoted as theHIO-SHIO. Speaking more concretely, in the HIO-SHIO, first, the HIO is executed by a prescribed num-ber of iterations and, then, the SHIO is run using, asthe initial input object, the input object that producedthe output object with the smallest object-domain er-ror from among the HIO iterations. In our simula-tions, to understand the performance of the HIO�CPHIO more deeply, the results obtained by the HIO-SHIO were compared with the results by the HIO�CPHIO. See Fig. 8. This figure gives the relationof Etm versus NR for the reconstructed objects by theHIO-SHIO, which consists of 6 � 104 iterations of theSHIO with � � 0.5 preceded by 3 � 104 iterations ofthe HIO with � � 0.2. In this figure, that relationfor the reconstructed objects by 300 cycles of theHIO�CPHIO �70�930� is also given, although it hasalready been presented in Fig. 5. This figure tells usthat the HIO�CPHIO �70�930� could produce betterresults than the HIO-SHIO except when NR � 3%.Note that the noise of level NR � 3% could not disturbthese two algorithms much.

To conclude, we would like to show the resultsobtained by the �HIO�CPHIO�-SHIO, that is, theSHIO preceded by the HIO�CPHIO. See Fig. 8again. In this Figure, the relation of Etm versus NRfor the reconstructed objects by 6 � 104 iterations ofthe SHIO with � � 0.5 preceded by 300 cycles of theHIO�CPHIO �70:930� is plotted. It is seen that thegraph of Etm obtained by this algorithm lies lowerthan the graph for the reconstructed objects by 300cycles of the HIO�CPHIO �70:930�. However, Figs.9 and 7�a� show us that the visual quality of the

Fig. 6. Reconstructed objects in the case in which NR � 8%. �a�Reconstructed object by 300 cycles of the HIO�CPHIO �70�930�, forwhich Eom � 3.71 � 102 and Etm � 1.84 � 101. �b� Recon-structed object by 3 � 104 iterations of the HIO alone, for whichEom � 7.10 � 102, Etm � 2.28 � 101.

Fig. 7. Reconstructed objects in the case in which NR � 15%. �a�Reconstructed object by 300 cycles of the HIO�CPHIO �70�930�, forwhich Eom � 5.62 � 102 and Etm � 3.01 � 101. �b� Recon-structed object by 3 � 104 iterations of the HIO alone, for whichEom � 1.03 � 101 and Etm � 4.00 � 101.


reconstructed object by this algorithm was not im-proved appreciably, when compared with the visualquality of the reconstructed object by 300 cycles of theHIO�CPHIO �70�930�. The results given in this lastparagraph also demonstrate the effectiveness of theproposed algorithm, the HIO�CPHIO.

5. Conclusion

The HIO is in many cases combined with the ER toattempt to stabilize the HIO. However, this combi-nation algorithm, the HIO�ER, has a serious prob-lem. That is, in the HIO�ER, because the values ofthe input object outside the support are set to be zeroin the intervals in which the ER is implemented, theHIO�ER is more likely to fall into a periodic statethan the HIO alone.

The purpose of this paper was to deal with thisproblem inherent in the HIO�ER. To this end, weemphasized that, because the CPHIO can converge toone of the output-stagnation objects of the HIO un-less the value of the feedback parameter is too large,and because the convergence output objects of the ERare the members of the set of the output-stagnationobjects of the HIO, the CPHIO can be thought of as an

extension of the ER for the case in which the inputobject outside the support can have nonzero values,even though the Fourier-domain and the object-domain errors do not always decrease monotonically.Thus we proposed to combine the HIO not with theER but with the CPHIO and denoted the combinedalgorithm as the HIO�CPHIO.

The results of the computer simulations showedthat, in the noise-free case, the HIO�CPHIO couldovercome the difficulty inherent in the HIO�ER.More concretely, the HIO�CPHIO could reconstructcompletely the object for which the HIO�ER fell intoa periodic state, just as the HIO alone could.

The superiority of the HIO�CPHIO compared tothe HIO alone was then shown by the computer sim-ulation results in the contaminated case. That is, itwas demonstrated through the computer simulationresults that the HIO�CPHIO could reconstruct theobjects with visual quality that was much better thanthe visual quality of the objects reconstructed by theHIO alone. In the contaminated case, we have asituation in which a solution does not exist. For thissituation, in Ref. 13, we introduced a quasi-solutionthat is significant. Because the quasi-solution is amember of the set of the output-stagnation objects ofthe HIO and because the HIO and, thus, the HIO�CPHIO cannot be fixed at any object when the solu-tion does not exist, it is important, especially in thecontaminated case, for the HIO�CPHIO to have theability to visit many territories of the output-stagnation objects and the ability to come close to theoutput-stagnation objects in those territories. Oursimulation results suggested that, in the HIO�CPHIO, those abilities could be strengthened by in-creasing the number of cycles and the number ofiterations of the CPHIO in each cycle.

In Ref. 13, to attempt to find the quasi-solution, weconstructed the SHIO in which the CPHIO and theemerging term were used separately. Then, be-cause the SHIO appeared to have a forte for search-ing thoroughly a comparatively small area in theMN-dimensional space, looking for the output-stagnation objects, the SHIO was preceded by theHIO itself and the resultant algorithm was denotedas the HIO-SHIO. In this paper, to understand theperformance of the HIO�CPHIO more thoroughly,computer simulation results obtained by the HIO-SHIO were compared with those obtained by theHIO�CPHIO. This comparison, along with the com-parison between the results obtained by the SHIOpreceded by the HIO�CPHIO and the results ob-tained by the HIO�CPHIO, confirmed further the ef-fectiveness of the HIO�CPHIO.

In closing, we would like to point out the following:we demonstrated the effectiveness of the HIO�CPHIO, especially in the contaminated case. TheHIO�CPHIO, however, has a drawback. Its compu-tational burden is considerably more severe than, forexample, the HIO alone. We are now investigatingways to decrease the computational burden of theHIO�CPHIO.

Fig. 8. Relations of Etm versus NR: �filled triangle� for the re-constructed objects by 6 � 104 iterations of the SHIO preceded by3 � 104 iterations of the HIO: �open circle� for the reconstructedobjects by 6 � 104 iterations of the SHIO preceded by 300 cycles ofthe HIO�CPHIO �70�930�. In this figure, the relation for thereconstructed objects by 300 cycles of the HIO�CPHIO �70�930� isalso indicated by use of the �filled circle� symbol, although thisrelation has already been presented in Fig. 5.

Fig. 9. Reconstructed object by 6 � 104 iterations of the SHIOpreceded by 300 cycles of the HIO�CPHIO �70�930�, for whichEom � 5.22 � 102 and Etm � 2.81 � 101; NR � 15%.


Appendix A. Derivation of the Aspect of the HIO that isReferred to in Section 1

Let us derive the aspect of the HIO which is referredto in Section 1 and is used in the discussion in Sub-section 3.B.

When the support imposed as the object-domainconstraint is represented as S, it can easily be recog-nized that

�� x, y�� x, y��S� � �� x, y�� x, y��D� (A1)

holds at each iteration. Therefore from the lowerpart of Eq. �16�,

gm1NS� x, y� � gm

NS� x, y� � �g mNS� x, y� (A2)

is obtained, where gmNS�x, y� is defined by

gmNS� x, y� � �0 � x, y� � S

gm� x, y� � x, y��S , (A3)

and is called the NS part of gm�x, y�. g mNS�x, y� is, of

course, the NS part of g m�x, y�. By using Eq. �A2�repeatedly, we can derive

gm1NS� x, y� � g0

NS� x, y� � � �k�0

m

g kNS� x, y�. (A4)

Eq. �A4� demonstrates that, in the NS part of the mthinput object of the HIO, all of the values of the NS partsof the output objects produced until the �m 1�thiteration accumulate. From this fact it is conjecturedthat, if m � n, the probability is extremely low that

gmNS� x, y� � gn

NS� x, y� (A5)

holds for every �x, y�. This means that the probabil-ity is also conjectured to be extremely low that theHIO will fall into a periodic state.

References and Notes1. J. R. Fienup, “Phase retrieval algorithms: a comparison,”

Appl. Opt. 21, 2758–2769 �1982�.2. J. C. Dainty and J. R. Fienup, “Phase retrieval and image

reconstruction for astronomy,” in Image Recovery: Theoryand Application, H. Stark, ed. �Academic, San Diego, Calif.,1987�, Chap. 7, pp. 231–275.

3. R. W. Gerchberg and W. O. Saxton, “A practical algorithm forthe determination of phase from image and diffraction planepictures,” Optik 35, 237–246 �1972�.

4. R. G. Lane, “Recovery of complex images from Fourier magni-tude,” Opt. Commun. 63, 6–10 �1987�.

5. C. R. Parker and P. J. Bones, “Convergence of iterative phaseretrieval improved by utilizing zero sheets,” Opt. Commun. 92,209–214 �1992�.

6. J. R. Fienup and C. C. Wackerman, “Phase-retrieval stagna-

tion problems and solutions,” J. Opt. Soc. Am. A 3, 1897–1907�1986�.

7. J. H. Seldin and J. R. Fienup, “Numerical investigation of theuniqueness of phase retrieval,” J. Opt. Soc. Am. A 7, 412–427�1990�.

8. H. Takajo, T. Takahashi, H. Kawanami, and R. Ueda, “Nu-merical investigation of the iterative phase-retrieval stagna-tion problem: territories of convergence objects and holes intheir boundaries,” J. Opt. Soc. Am. A 14, 3175–3187 �1997�.

9. H. Takajo, T. Takahashi, R. Ueda, and M. Taninaka, “Study onthe convergence property of the hybrid input–output algo-rithm used for phase retrieval,” J. Opt. Soc. Am. A 15, 2849–2861 �1998�.

10. H. Takajo, T. Takahashi, and T. Shizuma, “Further study onthe convergence property of the hybrid input–output algo-rithm used for phase retrieval,” J. Opt. Soc. Am. A 16, 2163–2168 �1999�.

11. We use this simplification throughout this paper. That is, theterm “the support” always means “the support imposed as theobject-domain constraint.”

12. R. G. Lane, “Phase retrieval using conjugate gradient minimi-zation,” J. Modern Opt. 38, 1797–1813 �1991�.

13. H. Takajo, T. Shizuma, T. Takahashi, and S. Takahata, “Recon-struction of an object from its noisy Fourier modulus: Idealestimate of the object to be reconstructed and a method thatattempts to find that estimate,” Appl. Opt. 38, 5568–5576�1999�.

14. A. Levi and H. Stark, “Image restoration by the method ofgeneralized projections with application to restoration frommagnitude,” J. Opt. Soc. Am. A 1, 932–943 �1984�.

15. A. Levi and H. Stark, “Restoration from phase and magnitudeby generalized projections,” in Image Recovery: Theory andApplication, H. Stark, ed. �Academic, San Diego, Calif., 1987�,Chap. 8, pp. 277–320.

16. Y. M. Bruck and L. G. Sodin, “On the ambiguity of the imagereconstruction problem,” Opt. Commun. 30, 304–308 �1979�.

17. M. H. Hayes, “The reconstruction of a multidimensional se-quence from the phase of its Fourier transform,” IEEE Trans.Acoust. Speech Signal Process. ASSP-30, 140–154 �1982�.

18. The CPHIO can converge to one of the output-stagnation ob-jects of the HIO unless the value of � is too large. Thereforethe CPHIO can relate any object in the MN-dimensional spaceto some output-stagnation object in the sense that, if theCPHIO starts from the object, it reaches the output-stagnationobject. The territory of an output-stagnation object is definedas the subspace in the MN-dimensional space that is formed bythe set of initial input objects related to the output-stagnationobject in this sense.

19. As the size of the object to be reconstructed becomes large, thenumber of the output-stagnation objects increases dramati-cally and, in addition, the way the territories of the output-stagnation objects are formed in the MN-dimensional spacebecomes complicated. So, strictly speaking, Fig. 3�a� does notshow that the HIO�ER and, thus, Eom fell into a ‘completely’periodic state as in the case of the 2 � 2 objects with L-shapedsupport, which was discussed in Ref. 10. Figure 3�a� does,however, allow us to say that Eom fell into an ‘effectively’periodic state.


Documents

Reconstruction of an Object from its Fourier Modulus: Development of the Combination Algorithm Composed of the Hybrid Input–Output Algorithm and Its Converging Part