Families of stationary patterns producing illusory ... Thus, any system has to interpret the rigid motion encoded in the sequence of images perceived by its retinae. As a system moves

Families of stationary patterns producing illusorymovement: insights into the visual system

CORNELIA FERMULLER, ROBERT PLESS and YIANNIS ALOIMONOS

Computer Vision Laboratory, Center for Automation Research, Institute for Advanced Computer Studies,Computer Science Department, University of Maryland, College Park, MD 20742-3275, USA

SUMMARY

A computational explanation of the illusory movement experienced upon extended viewing of Enigma,a static figure painted by Leviant, is presented. The explanation relies on a model for the interpretationof three-dimensional motion information contained in retinal motion measurements. This model showsthat the Enigma figure is a special case of a larger class of figures exhibiting the same illusorymovement and these figures are introduced here. Our explanation suggests that eye movements and/oraccommodation changes cause weak retinal motion signals, which are interpreted by higher-levelprocesses in a way that gives rise to these illusions, and proposes a number of new experiments tounravel the functional structure of the motion pathway.

1. INTRODUCTION

A visual illusion has recently given rise to a contro-versy in the vision research community. This illusionarises when viewing Enigma, a static figure paintedby Leviant. Enigma, shown in figure 1a, consists ofradial lines emanating from the centre of the im-age and interrupted by a set of concentric uniformlycoloured rings. Upon extended viewing of Enigma,most human subjects perceive illusory movement in-side the rings which keeps changing direction.

The nature of the controversy lies in the view ofresearchers regarding the location in the brain wherethe illusion takes place and the involved processes.Zeki (1994, 1995) and his co-workers (Zeki et al.1993) argue that higher-level processes are respon-sible for the perception of illusory movement. Thus,in their view, the Enigma static stimulus ‘inducesactivity in a given region of the visual cortex whichthen invests the stimulus with a particular percep-tual quality, the latter being entirely the constructof the brain’. Their arguments are based on mea-surements of changes in regional cerebral blood flow(rCBF) using the technique of positron emission to-mography. When comparing the rCBF measurementsduring the viewing of Enigma and the reference im-age shown in figure 1b (radial lines pass throughthe rings), they found comparable values in V1—an early area in visual processing, and for Enigmasignificantly higher values in V5 (or MT)—a laterarea in visual motion processing. In a diametricallyopposed viewpoint, Gregory (1993, 1995) argues forlow-level processes as the cause of the illusion. Hesuggests that the same processes responsible for theMacKay illusion (see figure 1c and MacKay (1957,1958)) cause illusory movement in Enigma. He pro-poses that ‘the dynamic shimmer seen in these static

figures has an optical cause: changes of size of theretinal image due to the usual rapid ‘hunting’ ofaccommodation’. A third opinion was recently pub-lished (Mon-Williams & Wann 1996) suggesting thatsmall roto-translational eye movements may be thecause of the illusion.

In this paper we present a computational expla-nation of the Leviant illusion. Our view is that theperception of illusory movement is due to the particu-lar architecture of the visual motion pathway, whichtakes as input, weak retinal motion measurementsdue to small eye movements. We agree with Gregorythat weak retinal motion signals due to accommo-dation or eye movements play an important role inthe creation of the illusion, but while such retinalmotion signals are sufficient to explain the MacKayeffect, they cannot by themselves explain the Leviantillusory movement. We also agree with Zeki that thereason for the illusion lies mainly in high-level pro-cesses. These processes must be triggered by retinalmotion signals, however weak they are. The majorcontribution of this work lies in the explanation ofthese higher-level processes in computational terms.

The basis of our approach is the computationaltheory of visual motion interpretation that we havedeveloped. Since autonomous systems, biological orartificial, move in their dynamic and changing envi-ronments, the most basic tasks they need to performare the understanding of their own three-dimensionalmotion as well as those of other moving entities.Thus, they have to solve the so-called egomotionestimation problem and the motion segmentationproblem. We have developed a theory that accountsfor the robust and efficient estimation of three-dimensional motion (see Fermuller 1995; Fermuller& Aloimonos 1995a, b, 1997). This theory postulatesa general architecture for its implementation, and

Proc. R. Soc. Lond. B (1997) 264, 795–806 c© 1997 The Royal SocietyPrinted in Great Britain 795 TEX Paper

796 C. Fermuller, R. Pless and Y. Aloimonos Insights into the visual system

this architecture is consistent with neurobiologicalfindings concerning the motion pathway in primates.Our hypothesis is that this architecture can serve asa working model for the human motion pathway andit is on this basis that we can explain the Leviantillusion.

The gist of the computational theory, as explainedin the next section, is that retinal motion measure-ments along various orientations all over the imageare grouped in particular ways. Each group consistsof motion measurements along predefined orienta-tion fields, which are called coaxis and copoint fieldsand possess a number of important properties mak-ing three-dimensional motion estimation most easyfrom a computational perspective.

The Enigma figure has the property that all itsmarkings can give rise to motion vectors belongingto only one of the predefined orientation fields. Ifindeed the Leviant illusion is an artifact of the par-ticular way the motion pathway is constructed forthe purpose of three-dimensional motion estimation,then any other image in the style of Enigma, with theproperty that all its markings can give rise to motionvectors belonging to only one of the predefined ori-entation fields, must give rise to a similar illusorymovement. We show that this is indeed the case. Onthe basis of our computational theory we can pro-duce a large number of images that give rise to illu-sory movement, some of which will be shown here.The theory also predicts that other figures not basedon the predefined orientation fields, even if they aresuperficially similar, will not cause the same illusorymovement.

The organization of the paper is as follows. Sec-tion 2 explains the computational model for the in-terpretation of visual motion on which this paper isbased. Section 3 provides a number of new imagesgiving rise to illusory movement, § 4 demonstratesthat the illusion cannot be due to a MacKay effectand gives a detailed explanation of the computationalsteps leading to the experience of it, and § 5 concludesthe work.

2. A COMPUTATIONAL THEORY OF THEANALYSIS OF VISUAL MOTION

Any system that moves in its environment has toreach an understanding of its own motion. Althoughan organism may move in a non-rigid manner as awhole, with its head, arms, legs or wings undergoingdifferent motions, the eyes move rigidly—that is, asa sum of an instantaneous translation and rotation.Thus, any system has to interpret the rigid motionencoded in the sequence of images perceived by itsretinae.

As a system moves in its environment, each pointof the environment has a three-dimensional velocityvector with respect to the system. To describe theperceived motion on the image plane the concept ofthe visual motion field has been used. This field cor-responds to the projection of the three-dimensionalvelocity vectors on the image plane; it depends on

(a)

(b)

(c)

Figure 1. (a) Enigma, giving rise to the Leviant illusion:fixation at the centre results in perception of a rotarymotion inside the rings. (b) The control figure used inZeki et al. (1993) in which there is no perception of rotarymotion. (c) MacKay rays, giving rise to a shimmering atthe centre and a circular propeller-like movement in theother parts of the figure.

Proc. R. Soc. Lond. B (1997)

Insights into the visual system C. Fermuller, R. Pless and Y. Aloimonos 797

FOE

(a) (b) (c)

AOR

Figure 2. (a) Purely translational; (b) purely rotational; and (c) general motion field due to rigid motion.

PP

(a) (b) (c)

Figure 3. Copoint fields.

(a) (b) (c)

Figure 4. Coaxis fields.

the three-dimensional motion and the structure ofthe scene in view. If the system’s motion is purelytranslational the motion has the structure of thefield shown in figure 2a. It consists of vectors em-anating from the point where the translation axispierces the image plane, called the focus of expan-sion (FOE) or focus of contraction (FOC), dependingon whether the observer approaches or moves awayfrom the scene. The length of each vector is inverselyproportional to the distance from the scene. If thesystem’s motion is purely rotational the motion fieldhas the structure of the field shown in figure 2b. Itconsists of vectors tangential to conic sections de-fined by the intersection of the image plane with thefamily of cones having the rotation axis as their axis.The point of intersection of the rotation axis withthe image plane is called the axis of rotation (AOR).If the system moves in a general manner involvingboth translation and rotation then the motion fieldamounts to the addition of a translational and ro-

tational field and no longer has a simple structure(for an example, see figure 2c). The problem of esti-mating a system’s three-dimensional motion then be-comes equivalent to the hard problem of decouplingthe translation from the rotation and recognizing thepositions of the FOE and AOR.

In actuality, the problem of three-dimensional mo-tion estimation is even harder because the measure-ments of retinal motion that can be made on theimage are due to movements of light patterns only,and this is not equivalent to the motion field. To de-note the exact movement of every point in the image,the term optical flow field has been used. Accuratevalues of the optical flow field, however, cannot pos-sibly be computed; in general, on the basis of localinformation, only the component of the flow perpen-dicular to edges in the image can be estimated—thisis the well-known aperture problem. In many cases, itis possible to obtain additional flow information (e.g.from features of high curvature, corners). Thus, the



+

--

--

--

-

----

-

--

+++

++

+

+ + + + +

-

+

-

-

-+

+

+

+

+

+

+

+

+

-

-

-

-

-

-

-

-

-

-

(a) (b)

Figure 5. Copoint and coaxis fields filled with qualitativemotion measurements.

input used for further motion processing is some par-tial optical flow information. A system, in order toreliably estimate its three-dimensional motion, mustmake use of retinal motion measurements globallyand in a redundant way. We have shown that this ispossible by using only the signs of flow measurementsin various directions. The major idea is based on se-lecting groups of measurements of the sign of retinalmotion along predefined directions, which form pat-terns in the image. These patterns encode the three-dimensional motion in a simple manner. The rest ofthis section summarizes the theory. For more infor-mation, the reader is referred to Fermuller (1995) andFermuller & Aloimonos (1995a, b, 1997).

We will consider two classes of orientation fieldsthat are essential in our approach. Along these ori-entation fields, image motion measurements will begrouped together.

The first class, called copoint fields, is defined byone point, P , on the image plane (this point could lieinside or outside the image). A copoint field consistsof unit vectors that at every image point are per-pendicular to the line connecting that point with P .Figure 3a–c shows three examples of copoint fieldswith point P : at the centre of the image (a); in theperiphery of the image (b); and at infinity (c).

The second class, called coaxis fields, is definedby one axis (that could intersect or be parallel tothe image plane) passing through the origin—nodalpoint—of the eye. Considering one axis, s, the coaxisfield is defined by the unit vectors that are normalto the conic sections resulting from the intersectionof the image plane with the family of cones havings as their axis. Figure 4a–c shows three examplesof coaxis fields. In (a), axis s is perpendicular to theimage at its centre. In (b), axis s intersects the imageat point s, and in (c) axis s is horizontal and parallelto the image plane.

Our method proceeds by making measurements ofthe sign of retinal motion along the orientations ofone coaxis or copoint field, and it does so for severalfields. When measurement of image motion is per-formed at a point (x, y) along a direction n, thereare three possibilities. Either image motion is mea-sured in the same direction as n, in which case we de-note the measurement by ‘+’; image motion is mea-sured in the opposite direction, where the measure-ment is denoted by ‘−’; or there is no motion, and the

measurement is denoted by ‘0’. Such measurementscan be easily made using Reichardt-like detectors orequivalent energy models (Adelson & Bergen 1985;Grzywacz et al. 1995; Reichardt 1961, 1987; Snippe& Koenderink 1994; Van Santen & Sperling 1984).

Thus, the input to the system is a number ofcoaxis and copoint fields with every vector in eachfield labelled ‘+’, ‘−’ or ‘0’, and it is on the basisof these ‘maps’ that subsequent processing for three-dimensional motion estimation will take place. (Fig-ure 5 shows one copoint and one coaxis field ‘filled’with ‘qualitative’ motion measurements.)

The maps, consisting of the discrete measurements+, − and 0, encode the three-dimensional motion inthe form of global patterns. We provide next a simpleexample of such a pattern and we refer later to thegeneral case.

Consider the coaxis vectors due to an axis perpen-dicular to the image at its origin. (As shown in fig-ure 5a, these vectors are perpendicular to the circlescentred at the origin.)

Assume first that there is only translation. In fig-ure 6a, which illustrates this situation, the coaxisvectors are shown in boldface and the motion vec-tors (which are actually unknown) in grey. Clearly,the motion vectors emanate from the FOE. The valueof the sign of retinal motion along a coaxis vector ata point depends on the angle this vector makes withthe motion vector (since measurement of retinal mo-tion along a coaxis vector amounts to the projectionof the motion vector at that point on the coaxis vec-tor).

For the coaxis vectors at the points lying on thecircle with diameter equal to the segment connectingthe FOE to the image centre, the angle between themotion vector and the coaxis vector is 90◦. For thepoints inside the circle the angle is greater than 90◦and for the points outside it is smaller than 90◦. Thismeans simply that inside the circle the measurementof the sign of retinal motion along the coaxis vectorswill be negative, outside the circle positive, and onthe circle zero.

If there is only rotation, a similar pattern is ob-tained. Rotation exists around all three axes, x, yand z, but the rotation around the z-axis does notcontribute any value to the image motion measure-ments along these coaxis vectors because the flowit produces is perpendicular to them. In this case,measurements of the sign of image motion along thecoaxis vectors will be positive and negative, sepa-rated by a straight line passing through the imageorigin. The line represents the projection of the ro-tation axis (α, β, γ) on the image plane (figure 6b).

For the general case of rigid motion we need tocombine the results of figure 6a, b. Superimposing thetwo figures we find that whenever positive and pos-itive come together the result will be positive, whilewhen negative meets negative the result will be neg-ative. Whenever positive and negative come togetherthe result depends on the actual values and cannot beknown without knowledge of the depth of the scene.

Thus, for the coaxis vectors considered, the mea-surements of the sign of image motion along them



(a) (b) (c)

y = βα

x FOE: Negative

: Positive

: Don't know

FOE

O

Figure 6. Motion pattern for a coaxis vector with centre at the origin. (a) Motion measurements due to translation areseparated by a circle, which passes through the FOE, according to the signs of their values (+,−, 0). (b) Separationof motion measurements due to rotation by a line passing through the AOR, lying somewhere on the line y = (β/α)x.(c) Motion pattern for a general rigid motion.

AOR

(a) (b)

FOEs0

s0

AORFOE

: Negative

: Positive

: Don't know

Figure 7. (a) Coaxis vectors and (b) copoint vectors in the plane with the patterns superimposed. Note that the ‘don’tknow’ area contains both positive and negative vectors. In (a) s0 is the point where the axis pierces the image planeand in (b) s0 is the point defining the field.

form a global pattern on the image defined by a cir-cle and a straight line (figure 6c), defining an area ofonly positive values, an area of only negative valuesand an area where both positive and negative valuesare possible depending on the scene. Localization ofthis global pattern in the image provides informationabout three-dimensional motion. Indeed, the FOE isthe antidiametric point of the origin, and the linerepresents the projection of the rotation axis on theimage.

Figure 7 describes the global patterns for generalcoaxis and copoint fields. In both cases the patternis defined by a straight line and a conic section. Forcoaxis fields the conic section separates the transla-tional measurements into positive and negative, andthe straight line separates the rotational measure-ments. For the copoint fields we have the dual situ-ation. In figure 7a, the conic section passes throughthe FOE and the straight line through the AOR. Infigure 7b, the dual situation is observed.

Thus, performing measurements along the vectorsof several coaxis and copoint fields, we obtain manypatterns which result in an unambiguous determina-tion of the FOE and the AOR (see Fermuller (1995)and Fermuller & Aloimonos (1995a, b, 1997) for de-tails).

These patterns are simple in the sense that they re-duce the complexity of the three-dimensional motion

estimation problem in the most efficient way possi-ble. While general rigid motion is encoded in the im-age plane in the form of five parameters—two for thedirection of translation and three for the rotation—each of these depends on only three parameters. Thecoaxis and copoint fields underlying these patternsare the only vector fields that can give rise to such asimplification.

The theory described above can be used by anysystem with sufficient computing power to estimatethe FOE and AOR very robustly from the signs ofimage velocity measurements. This is not to say thatall aspects of motion estimation must be based onthis theory. Undoubtedly, any system uses additionalextraretinal information to solve this problem, suchas inertial sensor information (Aloimonos 1993). Inaddition, systems employ estimates of the values ofthe image velocity measurements in different ways toobtain velocity information.

3. NEW ILLUSIONS IN THE STYLE OFENIGMA

There exists a relationship between the structureof the Enigma figure and the copoint vector field withcentre, P , at the centre of the image. The radial raysof Enigma are perpendicular to these copoint vectorsand the homogeneous rings are tangential to them,



(a)

(b)

Figure 8. Illusory movement figures based on copointfields. Experience of the illusion is facilitated by fixat-ing on the white dot displayed at the image centre.

and thus normal to the rays. We applied this rela-tionship to obtain new figures from different copointfields. Figure 8 shows two such examples with thecentre of the copoint field in 8a being at the peripheryof the image, and in 8b at infinity. Both figures pro-duce illusory movement inside the bands. It is worthnoting that example 8b is well known in the literature(MacKay 1961).

In a similar manner we can apply the same prin-ciple to coaxis vector fields and obtain new figuresexhibiting the same illusory movement. In particu-lar, given a coaxis vector field, we choose rays per-pendicular to the coaxis vectors and homogeneousbands tangential to them. Figure 9a–c gives exam-ples of three different coaxis fields.

(a)

(b)

(c)

Figure 9. Illusory movement figures based on coaxis fields.As the coaxis vectors are defined by the intersection ofthe image plane with a family of cones, the related figuresdepend on the focal length. Here the focal length is chosento be about equal to the size of the image. Thus, theillusions are best experienced at a short distance.



(a)

(b)

Figure 10. No significant illusory movement is perceivedwith figures based on vector fields different from coaxisand copoint fields.

No significant illusory movement is experiencedwhen the figure is constructed by applying the sameprinciple to vector fields different from coaxis or co-point fields. This is demonstrated in figure 10, show-ing ‘leaf’ and ‘sinusoidal’ patterns, respectively.

4. EXPLANATION OF ENIGMA ANDRELATED ILLUSIONS

A look at motion interpretation from a purely com-putational point of view gives us insight into thecomplexity and possible structure of the computa-tions involved in three-dimensional motion estima-tion and segmentation. In § 2 a theory was describedthat showed how to estimate the three-dimensionalmotion of a system moving rigidly in a static envi-

ronment on the basis of global patterns of the spa-tiotemporal measurements. Turning from the theoryto an implementation, we have to consider the factthat systems moving in the real world don’t move instatic environments. They are confronted with scenescontaining objects moving in various ways. In orderto estimate three-dimensional motion from patterns,the image must be segmented into areas of coher-ent motion; however, the segmentation problem fora moving system is very difficult. It is an ill-posedproblem and at this point we do not know the bestway to solve it. We do, however, understand someproperties that an optimal solution must have.

Segmentation involves both detection and localiza-tion, the two problems giving rise to the antagonisticconflict at the heart of pattern recognition. From theviewpoint of signal processing, it has been proved(Gabor 1946) that any single (linear) operator cananswer only one of these questions with sufficient ac-curacy. In order to detect or recognize the existenceof different motions within image patches, a suffi-ciently large number of flow measurements must beavailable. To obtain the measurements, some form ofsmoothing or averaging in the spatial as well as thetemporal domain has to be done. On the other hand,the localization of objects requires the use of veryaccurate local flow information.

It can easily be understood that it is not possibleto perform the segmentation solely on the basis ofimage measurements. The reason is that motion, aswell as depth discontinuities, manifest themselves asdiscontinuities in the flow field. Furthermore, even ifdifferent motions could be separated, there remainsthe question of which motion belongs to the staticbackground and which motions belong to the inde-pendently moving objects. In order to segment suc-cessfully, some additional three-dimensional motioninformation and probably some shape informationabout the scene accumulated over time has to betaken into account.

The logical consequence of the interplay betweenthe two problems of motion estimation and segmen-tation is that they are closely coupled and must besolved simultaneously. An optimal solution for seg-mentation requires the system to first detect differ-ent motions in the image and obtain some estimate ofthe three-dimensional motion with the use of a globalflow field representation. The derived estimates thenhave to be utilized to accurately localize and also rec-ognize independently moving objects from local flowfield information. In an additional step the resultsof the segmentation could be used for more accuratemotion estimation.

From the biological side, in the past decades muchinformation has been uncovered about the structureand simple functional properties of neurones in thevisual motion pathways of primates (Zeki 1993). It iswell established that from the earliest stages of thePa ganglion cells and magnocellular LGN cells, overthe various layers in V1 to area MT and MST, anincrease in the sizes of the receptive fields occurs.Furthermore, an onset of orientation selectivity hasbeen observed, when a bar is moved perpendicular



to its direction. Whereas the neurones in the ear-liest stages (Pa ganglion cells, magnocellular LGNcells and cells of layer 4Ca in area V1) respond al-most equally well to every direction, cells in layer 4Cof V1 show a preference for a particular orientationand in layer 4B and 6 of V1 as well as in V2 andMT cells are direction-selective (Movshon 1990) (seefigure 11). For cells later in the motion pathway inMT and MST, which can have very large receptivefields ranging from 30 to 100 ◦C, selectivity to par-ticular three-dimensional-motion configurations hasbeen reported (Andersen et al. 1990; Duffy & Wurtz1991a, b; Lagae 1991; Orban et al. 1992; Tanaka &Saito 1989).

The neurobiological findings just described do notcontradict our theory of motion estimation. One caneasily envisage an architecture that, using neuroneswith the properties of those in a primate’s motionpathway, implements a global decomposition of themotion field using the signs of motion vectors alongappropriately chosen directions. Neurones of the kindshown in figure 11a could be involved in the estima-tion of local retinal motion information; they couldbe thought of as computing whether the projectionof retinal motion along some direction is positive ornegative. Neurones of the kind shown in figure 11bcould be involved in the selection of local vectorsin particular directions, as parts of the various pat-terns discussed. Neurones of the kind shown in fig-ure 11c could be involved in computing the signs(positive or negative) of pattern vectors for areas inthe image; that is, they might compute, for imagepatches of different sizes, whether the flow there ispositive or negative. Finally, neurones of the kindfound in MT and MST could be the ones that piecetogether the parts of the patterns previously found,into global patterns that are matched with prestoredglobal patterns. Such matches provide informationabout three-dimensional motion.

The knowledge we have about the neurobiologyof the motion pathway, together with the computa-tional theory of motion estimation and the computa-tional arguments regarding segmentation, provide uswith an explanation for the illusions described above.

Before we give further details we apply our com-putational principles to show that the illusory move-ment seen in the MacKay rays can be explained byprocesses taking place at very early stages. Subse-quently, we discuss why the Enigma figure, and ournew figures, defy this type of explanation.

MacKay reported illusions of movement in imagesof repetitive patterns. For rays emanating from theimage centre (see figure 1c) he describes (MacKay1958) two different kinds of movement: a shimmerin the middle of the image and a circular movementaway from the centre.

Small eye movements due to instability as well assmall accommodative movements give rise to retinalmotion signals. For simplicity we model this move-ment as a translation along the x-axis, but similar re-sults would follow from other movements. At the veryearly processing stages the receptive fields of cells are

very small, and thus the motion measurements gen-erated, e.g. by biological models like Reichardt detec-tors or by energy-based models, approximate normalflow measurements (i.e. motion measurements per-pendicular to edges). For a small translational mo-tion along the x-axis, the normal flow field is shownin figure 12. The irregularity of the motion in thecentre of the figure—the shimmering—is created be-cause the translational image motion is greater thanthe separation of the rays, causing the local mo-tion detectors to give results that are incorrect andvary rapidly as functions of their spatial coordinates(the so-called aliasing problem). Outside of this cen-tral region, the motion field is both spatially smoothand consistent with motion along the complementarypatterns proposed by MacKay (1957). A random se-quence of small translational motions of the eye willcontain subsequences of temporally consistent imagemotion, giving rise to the occasional rapid propeller-like motions around the edge of the figure.

This explanation, involving only low-level pro-cesses, cannot possibly account for the movementseen in the Leviant illusion.

If we consider a small translational eye movement,we obtain a motion field like the one in figure 13a.Since the regions inside the rings are completely ho-mogeneous and there are no features to provide inputto the motion detectors, no motion is detected there.Regardless of accommodation movements, microsac-cades or slow drifts of the eye, the putative causes ofthe MacKay motion (Gregory 1993; Mon-Williams &Wann 1996), the retinal input on the region of theretina that is seeing the rings does not change. How-ever, the clear perception is of a motion exclusivelywithin the rings (figure 13b).

One might ask whether processing very early inthe motion pathway (if not the retinal signals them-selves) could create the perceived movement. Suchprocessing could only consist of spatial and temporalsmoothing. Indeed, spatial and temporal integrationcauses the cancellation of the movement along therays. However, it alone would not give rise to move-ment within the rings. Thus simple low-level process-ing cannot explain the illusion.

The new illusory movement patterns support ourtheory about the construction of the early visual sys-tem for the purpose of three-dimensional motion es-timation and segmentation, which is as follows.

We assume that when viewing the images and fix-ating on a point, some small eye movements takeplace which at the earliest stages cause the gener-ation of a flow field. The motion measurements areperpendicular to the rays, as shown in figure 13a. Atthe next stages of the motion pathway there are neu-rones having increasingly large receptive fields whichare tuned to motion signals that form parts of coaxisand copoint patterns.

As described earlier, various layers of cells par-ticipate in the process of evaluating motion pat-terns, and at the final stage an estimate of three-dimensional motion is obtained. Each of the neuronesinvolved receives flow field information from earlierneurones, when available, and performs a spatial and



A

B

C

D

V1, layer 4C

V1, layer 4B

V1, layer 4CLGNdRetina

Receptive fields

+

_

_

_

++

+

--

-

--+

+-

+

++ +

+

-

-

-

-

-

Directional tuning

0.1 degree

0.2 degree

0.2 degree

1.0 degree

V1, layer 6 V2

MT

+

+_

_

_

_

_

Figure 11. The spatial structure of visual receptive fields and their directional selectivity at different levels of themotion pathway (from Movshon 1990): the spatial scales of the receptive fields (0.1◦, etc.) listed here are for neuronesat the centre of gaze; in the periphery these dimensions would be larger. The polar diagrams illustrate responses tovariation in the direction of a bar target oriented at right angles to its direction of motion. The angular coordinate inthe polar diagram indicates the direction of motion and the radial coordinate the magnitude of the response.

temporal integration or smoothing. The system eval-uates a large number of coaxis and copoint fields. Aset of neurones is devoted to each field, with the sin-gle neurones covering certain areas of the image andtheir receptive fields overlapping. For every illusoryfigure there should exist exactly one set of neuronesresponsible for evaluating the related vector field ora very similar one.

When exposed to one of the illusory figures, thereceptive fields of some of the cells in the set willcover both an area containing the rays and an areacontaining the homogeneous part. During the pro-cess of pattern evaluation due to the smoothing theseneurones create information within the homogeneousparts.

After the neurones with the largest receptive fieldsinvolved in pattern matching have been excited andhave produced three-dimensional motion estimates(which, however, mean nothing to the stationary ob-server), the estimates are fed back to the earlier cells,which then have to perform the exact localization ofmoving objects.

At this stage, temporal integration must cancelout the flow information within the areas covered bythe rays. It, however, does not cancel out the motionwithin the homogeneous regions since the responsesfrom the neurones there do not contradict an existingmotion in these areas. Since the system’s task, dur-ing this processing stage, is to accurately segment thescene, the edges of the homogeneous regions are insome way perceived as motion discontinuities, mo-tion within the homogeneous regions is reinforced,and as a result a strong motion in one direction isperceived. As the small eye movements change suf-ficiently to change the directions of the elementarymotion measurements, the direction of motion in thehomogeneous areas also changes.

As mentioned before, our explanation of segmenta-tion is still at a high level. In the explanation given,we concentrated only on motion processes, in par-ticular, three-dimensional motion interpretation andmotion segmentation. Recently, neurones believed tobe involved in shape processing have been identifiedwhich respond to patterns similar in structure to the



0 0.4

0.20-0.2-0.40

0.4

0.1

0.05

0

–0.05

–0.1 –0.05 0 0.05 0.1

–0.1

0.2

0

-0.2

-0.4

Figure 12. Motion measurements due to small translational eye movements in MacKay rays. On the right is an expandedview of the centre of the flow field; the spatially frequent changes in the velocity constitute the shimmer in the centreof the image.

0.60.40.20-0.2-0.4-0.6

0.6

0.4

0.2

0

-0.2

-0.4

10.50-0.5-1

1

0.5

0

-0.5

(a) (b)

Figure 13. (a) Motion measurements due to small translational eye movements in Enigma. (b) Perceived flow field inLeviant illusion.

coaxis and copoint fields (Gallant et al. 1993). It maybe the case that other processes devoted to the anal-ysis of shape, and also to the analysis of colour or in-tensity, play an additional role in the creation of theillusion. As a matter of fact, intensity influences thestrength of the illusion. As already observed by Le-viant (1996), the illusory movement appears weakestfor black or white bands and strongest for interme-diate shades of grey.

The explanation we provided concentrated on theessential computations, and we have performed testswith additional figures to obtain support for thesteps of the computational theory as follows. First,in figure 14a we replaced the regular radial line pat-tern with a pattern of random radial line segments.The perception of illusory movement still occurs, al-though some viewers report it to be of a weaker na-ture. This figure provides evidence that it is not cellstuned to very regular patterns (i.e. rays at equal dis-tance and rings perpendicular to these rays) whichare responsible for the illusion. Instead, it supportsthe theory that single flow values perpendicular to

the rays are computed which at later stages are con-glomerated. Together with figure 14b, which does notproduce the MacKay effect, it also provides evidencethat the processes responsible for the Enigma illu-sion must be different from those responsible for theMacKay illusion. Figure 14b consists of random linesegments only. As can be verified by examination,these lines do not give rise to a MacKay effect; fig-ure 14a, on the other hand, does give rise to motionperception within the rings.

(2) Our theory stipulates that within the systemthere exist neurones having increasingly large recep-tive fields which piece together the parts of the globalmotion patterns. If this is the case, weak perceptionsof movement should occur within image patches if afigure is viewed which produces motion vectors lo-cally similar to a coaxis or copoint field. This is in-deed the case, as can be verified by viewing figure 15.Also, some viewers report very weak, localized move-ment within the sinusoidal bands of figure 10, par-ticularly in the parts of the bands which are nearlystraight.



(a)

(b)

Figure 14. Replacing the regular radial lines with randomline segments does not affect the rotary motion in (a) butcauses the MacKay effect to disappear in (b).

5. CONCLUSIONS

A computational explanation of the illusory move-ment experienced when viewing the Enigma figurehas been presented using a model for visual mo-tion interpretation developed earlier (Fermuller 1995;Fermuller & Aloimonos 1995a, b, 1997). This model isconsistent with all available neurobiological evidence.The underlying idea is that patterns of retinal motionmeasurements are formed from the visual flow field;these patterns encode three-dimensional informationin a simple manner. The structure of one of thesepatterns is related to the structure of the Enigmafigure. The principle of this relationship was used to

Figure 15. The spiral pattern is locally similar to an illu-sory pattern due to a coaxis field. A weak perception ofmovement occurs locally.

create, based on other patterns, new figures that alsoexhibit illusory movement.

Regarding the debate taking place (Gregory 1995;Zeki 1994, 1995) over whether the illusion is dueto only high-level processes or only optical causes,our analysis suggests that weak retinal motion sig-nals due to optical causes constitute the triggeringof the illusion; however, the major reasons for thisillusory movement are high-level processes accessingthe retinal motion measurements for the purpose ofthree-dimensional motion interpretation. We agreewith Zeki that examples like this, where the brainappears to go beyond the information given, can giveinsight into the processes of visual perception.

For example, the computational models proposedin this paper suggest a number of neurobiological ex-periments aimed at unravelling the functional archi-tecture of the motion pathway with regard to theinterpretation of three-dimensional information fromvisual motion. Our analysis suggests a working modelfor the functional architecture of the motion path-way which, from a computational standpoint, is opti-mal. The model explains processes involved in three-dimensional motion estimation. At this time, though,the interplay between three-dimensional motion es-timation, segmentation, and also shape and colourprocessing, is understood only at a high level. Webelieve that it is up to the synergistic efforts of em-pirical and computational scientists to unravel themysteries of these computational processes. In ouropinion, a good first step in that direction should bea hunt for neurones in the motion pathway that rep-resent parts of some or all of the coaxis and copointmotion-sensitive orientation fields.

Special thanks to Sara Larson for her editorial and graph-ics assistance. The support of the Office of Naval Re-search under Grant N00014-96-1-0587 is gratefully ac-knowledged.



REFERENCES

Adelson, E. & Bergen, J. 1985 Spatiotemporal energymodels for the perception of motion. J. Opt. Soc. Am.A 2, 284–299.

Aloimonos, Y. (ed.) 1993 Active perception. Advances incomputer vision. Hillsdale, NJ: Lawrence Erlbaum.

Andersen, R., Snowden, R., Treue, S. & Graziano, M.1990 Hierarchical processing of motion in the visualcortex of monkey. In The brain (Cold Spring HarborSymp. on Quantitative Biology), vol. 55, pp. 741–748.Cold Spring Harbor Laboratory Press.

Duffy, C. & Wurtz, R. 1991a Sensitivity of MST neu-rons to optical flow stimuli. I. A continuum of responseselectivity to large field stimuli. J. Neurophysiol. 65,1329–1345.

Duffy, C. & Wurtz, R. 1991b Sensitivity of MST neuronsto optical flow stimuli. II. Mechanisms of response se-lectivity revealed by small field stimuli. J. Neurophys-iol. 65, 1346–1359.

Fermuller, C. 1995 Passive navigation as a pattern recog-nition problem. Int. Jl Computer Vision 14, 147–158.

Fermuller, C. & Aloimonos, Y. 1995a Direct perception ofthree-dimensional motion from patterns of visual mo-tion. Science 270, 1973–1976.

Fermuller, C. & Aloimonos, Y. 1995b Qualitative egomo-tion. Int. Jl Computer Vision 15, 7–29.

Fermuller, C. & Aloimonos, Y. 1997 On the geometryof visual correspondence. Int. Jl Computer Vision. (Inthe press.)

Gabor, D. 1946 Theory of communication. J. IEE ’93,part III, 429–457.

Gallant, J., Braun, J. & Essen, D. C. V. 1993 Selec-tivity for polar, hyperbolic, and Cartesian gratings inmacaque visual cortex. Science 259, 100–103.

Gregory, R. L. 1993 A comment: MacKay rays shimmerdue to accommodation changes. Proc. R. Soc. Lond.B 253, 123.

Gregory, R. L. 1995 Brain-created visual motion: an illu-sion? Proc. R. Soc. Lond. B 260, 167–168.

Grzywacz, N., Watamaniuk, S. & McKee, S. 1995 Tem-poral coherence theory for the detection and measure-ment of visual motion. Vision Res. 35, 3183–3203.

Lagae, L. 1991 A neurophysiological study of optic flowanalysis in the monkey brain. Ph.D. thesis, Leuven,Belgium.

Leviant, I. 1996 Does ‘brain-power’ make Enigma spin?Proc. R. Soc. Lond. B 263, 997–1001.

MacKay, D. 1957 Moving visual images produced by reg-ular stationary patterns. Nature 180, 849–850.

MacKay, D. 1958 Moving visual images produced by reg-ular stationary patterns. Nature 181, 362–363.

MacKay, D. 1961 Interactive processes in visual percep-tion. In Sensory communication (ed. W. Rosenblith),pp. 339–355. Cambridge, MA: MIT Press.

Mon-Williams, M. & Wann, J. 1996 An illusion thatavoids focus. Proc. R. Soc. Lond. B 263, 573–578.

Movshon, A. 1990 Visual processing of moving images. InImages and understanding (ed. H. Barlow, C. Blake-more & M. Weston-Smith), pp. 122–137. CambridgeUniversity Press.

Orban, G., Lagae, L., Verri, A., Raiguel, S., Xiao, D.,Maes, H. & Torre, V. 1992 First order analysis of opti-cal flow in monkey brain. Proc. Natn. Acad. Sci. USA89, 2595–2599.

Reichardt, W. 1961 Autocorrelation, a principle for evalu-ation of sensory information by the central nervous sys-tem. In Sensory communication (ed. W. Rosenblith),pp. 303–317. Cambridge, MA: MIT Press.

Reichardt, W. 1987 Evaluation of optical motion infor-mation by movement detectors. J. Comp. Physiol. 161,533–547.

Snippe, H. & Koenderink, J. 1994 Extraction of opticalvelocity by use of multi-input Reichardt detectors. J.Opt. Soc. Am. A 11, 1222–1236.

Tanaka, K. & Saito, H.-A. 1989 Analysis of motion of thevisual field by direction, expansion/contraction, androtation cells illustrated in the dorsal part of the me-dial superior temporal area of the macaque monkey. J.Neurophysiol. 62, 626–641.

van Santen, J. & Sperling, G. 1984 Temporal covariancemodel of human motion perception. J. Opt. Soc. Am.A 1, 451–473.

Zeki, S. 1993 A vision of the brain. Oxford: Blackwell.Zeki, S. 1994 The cortical Enigma: a reply to Professor

Gregory. Proc. R. Soc. Lond. B 257, 243–245.Zeki, S. 1995 Phenomenal motion seen through artificial

intra-ocular lenses. Proc. R. Soc. Lond. B 260, 165–166.

Zeki, S., Watson, J. D. G. & Frackowiak, R. S. J. 1993Going beyond the information given: the relation ofillusory visual motion to brain activity. Proc. R. Soc.Lond. B 252, 215–222.

Received 20 January 1997; accepted 12 February 1997


Documents

Families of stationary patterns producing illusory ... Thus, any system has to interpret the rigid motion encoded in the sequence of images perceived by its retinae. As a system moves