BioI. Cybernetics 19,39-54 (1975)@ by Springer-Verlag (1975)

The Orientation of Flies Towards Visual Patterns:

On the Search for the Underlying Functional InteractionsGad Geiger and Tomaso Poggio

Max-Planck-Institut fUr biologische Kybernetik, Ttibingen, FRO

Received: October 17, 1974

Abstract

The average optomotor response of insects to a given visualstimulus (measured in open-loop conditions) can be decomposedinto a direction sensitive and a direction insensitive component.This decomposition is conceptual and always possible. The directionsensitive optomotor response represents the "classical" optomotorreflex, already studied in previous investigations; the directioninsensitive optomotor response is strictly connected to the orienta-tion and tracking behaviour (see the work of Reichardt and co-workers). Thus a characterization of the direction insensitiveresponse is useful in clarifying the nervous mechanisms under-lying the orientation behaviour. For this reason we study in thispaper the direction insensitive optomotor (torque) response of fixedflying flies Musca domestica. Periodic gratings, either moving orflickering, represent our main stimulus, since the dependence ofthe fly response on the spatial wavelength can unravel the presenceand properties of the underlying lateral interactions. In this con-nection an extension of the Volterra series formalism to multi-input (nervous) networks is first outlined in order to connect our(behavioural) input-output data with the interactive structure of thenetwork. A number of results concerning, for instance, the responseof such networks to flickered and moving gratings are derived;they are not restricted to our behavioural results and may be relevantin other fields of neuroscience.

These theoretical considerations provide the logical frameworkof our experimental investigation. The main results are:

a) the direction insensitive optomotor response depends onthe spatial frequency of a moving grating, implying the existence of(nonlinear) lateral interactions,

b) its wavelength dependence changes with age, unlike thedirection sensitive response,

c) both the direction insensitive response and the (closed loop)orientation behaviour are present only in the lower part of the eye;on the other hand the direction sensitive response is present inevery part of the two eyes.

Furthermore the attraction towards a flickered periodicgrating shows, as theoretically expected, a wavelength-dependencesimilar to that of the direction insensitive response, again presentonly in the lower part of the eye. The interactions which affectthe orientation response are selective with respect to the spatio-temporal mapping of the pattern onto the receptor array. It isconjectured that these interactions are the basic mechanismsunderlying spontaneous pattern discrimination in flies. Theirpossible organization is further discussed in terms of our formalism.Moreover our data suggest that two specific nervous circuitriescorrespond to our conceptual decomposition of the optomotorresponse.

1. Introduction

The question of how visual motion is evaluatedby insects is essential and preliminary in understandinghow visual information is processed by insect nervoussystems. Reactions to relative movement are notonly critical for the insect's behaviour: being also avery stable and common reflex, they represent agood opportunity for trying to relate a precise functionwith the underlying nervous mechanisms. Moreover,recently it became clear that movement perceptionis also strictly connected to the visual orientation andtracking behaviour of flies (Reichardt, 1973; Poggioand Reichardt, 1973a; Poggio, 1973; Virsik and Rei-chardt, 1974), so adding to the problem a new di-mension; namely, an opening towards the questions of,spontaneous pattern preference and perhaps patternrecognition. In flies the stabilized retinal image of asmall object does not elicit any average attraction;movement or flicker is necessary to mediate positioninformation (Reichardt, 1973; Pick, 1974a).Since eachreceptor in the compound eye of the fly transduces alocal light intensity, evaluation of visual stimuli isgiven by one or more interacting (at the nervous level)inputs: at least two inputs are required for selectivemotion detection. The visual surrounding is imagedon a 2 dimensional array of receptors which are theinputs to a (nonlinear) interactive network.

A general characterization of such many inputnetworks has been recently introduced with the aidof the Volterra series formalism and applied to thefly's visual system (Poggio and Reichardt, 1973b;Poggio, 1974a; Marmarelis and McCann, 1973).Within this treatment, see Chapter 2.1, it is especiallyclear that the average output, either behavioural orelectrophysiological, of such a many input networkcan always be decomposed into a component whichis direction sensitive(Yd.) and a component which isdirection insensitive (Ydi)'In the following we w~llbemainly interested in the optomotor response of flies,

40

defined as the behavioural motor reaction (in ourcase, torque) to visual stimuli (see Poggio, 1973)."Direction sensitive" is defined as the part of theaverage optomotor response which reverses sign underthe operation of inverting the direction of motion ofa given pattern, "direction insensitive" is the partwhich is invariant under the same operation. Ofcourse this definition of optomotor response, unlikeprevious definitions, is not restricted to the responseto moving stimuli, but also includes response to flickerand stationary patterns. For instance, the termmovement as opposed to flicker can be misleading,since its definition is dependent on the moving patternitself and on the receptor organization. Clearly a net-work (like the Chlorophanus model, Reichardt, 1961)which is only direction sensitive does not respond,in the average, to flicker. On the other hand a networkwhich is direction insensitive (for instance with nolateral interactions between the inputs) may show anaverage reaction to flicker as well as to movement.

From the work of Reichardt (1973) it is clear thatthe direction insensitive part of the average reaction isessential for the orientation and tracking behaviour offlies towards visual objects. A phenomenologicalapproach (Poggio and Reichardt, 1973a) has reducedthe fixation behaviour of flies to their open loopreaction (position dependent) towards an object.From the knowledge of the "attraction" profile of agiven pattern, a phenomenological equation allowsone to predict in stochastic terms the associatedorientation behaviour of the fly for tracking as wellas chasing (Poggio and Reichardt, 1973a; Reichardtand Poggio, 1974; Land and Collett, 1974). Only thedirection insensitive optomotor reaction can providethe position dependent "attractiveness" profile Delp)(Reichardt, 1973), which underlies the orientationbehaviour of flies towards small objects. The directionsensitive part of the average output must be due tononlinear interactions between input channels; thedirection insensitive part may arise from interactingas well as from non interacting input channels.Actually in terms of the Volterra formalism (Chapter 2)it is possible to characterize precisely the interactionswhich are responsible for the direction sensitive andthose responsible for the direction insensitive response.Under some simple conditions the selfkernels and thesymmetrical parts of the kernels of every order (seeChapter 2) give the direction insensitive optomotorreaction. The antisymmetric component of the kernelsgive the direction sensitive optomotor response. Thelatter has been studied in much detail and a largeamount of data can be usefully interpreted throughthe Volterra formalism, leading to some general

. conclusions concerning the interactive organizationof the system (Poggio and Reichardt, 1973b; Poggio,1973). On the other hand comparably little is knownabout the direction insensitive optomotor reaction:it is the purpose of the present work to characterizesome functional properties of the interactions under-lying the orientation behaviour. Some experiments(Pick, 1974a) suggest that average position informa-tion (direction insensitive response) for narrowobjectsis conveyed by direct channels without need oflateral interactions. A number of questions arises: forinstance, on a broader range do interactions affect theorientation response? Suggestions in this directionsare given by the limitations of the "superpositionprinciple" (Reichardt and Poggio, 1974),and supportedby other experimental evidence (Virsik and Reichardt,1974; Pick, 1974b; Geiger, 1974).Clearly, interactionsaffecting the direction insensitive response may pro-vide, as it will be discussed later, a powerful mechanismfor spontaneous pattern preference. Moreover, doesthe conceptual separation into symmetric interactionsand antisymmetric ones correspond to two physiologi-cally distinct systems? The fixation behaviour is knownto take place mainly in the lower part of eye (Reichardt,1973); does this property hold true also for thedirection insensitive response measured here? Andwhat is the connection between the reaction to amoving and a flickered pattern?

. We will now attempt to answer these and connectedquestions, by means of an experimental characteriza-tion of the direction insensitive optomotor reaction toperiodic patterns.

Before introducing the experimental results we willsummarize, for completness, the theoretical back-ground of this work, which is the Volterra formalismfor many input-systems. Its full comprehension isnot necessary for the understanding of our mainresults. The formalism is quite general and may beusefully applied to a number of topics in the neuro-sciences: in this sense this work can also be consideredas one instance of such an application.

2. The TheoreticalBackground1

The Volterra series formalism which will be out-lined here is an approximate description of non-linear many input networks; it provides a canonicaldescription of the nonlinear interactions of the net-

1 This part extends some results presented in earlier papers(Poggio and Reichardt, 1973b; Poggio, 1973, 1974a), where therelevant bibliography relative to Volterra systems is also cited.A more complete version of the theory summarized here is inpreparation (Poggio, 1974c).

work. Various types of decomposition of the Volterraseries facilitate the interpretation of the interactionsand connect them with functional properties of theinput-output map. The various interaction terms canbe represented by simple graphs; they are a usefulaid in interpreting the mathematics. Symmetry prop-erties of the Volterra kernels associated to the variousinteractions can be related to functional propertiesof the network: in this way functional symmetriesmay be reduced to structural properties. The parallelprocessing implemented by such nonlinear networksis capable of selective features extractions and can beessentially richer than in the linear case. The formalismmay describe (many inputs) nervous networks eitherto characterize the underlying interactions throughinput-output experiments or to predict properties ofthe (cooperative) network response to arbitrary spatio-temporal stimuli, if information about the interactionsis available. A number of general properties likesuperposition, phase invariance, equivalence of flick-ered and moving periodic patterns can be usefullyinterpreted in terms of the Volterra formalism. Inaddition to the behavioural data of this paper manyother neurophysiological data may be easily sum-marized and characterized through this approach:the visual system of vertebrates, some neurophysio-logical aspects of auditory pattern recognition, modelsof the eye movement system2 offer a wealth of op-portunities. In all these cases the many inputsformalism presented here seems appropriate to dealwith the distributed spatio-temporal nature of thedata-processing performed by nervous structures.

2.1. The Volterra Series for n-Input Systems

The input-output relation of an-input n-outputnetwork which admits a Volterra representation isgiven by

00 j

Yk(t) = gt + I I n Xi. *j gL..i}'j=1 ib...i} r=1

where */ is defined as

Xl'" X/ */ g1.../

= s... Sxdt-'1)...X/(t-,/)gl...z(r1...,/)d'1...d,/.Equation (1) is a straightforward generalization of theconvolution.

2 Clear nonlinearities of the oculomotor system (St. Cyr andFender, 1969) should be embodied in a many input modelrecognizing the fact that neural processing takes place in spacenot just in time (Robinson, 1973). In this sense the formalismintroduced here seems a promising approach.

41

Of course the series can be extended to describe time-dependent

systems: in this case, however, the time integrals are not any moregeneralised convolutions. On the other hand Eq. (I) can be restrictedto time and space invariant systems, giving

y(r) = go+ J f(r- r')g1(r')dr' + H f(r-r') fIr -r")

g2(r',r")dr'dr"+ ...,(3)

where r is the vector r = (~). This extends to nonlinear spatial and

temporal interactions a linear approach already used in analysis ofbiological systems (Seelen and Reinig, 1972; Knight, 1973).

(1)

2.2. Decompositions and Graphs

Equation (1) (or its non stationary extension)suggests an immediate conceptual decomposition:with respect to nonlinearities of the j-th order an-input Volterra system can always be decomposed

into the linear sum of (;)j-input systems. This decom-position allows useful simplifications; the interpreta-tion of the various interaction terms can be facilitatedby the use of obvious diagrams. For instance Fig. 1arepresents a second order interaction between i and jto k and Fig. 1b represents a second order selfkernel.The decomposition stated before can be representedfor second order nonlinearities as in Fig. 2. Charac-teristic properties can be associated to interactionsof a given order. Second order interactions, forinstance, have a kind of superposition property inthe average: if more Fourier components are presentin the time input(s), the average response of a secondorder network is the sum of the average responsesto each component, separately [see Eq. (16)]. Thisgeneral property underlies a number of remarkablefeatures of the average output of second order net-works: for instance the "phase invariance" property(see Poggi!) and Reichardt, 1973b) and the pro-portionality between average response to a flickeredand a moving periodic pattern (see later).

On the other hand two input networks show anaverage response to two sinusoidal inputs whichdepends on the relative phase in a way which ischaracteristic for the order of nonlinearity 3 allowingan experimental characterization. Another generalkind of decomposition of the interaction terms ofEg. (1) is possible: a second order interaction betweentwo inputs, for example, can be always approximatedby a series of products of linear transformations of the

(2)

3 If the relative phase is J 1p,the temporal frequency ())oand theorder of nonlinearity of the network 2n* the formula is

n'y= I kn(wo)cosnJ 1p+ hn(wo)sinnJ 1p+ ko(wo).

42

a l' bFig. Ia and b

IV= v..J(" + 1V" +1 1 1 j

Fig. 2

inputs. The property can be extended to higher ordernonlinearities for both the time and the space domain(Poggio, 1974c).

A decomposition of this kind is interesting for problems ofoptimal nonlinear estimation; for instance it was used to obtain anonlinear optimal algorithm of associative memory (Poggio,I974b), extending results obtained for the linear case (Kohonenand Ruohonen, 1973).Furthermore through such a decompositionthe processing and "feature extraction" properties of a nonlinearinteractive network can be characterized (Poggio, 1974a).

Other, more specific decompositions of the termsof the Volterra series Eq. (1) are also possible, either(a) with respect to sOme invariance (or symmetry)properties of the interactions or (b) with respect totheir eventual sequential structure. For instance (a)one can always decompose a cross-kernel into asymmetric gS(01...0n)=gS(On...01) and an antisym-metric part g"(01'" on)= - g"(on... 01) as graphicallyrepresented in Fig. 3 for the second order case.Dynamic properties of the output can be connectedto symmetry properties of the interactions: as anexample an antisymmetric, two inputs, second ordernetwork has a constant response to inputs consistingof a single Fourier component, independently fromtheir phase relation. The output varies with time if andonly if the interaction kernel contains a symmetricpart (gH01' 02)= g2(02' 01»)'

In the Case (b) an n-th order interaction may arisefrom a sequence of lower order interactions: for

- +

rII

1 1, ,;,Fig. 3

l,"n"..",;,

a

TI TI1/1

fj1

fP1

b c

Fig. 4a-c

instance a fourth order term may be described by oneof the sequential decompositions of Fig. 4. Differentsequences of nonlinearities correspond to differentproperties of the input-output map. Depending onthe specific case and on the amount of priori informa-tion available it is often possible to discriminatebetween alternative structures without actually meas-uring the kernels of the system.

2.3. Visual Patterns and n-Inputs Volterra Systems

In the case of a (I-dimensional) network having photoreceptorinput, the transduction of the visual pattern onto the photoreceptorlayer has to be described, taking into account the angular sensitivitydistribution of the receptors. Considering the network as time-invariant (but not space-invariant) we have

00 j

Yk(t)=h~+ L J".JU'(~I"'~j;t-1:1...t-1:)n f(~,,1:,)j~1 ,~I

(5)

d~I...d~jd1:I...d1:j'

where f(~, 1:) represents the light intensity distribution (around asuitable mean level) of the stimulus. In the above assumptions the

0 can be written asU'(~I...~j;1:I...1:)= L ei,(~I)...eij(O

i,...ij(6)

gt...i,(1:1, ... 1:),

where g are the Volterra kernels of Eq. (1) and edx) is the angularsensitivity distribution of receptor k, with respect to a suitablecoordinate system. For instance

u~ (~1' ~2; 1:1,1:2) = L ei(~I) ej(~2) g~j(1:1' 1:2) .ij

(7)

This is equivalent to use Eq. (1) with

Xi(t) = Jei(S)f(s, t) ds .If the time dependence of the stimulus is due to the (rigid) motion~(t) of a spatial pattern f(x) we may write

Xi(t) = J ei(S)f(s - Wi) ds = (£Ii@ f)~(t) .

On the other hand if f(s, t) = f(s) a(t) we obtain

Xi(t) = a(t)(ei@f)~~O'

Assuming that the Xi(t) can be written as Fourier series

(With a basic frequency w* = ~:), we obtain

withXi(t) = :E bi,n einro"

T'1 2J . .b. = - x.(t) e-.nro'dt .

',. T* T' '-T

We want to evaluate the coefficients bi,.. We define, for q = 2; ,

1 .Ri(q) = - J e-.qx J ei(S)f(s-x)dsdx

21t

which becomes, if f(x) is even,

Ri(q) = Ili(q) j(q) = Ilo(q) j(q) e-iq~i = Ro(q) e-iq~t ,

assuming that the angular sensitivity distributions of the receptorsare the same (£Ii(x) = eo (x -1J.'i))'

For rigid motions of the pattern f(x) it is convenient to defineT'

1 21;.(q)= - J dt ei(q~(t)-.ro")T* .T'

-T

On the other hand for modulation of a stationary pattern [seeEq. (10)] the definition is

T'1 -

J2 ..1;.= T* T' dt e-'.'" 'a(t).2

In both cases the coefficients bi.. can be written as

bi,. = J 1;.(q) Ri(q) dq = J 1;.(q) Ro(q) e-iq~, dqor

bi.. = b.(1J.'i)= rl {1;.(q)Ro(q)}.

Thus Eqs. (11) and (15) describe in general the trans-duction between the visual pattern and the lightintensity inputs into the channels of the network.The application of the basic Volterra series Eq. (1) isthen straightforward: for instance the average outputYk is given by

n

Yk= g~ + LGt(O) bi,o + I L Gtj(qw*, - qw*)i ij q

n

. b;,qbj,_q + LL Gtjh[qw*, pw*, (- p - q) w*]ijh pq

.bi,qbj,pbh,_p-q+'" .

43

(8)

Inspection ofEq. (16)shows that quadratic interactions[the third term in the right hand of Eq. (16)] satisfythe property of "superposition in the average":Fourier components in the inputs do not interferein their average contribution.

(9)

(10) 2.4. Moving Periodic Patterns.Direction Sensitive and Direction Insensitive Response

A widely used input stimulus is a moving periodicgrating. Through the Volterra formalism it is possibleto connect the output of the system to its interactionorganization. The average output can be alwaysdecomposed into a direction sensitive and a directioninsensitive components which in turn can be connected,under specific conditions, to characteristic parts of theinteractions. In the following we will be mainlyconcerned with the average response of the network.

(11)

(12)If a pattern is movin g at constant speed (~(t)= wt), Eq. (9)

represents a linear transformation on the time function f(wt).In fact from Eq. (15) one obtains

(13) b. =R (nw*

)'.' i W-' (17)If the pattern f(x) contains a single spatial Fourier component of

spatial period qo = ~1t, Eq. (17) becomes0

(14a)-in "'" ~,

(wo)bi.. = .5..:1:1e W Ilo --;- , (18)

(14b)

with w* = wo = qow = ~ w, apart from constant factors depending..to

on the contrast of the pattern. From this equation it is easy toderive the average output of a Volterra system for a spatial periodicpattern moving at constant speed. Calling LIqJ the maximumcommon divisor of the angular spacing between interacting receptorsone obtains

N

(. 2.~" . 2'~"

)f= Ln P.(wo)e-'.~ +P.(-wo)e'.~ ,0(ISa) (19)(ISb)

where N depends on the geometry of the interactions and on thedegree of nonlinearity of the system.The functions P.(wo) arederived from the kernels G and depend on the effective contrastof the pattern, which is a function of its actual contrast, of theangular sensitivity distribution of the receptors and of ..to. Sincethey can be considered as the Fourier transform of a real timefunction, Eq. (19) can be rewritten as

~ N 21tLlqJ N 21tLlqJ

Y= L. h:(wo) sinn ~ + L. k:(wo)cosn ,-- + k~(wo),(20)1 0 1 Jl,o

where h:(wo) is the imaginary (and odd) part of 2P.(wo) and k:(wo)is the real (and even) part of 2P.(wo)' With respect to a movingpattern it is useful to define the operator of "direction inversion"DI and the quantities

(16)

Ydi = HV+DIY)=W+ y),

Yd>= HV-DIY) = HY- y),

(21a)

(21b)

44

which represent the direction insensitive and the direction sensitivepart of the output of the network. This decomposition is alwayspossible and leads to:

ji = Ydi+ Yd., (22a)

ji = Ydi- Yd,' (22b)

In the case of a symmetric pattern moving at constant angularvelocity, the time signals transduced by the receptors are al1 equal,apart from time translations. In their case the operator Dl simplyinverts the signs of the time shifts; in Eq. (19) the operator Dlchanges the sign of the phases giving

- ;, h . 2nAcpYd,=t :(wo)smn-I. AO

-;, ) 2nAcp k*(Ydi=t k:(wo cosn-+ 0 wo).I. AO

If the inputs (in 1 dimension!) are equal1y spaced the operator Dlgives, for arbitrarily moving (symmetric) patterns

Dl {xd'l) ... x.(-r.)} = Xn('I) ... XI('.) ,

coinciding therefore with the operation (on the network) of "in-verting" the inputs. The direction sensitive output is in this case dueto the antisymmetric part of Eq. (1).

In the various specific cases (see for instancePick, 1974b) it is possible to connect the directionsensitive and the direction insensitive response tosymmetry properties ofthe kernels ofEq. (1).Howevermuch care is needed in connecting operations onpatterns with symmetry properties of the interactions(reflecting operations on the network); moreover fora two dimensional network the problem is rathercomplex (Poggio, 1974c).

When a pattern moves at constant speed the con-nection with a term of order s, whose s time functionsare derived from equally spaced inputs, is the following:its average direction sensitive contribution is due tothe antisymmetric component Ga, the direction in-sensitive to the symmetric component GSof the kernel,where

Gf,...i,(W), ..., ws)= - Gt..i.(Ws, ..., WI)

Gfl...i.(W), ""ws)= Gfl...i.(W.. ...,w)).

Therefore the function h*(wo) and k*(wo) of Eq. (23)are derived from Ga and GS, respectively; k~ is con-nected to the selfkernels and to symmetrical parts ofthe crosskernels GS.

If the interacting inputs are not equal1y spaced one mayintroduce fictitious inputs and define corresponding "degeneratekernels", in which the number of variables is greater than the orderof the interaction. Then, with respect to the "degenerate kernels",the same property stated before holds, and also in this case it ispossible to associate the direction insensitive response to a precisepart of the interactions (see Poggio, 1974c).

Only the symmetrical parts of the kernels give an averageresponse to an oscillating, symmetrical pattern, if the network is10cal1y homogeneous. In the following we will assume that thiscondition is actually satisfied. If the pattern is not symmetric

(23a)

difficulties may arise; moreover when either the amplitudes ofoscillations or the pattern dimensions are not small, local homo-geneity becomes a very restrictive condition 4.

To analyze the implications of Eq. (23), we assume that Yd.and Ydiare corrected for the Adependent reduction of the effectivecontrast. Under this condition the new "reduced kernels" hn and k.do not depend on A.The responses Yd. and Ydican be conveniently

.f

.f

1rewntten as unctIOns 0 p = ;:

N

Yd.(P)= I.h.(wo)sinnp(2nAcp),IN

Ydi(P) = I. k.(wo) cosn p(2n A cp)+ ko(wo) .0

Equations (26a) and (26b) are respectively the Fourier series of

an odd and an even function of p, with a basic period ~. InAcp

Eq. (26a) the first zero crossing (for p increasing from zero) takes

place for p;;; 2~ cp , implying that the first zero crossing for de-creasing A (from A = 00) must take place for A;;; 2L1cp. In Eq. (26b)

Ydi, as a function of p, is even around p = ~ and has a basic2Acp ..

d1 f

- d - . df . f 1peno -. I Ydian Yd. are consldere unctIons 0 both p = -Acp A

and Wo, the factorization property

(26a)

(26b)

(2=b)

(24)

Yd.(P, wo) = l(p) C(wo)

holds if and only if the "reduced kernels" h. satisfy to

(27)

h.(wo) = h. C(wo) . (28)

2.5. Flicker of Spatial Periodic Patterns

An obvious question at that point is the linkbetween moving patterns and flickered patterns.It is clear that if a system is linear the problem can beeasily answered, since the superposition propertyholds and each pattern can be represented as super-position of running waves. Clearly for nonlinearinteractions the problem is more complex, also withrespect to the average output.

In the following we consider the average output of a manyinput network to the flicker of a periodic pattern as given by

(25) f(t) = 10+ coswo t cosqox (29)

around the average intensity 10' Equations (14b), (15), (16), al1owthe calculation of the average response in the general case; forinstance nonlinearities of the second order give

Yflicke

which is different from Eq. (30). The term cOSqO(1/!i+ 1/!) in Eq. (30)codes the absolute spatial phase of the grating; however if the term

qO(1/!i+ 1/!) in Eq. (30) eventually assumes all phase values (this istrue if many homogeneously distributed second order networks arepresent), an average over the phases gives

£P2' £P3, are the phases at the inputs I, 2, 3 relative to

input 4 (depending on relative spatial positions of the inputs) and

G';' = G4(wo, - Wo, - Wo, wo); G~= G4(wo, - Wo, + Wo, - wo);

G:; = G4(wo,Wo, - Wo, - wo).

On the other hand

46

of the relative angular distance between interactingreceptors. In this case the concept of "receptive field"may have stilL a meaning: knowing the averageresponse of a two inputs second order network to twotime inputs it is possible to predict its average responseto an arbitrary spatial pattern. For higher order ofnonlinearity this correspondence does not hold anymore, since the n-th coefficient ofEq. (28b) is not aloneassociated to interactions between inputs spacedby nLJcpoTo predict the response ofa fourth order net-work to an arbitrary stimulus, experiments withfour inputs are, in general, necessary. On the other handselfkernels (linear and nonlinear) are completelycharacterized by one input experiments. Therefore itis clear that the concept of "receptive field" and itsmeaning are actually dependent on the type ofunderlying interactions between inputs. Interpreta-tions of input-output experiments on a many inputssystem are deeply affected by the presence of super-position properties. If a network is linear its dynamicresponse to an arbitrary spatio-temporal stimuluscan be obtained from the superposition of responsesto spatial components of the stimulus. In a similarway the average output of a second order networkis the sum of the average responses to the Fouriercomponents of the time inputs, as discussed earlier.Still in another sense the average output of a manyinputs i-output Volterra network with arbitrarynonlinearities but without nonlinear lateral inter-actions (self-kernels only) is the superposition of thesingle channels: the response to a given spatialpattern can be obtained as the sum of the responsesto parts of the pattern. On the other hand whenevercrosskernels exist the average response of a spatiallydistributed network is always context dependentand spatial decomposition of a pattern into "ele-mentary" components is not easily possible. In otherwords, the average response to a spatially localized"feature" (like a bar or a dot) may also depend on thewhole pattern 5 in which the "feature" is embedded.In a similar way if the order of nonlinearity is higherthan 2, interactions between different temporal fre-quencies affect the average response: single frequenciesmeasurements cannot predict anymore the dc-com-ponent of the output for arbitrary inputs. In conclusionthe meaning of the concept of receptive field dependsstrongly on the type of underlying interactions: of

5 An interesting problem concerns the possible decompositionof an input pattern in a number of "features" (not necessarilyspatially localized) on which the specific network operates in-dependently; although the answer is easy in the linear case, nogeneral results are yet available about the nonlinear interactionsconsidered here (see also Poggio, 1974a).

of course, the Volterra formalism can provide thenecessary theoretical language to deal with the variousspecific cases.

3. Experimental Results

In the preceeding chapter we have introduced thetheoretical considerations which provide the logicalframework of our investigation. The decompositionof the optomotor response into a direction sensitiveand a direction insensitive component has beenprecisely linked to underlying interactions and totheir symmetry properties; moreover, some importantinput conditions (moving periodic gratings, flickeredpa tterns ...) have been theoretically analyzed.

With this general background we present in thefollowing our experimental approach to a functionalcharacterization of the interactions underlying theorientati'on behaviour. The question about the ex-istence of nonlinear interactions affecting the directioninsensitive response will be solved, using the resultsof Chapter 2.4; the localization of the two componentsof the response either in the lower or in the upper partof the eye will also receive a simple experimentalanswer. Moreover an (unexpected) age dependenceof the direction insensitive component will suggestthe existence of 2 separate nervous networks corre-sponding to the conceptual decomposition into a"movement" response and an "orientation" response.

3.1. Methods

In order to achieve symmetrical stimulation, avoiding zerobias dependence, an experimental set-up was constructed as shownschematically in Fig. 5. This apparatus was made of three blocksof aligned bundle, bi-concave, cylindrical fibre optics. It enablesus to expose stimuli - independent of each other - to different partsof the fly's visual fields.

In this study the portions of the fly's visual field used were:tp= [ +45°, + 135°J,[ -45°, -135°J and various[}sectors(windowheights), which were limited to [}= [ + 50°, - 500J; where 1p is theangle between the fly's direction of flight and a point in the planeperpendicular to the fly's vertical axis, and [}is the angle between apoint in space and the equator of the fly's eye (the line of symmetrydividing the upper and lower half of the eye).

The stimulus was moved on the exterior of the fibre opticblock and was projected without distortion onto its inner surface(see Kapany, Eyer, and Keirn, 1957),by a disc mounted on a servo-motor. A slight difference in the pattern contrast within each fibreoptic block, i.e. between the wide and the narrow portions of theblocks, resulted in a deviation oflight flux of 11%.This experimentalset-up was homogeneously illuminated from the outside with abrightness of approximately 13ooApostilb (414 cd/m2). The lossof light flux caused by the absorption of the fibre optics and theopalescent films (with Lambert properties) facing them was ap-proximately 25% in the positions tp= 0°, 90°, -90°; and approxi-mately 36 % in the positions tp= 40°, 50°, 130°.

..0'

Opalesee ot film

a

Torq ueCompo

]

Illumination

[

Stimulus

Motor Motor

b

Fig. Sa and b. Diagram of the experimental set-up, (a) seen fromabove, (b) from behind. For details see text

Wild-type red-eyed, female Musca domestica from laboratorystocks were used as test flies. A fly with its head fIxed to the thoraxwas suspended from a torque compensator at the axis of the innercylindrical structure of the three fIbre optics blocks. The flyingfly was fIxed in a manner which made translatory and rotatorymovements impossible. The torque produced by the fly around itsvertical axis was transduced by the torque compensator into anelectric signal (voltage) which was then evaluated. For a full de-scription of the torque compensator, see Fermi and Reichardt(1963) and Gotz (1964).

Square periodical patterns were used as stimuli and were movedeither sequentially or simultaneously on both sides of the fly. ~Each measurement for a given stimulus lasted at least 1 min. and

was started at least 10sec after beginning of the stimulus movementin order to decrease influence of the previous stimulus (Geiger,1974). Torque histograms were made from each measurement.The integral of the histogram provided the mean value of the torque.This value was measured for a set of flies and averaged again.The errors indicated in the fIgures are the standard deviations ofthe mean.

A number of periodical patterns were used with the wave-lengths: }.= 2; 2,75; 3; 3.25; 4; 5.2; 6; 7,5; 9; 30 degrees for movingpatterns and}, = 3,5; 6; 9; 30 degrees for the flicker experiments.The patterns had a contrast modulation of90% for moving patternsand 74% for flickered patterns, measured in the positions tp= :t90°.In all the experiments the full width of the windows was exposedto the fly, but only a small pQrtion of its height (Ll8= 20°)in orderto minimize the}. distortions due to cyclindrical symmetry of theexperimental set-up. Manipulating the fly,prior to the experimenta-tion, to 8 = - 5° or 8 = = 5° was made with an error of :t 5°.

3.2. The Optomotor Response to Pattern Movement

To characterize the" direction insensitive com-ponent of the average optomotor response Ydiwe havemeasured its dependence on the spatial period A of asquare wave grating moved at constant angularvelocity. This kind of stimuli was extensively used toinvestigate the direction sensitive optomotor responseof flies (see reviews by Reichardt, 1969; Gotz, 1972).Its use here allows a comparison between the twocomponents of the response. Since a A-dependenceof the average response is associated to nonlinearinteractions between input channels (see Chapter 2),the A-portrait of Ydican immediately answer our firstquestion about the presence of lateral interactions.

A lateral sector of the right (left) lower half of theeye (.9=[-5", -25°], 1p=T:t45°, :t135°]) wasstimulated by the moving grating. The direction ofmovement of the stimulus was either progressive(front to back) or regressive (back to front) on oneside of the eye (see Gotz and Wenking, 1973). Theaverage torque response of the fly to a moving gratingof a certain wavelength was measured in four suc-cessive steps: the grating was moved progressively onthe right (1 in Fig. 5a), progressively on the left (2),regressively on the right (3), and regressively on theleft (4).

Depending on the wavelength A., the angular velocity of thepattern was chosen so as to keep constant the temporal frequency

000 = 2:W , the rate of events within the receptive fIeld of the

receptors. With respect to the right eye the direction insensitive Yd;and the direction sensitive Yd. components of the average responseare given by

Yd;=y+)i'

Yd.= y - )i'

[see also Eq. (21)] where y and y- represent the average responseswith sign to progressive and regressive motion of the pattern on the

48

2

a

e...~ 1=~~~=='..~ 0

0 =[-5~-25°]

~' I-//- --~I

II

"~~oA-?r"f1;

3b

2

\

11\ II 1- - //- - -1I

III

~;~

0 = [ -5~-25°]

.0

30A [deg]

Fig. 6a and b. The progressive (8) and the regressive (0) averageoptomotor responses to square wave gratings moving at constantangular velocity are plotted in Fig. 6a. In terms of the directionsshown in Fig. Sa the progressive response is Y;= Hy(l) - y(2»and the regressive one is YR= Hy(4) - y(3». The average torqueresponses y(l) ...y(4) of a flywere measured for each A; the sequencein which y(1)...y(4) were measured was randomly varied. Thedirection sensitive (0) and the direction insensitive (8) componentsof the optomotor response to square wave gratings are plotted inFig.6b. The data are derived from Fig.6a through Yd.= Y;;+ YR.Ydi= Yp - h. In both figures the contrast frequency is W/A= 2 Hz.Only the lower half of the eye is stimulated for 8 = [-5°, -25°J(8 = 0° represents the equator coordinate). Each point representsthe average of between 6 and 26 flies, 10-12 days old; the pointsfor A = 2°. 3°, 4°, 6°, 30° are associated to the same number (26) ofindividuals. The vertical bars represent the standard deviation ofthe mean. Subgroups of the population used for the experiments ofFig. 6 also show the same Adependence. All the points in the direc-tion insensitive response Ydi are significantly positive (p < 0.001)with the exception of A = 2". A= 5.2°. and A= 7.5°. The points atA = 2° and A = 3° do not belong to the same population; the point atA = 6° does not belong to the same population of either A= 7.5°or A = 4° (confidence limit p < 0.001). The measured values of Yd'at A= 4° and at A = 3.75° are not significantly different from zero.An equivalent experiment performed on a smaller group of fliesfor a contrast frequency W/A= 4 Hz showed a very similar A de-

pendence

0 5 10

right eye. In the assumption of mirror symmetry between the twoeyes zero-free quantities for both eyes are obtained as

Ydi = !(Yd; + Y~;)

Yd. = !(Yd.+ Y~.)

-"Progressive" and "regressive" responses are given by

Yp=Hji'- yl) = Yd.+ Ydi

YR= !(ji/- y') = Yd.- Ydi'

The flies Musca domestica used (see Chapter 3.1)in thisand most of the other experiments were 10-12 daysold 6: the reason for this will be discussed later.

Average data are presented in Fig. 6a for the pro-gressive and regressive responses, in Fig. 6b for thedirection insensitive and the direction sensitive com-ponent. The direction sensitive response Yds showsapproximately the usual A dependence (compareEckert, 1973)and will be discussed later. The directioninsensitive response Ydi is in most cases significantlydifferent from zero, implying that the underlyinginteractions are not simply the antisymmetric onesresponsible for direction sensitive movement detection[see Eq. (25)]. The response is (non trivially) ,1-dependent, proving the existence of (symmetric)interactions between inputs (the crosskernels ofour Volterra formalism). It is always positive

(for ~ = 2 Hz and ~= 4 HZ) in full consistencewith the probable simultaneous presence of positivecontributions not affected by lateral interactions(the selfkernels of the Volterra formalism, Fig. 1b).Its A-dependence may require a rather complexpattern of interactions, possibly of order higher thantwo. Controls with stationary gratings showed nosignificant responses, in agreement with the notionthat stabilized retinal images do not elicit any attrac-tion (Reichardt, 1973; Pick, 1974).Similar experiments

with a contrast frequency ~ = 2 Hz showed a2nsimilar A dependence, in the limits of the experimentalerrors. If this finding holds for many Q)values - as inthe case of Yds(Gotz, 1973; Eckert, 1973; Poggio andReichardt, 1973b)- the underlying interactions shouldobey a strong constraint [see Eq. (28)]. Of course thewavelength dependence of Ydi, which is influencedfor small A by the reduction in effective contrasttransfer [see Eq. (18)], may be also enhanced (forA> 6°) by saturation of the torque response at themotor output level. However, Ydi is different fromzero also in the A region where the direction sensitiveresponse seems to reach a saturation plateau; more-over, the attraction for flickered gratings (where

6 The average lifetime of a test-fly in our laboratory conditionsis about one month.

saturation can be excluded) shows (Fig. 9) the samesharp peak at A= 6°. Thus it seems possible that, ifsaturation effects are present, they take place beforethe motor output affecting in a significant way onlythe direction sensitive optomotor response. Additionaldata about the flickered gratings and measurementsof the direction insensitive response at lower contrastsare necessary to clear this point.

3.3. Optomotor Responses Below and Above the Equator

The fly fixates black stripes only if they are pre-sented to the lower half below the equator of thecompound eyes (Reichardt, 1973; Wehrhahn andReichardt, 1974; Wehner, 1972). According to ourinterpretation, this fact implies that the directioninsensitive optomotor response to periodic patternsis essentially present only in the lower part of thecompound eye. In fact the (closed loop) fixationbehaviour is connected to the open loop averageattraction towards an oscillating stripe (Poggio andReichardt, 1973). It is possible to show, under quitegeneral assumptions, that only symmetric interactions(see Chapter 2) can originate an average responsetowards a narrow object; furthermore the directioninsensitive response Ydi depends on symmetric inter-actions. The same experiment of Fig. 6 has beenrepeated, stimulating an equivalent sector of the upperpart of the eye. The result is shown in Fig. 7. Thedirection sensitive optomotor response remains, asexpected, essentially unchanged. However in thiscase the direction insensitive part of the optomotorresponse is not significantly different from zero forall the tested A values. Controls done with anothervertical window (8 = [:tSo, :tSOO]) and anothergroup of flies led to the same conclusion. The resultrepresents a clear support for the theoretical argumentthat the interactions underlying the orientation be-haviour also underly the direction insensitive response(see Reichardt, 1970; G6tz and Wenking, 1973).

The A-dependence of the direction sensitive opto-motor response of Fig.6b looks somewhat differentfrom earlier data (Eckert, 1973, Fig. Sa) and from theone obtained in the upper part (Fig. 7). The maindiscrepancy apparently is not the location of the zerocrossing (the points at A = 4° and A= 3.2 are notsignificantly different from zero) but the amplitudeof the negative response for A= 3°.In the experimentsof Eckert in which the two eyes were stimulatedsimultaneously with window parameters 1p= [ - 180°,+ 180°] and 8 = [ +22°, -22°], the value of thereaction for A= 3° was about - 1dyn cm. A com-parable value holds also for Fig. 7 but not for Fig. 6b.In a control experiment the direction sensitive opto-

49

3

}--7f---I/

//

/

!u/f ,7!~

Y

i

50

1,5

E..'"=;..,

'81'"=CI'...~

~=[ -5~-25°]

- - - -II- ---0 = [.~.25°]

0,5

--0

-0,5

0 10 30

A [deg]5

Fig. 8. The direction sensitive response was measured withthe simultaneous rotation of both cylinders according to

y(l, 4) - y(2, 3) . .Yd. = as a functIOn of J.,both 10 the upper (0) and2

in the lower (8) part of the eye. The data are averages from 5 flies(age: 10-12 days). The contrast frequency is wi). = 2 Hz

flicker. Pick (1974a) has shown that the averageposition (1p)-dependent attractiveness of a narrowblack stripe can be obtained simply through flickerstimulation. His results suggest that the 1p-dependentdirect channels (self-kernels) mediate this response.Of course "self-interactions" are influenced in thesame way by movement and flicker, provided that thetemporal parameters are the same. The antisymmetricinteractions, responsible for the direction sensitiveresponse Yd.. cannot clearly provide any averageresponse to flicker; however the symmetric interactionsunderlying the direction insensitive response give anon zero average response to flicker. The connectionbetween the average response to a moving periodicgrating (single Fourier component) and to a flickeringperiodic grating is derived in Chapter 2. Second orderinteractions always give the same A.-dependence forthe two stimuli; higher order interactions do not havethis property apart from special cases. The at-tractivenessofa "flickered" periodicsinusoidalpattern(exposed to the lower part of the fly's eye) is shown inFig. 9 as function of the wavelength. The characteristicpeak at A.= 6°is again significant and the A.dependence

E..'" 0,5=;..,

-a.'"=CI'...

~ 0

30

A [deg]Fig. 9. Long stripes of polarized film were cut, in the 1tand in the(J directions, and periodic sheets with J.= 3.5°, 6°, 9°, 30° wereassembled. The gratings used previously were replaced by thepolarized sheets;a rotating homogeneouspolarized film, in frontof the striped film provided a periodical alternating "flicker" of

the grating at the frequency ~ = 2 Hz on one side of the eye.21tThe spatiotemporal pattern oflight therefore is given by [see Eq. (29)]

1p!(1p,t)=Io+mcoswotcos21tT' The average response plotted0here represents the attraction towards the pattern, given by:

response= t(r, +r,), where r, represents the pattern attractionwhen the pattern is flickered on the left side and r, when the patternis flickered on the right side. Each point represents a series ofmeasurements on at least 5 individuals (age: 10-12 days). The fullcircles (8) refer to .9 = [ - 5°, - 25°] and the other ones (0) to.9 = [5°,25°]. An experiment (not shown here) with another groupof 5 flies gave, for 3 different A.,the same values for the flicker response

and the direction insensitive response

0 105

of the responseseemsquitesimilar to the onepresentedin Fig. 6b. The finding is fully consistent with thepossible presence of second order nonlinearities.However our data can neither exclude higher orderinteractions nor restrict them as outlined in Chapter 2,since insufficient accuracy and especially the limitednumber of measuredpoints (only4values of A.for onecontrast frequency) do not allow similar conclusions.Equations (33), (34) actually show (for instance forfourth order interactions) that the difference, in theirA.-dependence,between flicker and direction insensitiveresponsemaybe quite small in a large variety of cases.Figure 9 showsalso the results of the flickerexposedto the upper part of the eye on the same group offlies. Again no significant attraction is found. Theresult clearly supports our interpretation of the roleplayed by symmetric interactions in the responsesto flicker, in the direction insensitive optomotorresponses and in the orientation behaviour. Inter-estingly both the crosskernels (associated to lateralinteractions) and the selfkernels (direct channels)components seem to be present only in the lower partof the eye.

E"~ 0,5>.;:!.Q,j=='..Q

~ 0

0 5 10 30

A [deg]

Fig. 10. Square wave gratings were oscillated on one side and thenon the other side of the eye with an amplitude of :t 2.5° and afrequency of 2 Hz. The attraction towards the pattern was measured

as in the flicker case (see Fig. 9) for 5 flies (age: 10-12 days)

3.5. Response to Oscillating Patterns

Our calculations (see Chapter 2.6) suggest that theeffect of small oscillations of a periodic pattern shouldbe equivalent to its flicker, with respect to the averageresponse. Figure 10 shows the result of an experimentin which (square) gratings of different wavelengthswere oscillated with an average amplitude A = :t 2.5°at a frequency of 2 Hz.

Equivalence with the flicker response is expectedfor those A. (in this case only A.= 30°) satisfying to

2; A ~ 1; if this condition is not respected significantdeviations may occur. This expectation is confirmedby the data of Fig. 10; higher spatial harmonics dueto the square grating are not expected to play animportant role.

Interestingly a collection of broad stripes like thegrating with A.= 30°, if oscillated, elicits an orientationresponse which is not significantly different from zero.In this case, as in other ones (Reichardt and Poggio,1974),the simple superposition of the attractiveness ofthe individual stripes does not predict the correctresponse of the fly to the whole pattern. The reasonfor the failure of superposition is clearly the presenceof lateral (inhibitory) nonlinear interactions beside thedirect channels (selfkernels) (see also Chapter 2.7).

Interestingly, under the same condition of small

oscillations ( 2~ A ~ 1), the zero-mean (second order)antisymmetric component of the response (YdJ isexpected [Eq. (39)] to contain a strong component atthe oscillation frequency. This "linear" movementresponse, observed by Thorson (1965) in the locust,can be identified for patterns containing long wave-

51

6

rF~~t....=iIQ,j

~......

e= 5e;0<~e'-Q

4=Q-~"Q

~3

20 5 10

Age [days]

Fig. 11. The direction insensitive optomotor response of groupsof flies of different ages was measured as in Fig. 6b, as a function of A.,with a contrast frequency wlA = 2 Hz, for 8 = [-25°, -5°]. Onegroup of 4 flies was tested on the 4th, the 8th, and 11th day. Another4 groups of 5 flies each, were tested on the 2nd, on the 5th on the9th and on the 14th day, respectively. The average location of themaximum in the direction insensitive component is plotted in thefigure. The horizontal bars indicate the error in determining theexact age of the flies and the vertical bars represent the standarddeviation in the location of the maximum. Since only a few wave-lengths were used (A = 2°, 3°,4°, 6°, 9°, 30°), clearly the mean valuesas well as their standard deviations have only an indicative meaning.The direction sensitive response did not change with age; both thezero crossing and the overall shape of the curve did not show anysignificant variation. Individual variability of the direction in-sensitive optomotor response decreases with age; its final A-de-

pendent shape is the one represented in Fig.6b

15

lengths, with the linear speed-dependent term ofthe phenomenological equation describing the fly'sorientation behaviour (Poggio and Reichardt, 1973a).

3.6. The Implications of Getting Old

As mentioned earlier all the experiments reportedso far were performed on 10-12 days old femalesMusca domestica. Figure 11 explains why. During aseries of preliminar measurements it became clearthat consistent and reproducible measurements ofYdi(A.)werecriticallydependenton the age of the flies.In a series of experiments, condensed in Fig. 11, thedirection sensitive and direction insensitive responsewere measured as a function of the flies' age. Themaximum of the direction insensitive response shifts

- - - - - ~- - - - - - -

52

with age towards ..1.=6°; at the same time the initialgreat variability of the response decreases with age.The direction sensitive response (Yds)however does notsignificantly change: zero crossing, minimum valueand the overall shape seems to remain the samethroughout the first 14days. The apparent stabilityof the "classical" optomotor response Ydi, beingconsistent with all previous reports, would also ruleout the possibility that the variation with age ofYdi(A)is due to degeneration of the optics of the eye.It is certainly too early to give an interpretation of thisphenomenon: for instance selective disappearance ofinhibitory interactions may leave a quite stablestructure of interactions responsible for the wave-length dependence of Fig. 6b.

The age dependence of the direction insensitivepart of the optomotor response implies, according toour interpretation, that selective attractiveness of'patterns should also depend on age. The idea wassupported by a series of prelirninar experiments.During closed loop fixation experiments a group of5 "young" flies (2 days old) orientate better (with asmaller standard error in the stationary positionhistogram) towards a stripe 1.5° wide than towardsa stripe 3° wide. However for another group of flies(10 days old) the opposite was true. The differences areclearly significant. Controls at other contrasts havegiven consistent results. Sophistications of this ap-proach are presently planned: they may offer newinsights into the problem of spontaneous patternpreference of insects. Prelirninar attempts of affectingthe A dependence of Ydi by exposing a group of fliesfrom the third to the tenth day, to drifting grating(A= 6°),4 hrs a day, did not show any significant effect.However more experimental work is needed to clearthis point.

4. Discussion

Not all problems raised in the introduction havebeen completely answered by our experiments. Yetthe main question about the existence of selectivelateral interactions receives a clear positive answerthrough the A dependence of the direction insensitiveresponse (Fig. 6b). Moreover the expected connectionbetween closed loop orientation behaviour, directioninsensitive optomotor response and flicker responseis strongly supported by the functional differencesconsistently found between upper and lower half ofthe eye. These data [especially the lack of flickerresponse in the upper part of the eye (Fig. 9)J offer thechallenge to physiologists of identifying structuralcorrelates of the orientation response. On the otherhand, just this problem of the existence of two systems

------

which are separately responsible for the directionsensitive optomotor response and for the orientation(direction insensitive) response, is not yet clear. Anumber of data support the idea of two differentsystems. In addition to a different light intensitythreshold (Reichardt, 1973) and a different sensitivityto polarized light (Wehrhahn, 1975), the directioninsensitive response is mainly present in the lowerpart of the eye, unlike the direction sensitive optomotorresponse; moreover only the direction insensitiveresponse changes with age. However other data mayimply that the direction sensitive optomotor responseis affected by the presence of the orientation response.The meaning and the extent of the interaction betweenthe two hypothetical systems is so far unclear.

The age dependence of the direction insensitiveresponse brings up a number of new questions.Direction sensitive optomotor responses do notchange with age, in agreement with the idea that theyrepresent an essentially "automatic" reflex, basic forthe navigatory skills of many insects (G6tz, 1972).In asimilar way it may be conjectured that only the cross-kernels, underlying pattern selective attraction, areage dependent, unlike the selfkernels responsible forthe orientation "reflex" [the D(1p) characteristicsJ:the idea is at least consistent with the approximativeconstancy of the mean value (average over A.)of Ydifordifferent ages. Clearly much caution is needed since asatisfactory characterization of the age dependenteffects still requires a number of additional experi-ments.

As outlines in Chapter 2 the A dependence of thedirection insensitive average response contains in-formation about the organization of the underlyinginteractions. However the ignorance of the order ofnonlinearity involved strongly limits our interpreta-tion. For instance, we do not know if the superpositionproperty of the response with respect to differenttemporal Fourier components (see Chapter 2.2) holdsfor Ydi, adding the difficulty of evaluating the effectsdue to the square wave form of our gratings. Satura-tion effects at the motor output level (discussedearlier) may also introduce nonlinear interactionsof a rather trivial nature. Furthermore, since thecontrast dependence of the direction insensitiveoptomotor reaction is not known, we can not exactlycorrect for the A-dependent attenuation of the effectivepattern contrast, due to the gaussian-like angularsensitivity of the receptors. The difficulty arisesbecause we cannot simply use the contrast dependenceof the direction sensitive component (see Eckert,1973; Buchner, 1974) for the other component, whichmay well depend on interactions of a different order

(not second). However the data of Fig. 6b, showing2 peaks for A = 3° and ..1.=6°, are at least consistentwith the possibility that the basic sampling spacingAcp in Eq. (23b) is Acp= 2°. In fact Ydi (Fig. 6b) can becorrected, for the contrast reduction, if a lineardependence on contrast is assumed, taking intoaccount the angular sensitivity distribution of re-ceptors (rhabdomeres) 7/8 (Gotz, 1965; Eckert, 1973).When the function obtained in this way is plotted on

1 1the T scale, an even symmetry around 40 becomesrather clear. This would suggest 7 that the basic meansampling spacing Acpis equal to about Acp= 2°. Thehypothesis of linear dependence of the reaction oncontrast is quite arbitrary: however the conclusionabove holds true (in the range of the experimentalerrors) also for quite large deviations from thishypothesis. If Ydiis given by a "selfkernels" componentand a "crosskernels" component with different con-trast dependences, measurements at decreasing con-trasts may well lead to a decreasing wavelengthdependence of Ydi(for instance in the assumption thatthe crosskernels depend on higher order nonlinearitiesthan the selfkernels).

As we mentioned in the introduction, the directioninsensitive response underlies pattern attraction. AA-independent component of the orientation responses(from the selfkernels) probably gives the positiondependent D(tp) for narrow stripes (Reichardt, 1973;Pick, 1974a), and respects the superposition rule(Reichardt, 1973; Reichardt and Poggio, 1974); onthe other hand the A dependent component (from thecrosskernels) cannot respect the superposition rule.As a consequence the transitivity law for spontaneouspattern preference (if object A is preferred to Band Bto C, A should be preferred to C) only holds for thoseseparations between the patterns for which lateralinteractions are negligeable. In this sense, although theselfkernels alone can provide, through a nonlineartp-parametrization, a non-trivial closed-loop behaviour(with rich classification properties: Poggio andReichardt, 1973a) only the crosskernels, associated tolateral interactions, underly a real. pattern-selective,context-dependent attractiveness.

In conclusion a number of questions about themechanisms underlying the average orientation re-sponse remain open and some new problems (mostlyconnected to the "age effect") arise from our data.

7 The Ydi(~)curve, as extrapolated from available pointsseems to require at least the first four Fourier coefficients ofEq. (26b).Only in the case of second order interactions would the Fourier

coefficient of order n in Eq. (30b) correspond to actual interactionsbetween inputs spaced by nJ q>(see Chapter 2.7).

- - - -

53

Therefore a few new experiments are presentlyplanned. For instance we would like to know if thedirection insensitive response can be influenced byearly visual experience; the age effect and its de-pendence on various parameters will be also in-vestigated. Measurements of the contrast dependenceof the direction insensitive response in a variety ofconditions may provide sufficient information aboutthe order of nonlinearities involved. Other experimentstesting the validity of the superposition property (withrespect to the spatial Fourier components of a pattern)for the direction insensitive response should alsoprovide interesting information. Moreover, selectivestimulation of a small number of receptors (Pick,1974a, b) has already given important data in con-nection with the general problem of our paper; in asimilar way the foreground-background discrimina-tion effect (Virsik and Reichardt, 1974) is presentlyproviding a characterization of at least a part of theinteractions affecting the orientation response. Ofcourse, specific electrophysiological and anatomicaldata will be necessary in order to connect directlyour behavioural analysis to the actual structure of thesystem.

Finally we would like to propose that the Volterraformalism, outlined in Chapter 2, may be also appliedto a variety of cases in which one has to describe,classify and characterize interactions. The Volterradescription provides a suitable theoretical languageto deal with the complex interplay of spatial andtemporal parameters in nervous networks and withtheir spatially distributed organization (many inputs).It is therefore hoped that the work presented here,beside its specific interest, may also represent apreliminar paradigm for a number of problems inthe neurosciences (not restricted to behaviouralanalysis), where nonlinear interactions between sen-sory inputs must be characterized in terms of theirfunctional organization.

Acknowledgement. We would like to express our gratitude toW.Reichardt for advice and many helpful discussions. Suggestionsand comments were also contributed by E.Buchner, K.G.Gotz,B. Pick, R.Virsik, and Chr. Wehrhahn. Many thanks are due toI.Geiss for typing and N.Strausfeld for reading the English manu-script.

References

Buchner,E.: In preparation (1974)Eckert, H.: Optomotorische Untersuchung am visuellen System

der Stubenfliege Musca domestica. Kybernetik 14, 1-23 (1973)Fermi, G., Reichardt, W.: Optomotorische Reaktionen der Fliege

Musca domestica. Kybernetik 2, 15-28 (1963)Geiger,Gad: Optomotor responses of the Fly Musca domestica to

transient stimuli of edges and stripes. Kybernetik 16, 37-43 (1974)

--- - - -

54

Gotz, K. G.: Optomotorische Untersuchung des visuellen Systemseiniger Augenmutanten der Fruchtfliege Drosophila. Kybernetik 2,77-92 (1964)

Gotz,K.G.: Behavioral analysis of the visual system of the fruitflyDrosophila. Proc. Inform. Processing in Sight Sensory Systems.Pasadena (1965)

Gotz,K.G.: Principles of optomotor reactions in insects. Bib!.aphtha!. 82, 251-259 (1972)

Gotz,K.G., Wenking,H.: Visual control of locomotion in thewalking fruitfly Drosophila. J. compo Physio!. 85, 235-266 (1973)

Kapany,N.S., Eyer,J.A., Keim,R.E.: Fiber optics, Part II: Imagetransfer on static and dynamic scanning with fiber bundles. J.Opt. Soc. Amer. 47, 423-427 (1957)

Knight,B.: The Horseshoe crab eye: a little nervous system that issolvable. In: Some mathematical questions in biology. IV.Lectures on mathematics in the life sciences, Vo!. 5, pp. 113-144,Providence, Rhode Island: The American Mathematical Society,1973

Kohonen, T., Ruohonen,M.: Representation of associated data bymatrix operators. IEEE Trans. Comput. C-22, 701-702 (1973)

Land, M. F., Collett, T. S.: Chasing behaviour of houseflies (Fanniacanicularis). Z. compo Phys. 89,331 (1974)

Marmarelis, P.Z., McCann,G. D.: Development and applicationof white-noise modelling techniques for studies of insect visualnervous system. Kybernetik 12,74 (1973)

Pick, B.: Visual flicker induces orientation behaviour in the flymusca. Z. Naturforsch. 29a, 310-312 (1974)

Pick,B.: In preparation (1974b)Poggio, T.: Processing of visual information in insects: a theoretical

characterization. In: Drischel,H. (Ed.): Biocybernetics. VI. Leip-zig: Fischer 1973 (in print)

Poggio, T.: Processing of visual information in flies: from a phe-nomenological model towards the nervous mechanisms. In:Vegli,A. (Ed.): Proc. I. Symp. of the It. Soc. Biophysics, Camogli1973 (1974a)

Poggio, T.: On optimal nonlinear associative recall. Bio!. Cybernetics19 (1975)

Poggio, T.: A theory of nonlinear interactions in many inputs(nervous) systems. In preparation (l974c)

Poggio, T., Reichardt, W.: A theory of the pattern induced flightorientation of the fly Musca domestica. Kybernetik 12, 185-203(1973a)

Poggio, T., Reichardt, W.: Consideration on models of movementdetection. Kybernetik 13, 223-227 (1973b)

Reichardt, W.: Autocorrelation, a principle for the evaluation ofsensory information by the central nervous system. In: Rosen-blith, W. A. (Ed.): 'Sensory communication, p.303. Cambridge(Mass.): MIT Press 1961

Reichardt, W.: In: Schmitt,F. O. (Ed.): The insect eye as a model foranalysis of uptake, transduction, and processing of optical datain the nervous system. The neurosciences. Second Study Program1970

Reichardt, W.: Musterinduzierte Flugorientierung der Fliege Muscadomestica. Naturwissenschaften 60, 122-138 (1973)

Reichardt, W., Poggio, T.: A theory of the pattern induced flightorientation of the fly Musca domestica. II. Bio!' Cybernetics 18,69-80 (1975)

Robinson, D.: Models of saccadic eye-movement control system.Kybernetik 14, 71 (1973)

Seelen, W. v., Reinig,H.J.: Methoden zum Vergleich des Leistungs-vermogens biologischer und technischer Systeme. Kybernetik10, 1 (1972)

St.Cyr,G.J., Fender,D.H.: Nonlinearities of the human oculo-motor system: Gain. Vision Res. 2, 1235 (1969)

Virsik,R., Reichardt, W.: Tracking of moving objects by the flyMusca domestica. Naturwissenschaften 61, 132-133 (1974)

Thorson,J.: Small-signal analysis of a visual reflex in the locust.I-II. Kybernetik 3, 41-66 (1966)

Wehner,R.: Dorsoventral asymmetry in the visual field of the bee,Apis mellifera. J. compo Physio!. 77, 256 (1972)

Wehrhahn, C.: Verhaltensstudie zur musterinduzierten Hohenorien-tierung in der Fliege Musca domestica. Dissertation, UniversitatTtibingen 1974

Mr. Gad GeigerMax-Planck-Institut flir biologische KybernetikD- 7400 Ttibingen, Spemannstr. 38Federal Republic of Germany