Entropy and Cortical Activity:Information Theory and PETFindings

K.J. Friston,1 C. D. Frith,1 R. E. Passingham,1-2 R. J.Dolan,1 P. F. Liddle,1 and R. S. J. Frackowiak'

1 MRC Cyclotron Unit, Hammersmith Hospital,London W12 OHS, United Kingdom and2 Department of Experimental Psychology, OxfordUniversity, Oxford OX1 3UD, United Kingdom

Functional segregation requires convergence and diver-gence of neuroanatomical connections. Furthermore, thenature of functional segregation suggests that (1) sig-nals in convergent afferents are correlated and (2) sig-nals in divergent efferents are uncorrelated. The aim ofthis article is to show that this arrangement can bepredicted mathematically, using information theory andan idealized model of cortical processing.

In theory, the existence of bifurcating axons limitsthe number of independent output channels from anysmall cortical region, relative to the number of inputs.An information theoretic analysis of this special (highinputoutput ratio) constraint indicates that the maximaltransfer of information between inputs, to a corticalregion, and its outputs will occur when (1) extrinsicconnectivity to the area is organized such that the en-tropy of neural activity in afferents is optimally low and(2) connectivity intrinsic to the region is arranged tomaximize the entropy measured at the initial segmentsof projection neurons.

Under the constraints of the model, a low entropyis synonymous with high correlations between axonalfiring rates (and vice versa). Consequently this antisym-metric arrangement of functional activity in convergentand divergent connections underlying functional seg-regation is exactly that predicted by the principle ofmaximum preservation of information, considered in thecontext of axonal bifurcation.

The hypothesis that firing in convergent afferents iscorrelated (has low entropy) and spatially coherent wastested using positron emission tomogrephic measure-ments of cortical synaptic function in man. This hy-pothesis was confirmed.

Certain patterns of cortical projections are so commonthat they could amount to rules of cortical connec-tivity. "These rules revolve around one, apparently,overriding strategy that the cerebral cortex uses—thatof functional segregation" (Zeki, 1990). Functionalsegregation demands that cells with common func-tional properties be grouped together. This in turnnecessitates both convergence and divergence of cor-tical connections. Anatomical convergence is re-quired to assemble functionally distinct sets of sig-nals, distributed over a functionally heterogeneousarea, into a specialized area. Divergent efferents seg-regate and disseminate mixed signals to more spe-cialized regions. Convergence is seen on many scales.For example, the connections between VI and V5 areconvergent in the sense that one V5 cell receivesprojections from many VI cells, evidenced by thesmaller areal extent of V5 compared to VI and thelarger receptive fields found in V5 (Zeki, 1971). Sim-ilarly, there is convergent input from VI blobs (inwhich low spatial frequency and wavelength selec-tivity are represented) to the thin stripes of V2 inwhich cells have similar properties with larger recep-tive fields (Livingstone and Hubel, 1984) and fromseveral thin stripes in V2 to V4 (Zeki and Shipp, 1989).As convergent projections assemble similar attributesof the visual field, it is inferred that firing rates inconvergent afferents are correlated. Divergent con-nections, on the other hand, mediate redistributionof functionally distinct signals to different areas andsubareas. Indeed, it was on the basis of the anatomicalevidence for multiple and divergent projections fromVI to extrastriate areas that the role of VI as a func-tional segregator was first proposed (Zeki, 1975). Asdivergent connections parcel out functionally differ-ent signals to various extrastriate regions, it is con-cluded that firing rates in divergent efferents are large-ly uncorrelated. The capacity to decorrelate outputsis considered by some to be a central component offeature detection (e.g., Foldiak, 1989; Oja, 1989; Hor-nik and Kuan, 1992).

Alternative arrangements are unlikely on thegrounds that they would preclude categorization ofevents in the sensory field. In general, uncorrelatedsignals in convergent connections would confoundindependent features and disallow any subsequentseparate categorization on the basis of those features.In most processes with a biological flavor (e.g., Pois-

Cercbral Cortex May/June 1992;2:259-267; 1047-3211/92/14 00

son point processes and stationary Gaussian process-es), conflating two dissimilar inputs (A and B) is ir-reversible, where categorizaton can proceed on thebasis of A or B but not on the basis of A and B alone.Similarly, correlated signals in divergent efferentswould lead to a complete failure in segregating afunctionally mixed input and a potential failure toextract features necessary for categorization.

Functional segregation therefore suggests two an-tisymmetric features of cortical organization: (1) sig-nals in convergent afferents are correlated, and (2)signals in divergent efferents are uncorrelated. Theaim of this article is to show that this arrangementand its conceptual counterpart—functional segrega-tion—are exactly consistent with a simple formula-tion of cortical processing in information theoreticalterms.

By considering a particular constraint imposed byaxonal bifurcation, we demonstrate that informationtransfer is optimized when inputs to a cortical regionare substantially correlated (have an optimal and lowentropy) and outputs are uncorrelated (have a highentropy). We describe the theory below and provideempirical evidence of autocorrelated, coherent affer-ent activity in the cortex using PET measurements ofneurophysiology.

TheoryThe behavior of any cortical region, of small arbitrarydiameter, is characterized by its inputs (afferents),outputs (efferents), and the transformation of neuraldischarge patterns between the two. Using the ter-minology of Shepherd and Koch (1990), we defineinputs as extrinsic axons (arising from distant cells)giving rise to arborizations that synapse on cell pro-cesses within the region. Outputs are the single out-put points (initial segments) of projection (principalor relay) cells giving rise to at least one extrinsic axon.Extrinsic connections connect distant cortical regions,as distinct from intrinsic connections (e.g., interneu-rons or recurrent axonal collaterals).

This definition of an output is strictly neuroana-tomical, but functional independence is implicit. Twoaxons deriving from the same initial segment are con-sidered to be part of the same output and exhibit thesame pattern of firing. Only different outputs can fireindependently.

The synaptic, parasynaptic, and ephaptic transfor-mation of discharge patterns is effected by direct con-nections between extrinsic afferents and projectioncells (e.g., direct axodendritic synapses on the apicaldendrites of large pyramidal cells), by intrinsic con-nections (e.g., stellate interneurons), and by intrinsicrecurrent collaterals (Mountcastle, 1978; Powell,1981).

Axonal Bifurcation and Constraints onExtrinsic ConnectivityOne special aspect of brain connectivity, central tothe argument developed below, is that the numberof inputs to a conical region exceeds the number ofoutputs. This characteristic is the main constraint un-

der which the optimization of information transfer isconsidered. It is self-evident that the number of in-puts to a single neuron (multiple dendritic synapses)exceeds the number of outputs (single initial seg-ment) from that neuron. It is also self-evident, butperhaps not so obvious, that the number of inputs togray matter area will, on average, be substantiallygreater than the number of outputs. This is a conse-quence of bifurcating or nonrecurrent axonal collat-erals.

Imagine a closed surface or boundary at all gray-white matter interfaces. The number of axonal fiberscrossing that surface into white matter is less than orequal to the number crossing in the opposite direc-tion—the gray matter inputs (the possible inequalityresults from axonal bifurcation in the white mattervolume). Because one initial segment (output) cangive rise to several extrinsic axons, the number ofoutputs is less than the number of fibers enteringwhite matter, which in turn is less than or equal toinputs traversing the boundary in the other direction.Therefore, the number of outputs is less than thenumber of gray matter inputs. In general, if an efferentaxon bifurcates on average n times there will be n +1 axonal fibers for each output. Given that all extrinsicafferent fibers represent an input, the input:outputratio would be (n+ 1):1. This argument assumes thatthe difference between effectors and receptors is smallin comparison to the total number of extrinsic axons.

Double labeling experiments have demonstratedthat the Meynert cells of layer 6 in VI project throughbifurcating axons to both V5 and the superior collic-ulus (Fries et al., 1985). Nonrecurrent axonal collat-erals (e.g., bifurcating axons) can mediate backwardconnections. Studies of axonal bifurcation show that,in general, backward projections are less submodalityspecific than outward projections (Bullier and Ken-nedy, 1987; Shipp and Zeki, 1989a, b). Because back-ward projections are common, axonal bifurcation maybe ubiquitous.

Examples of High Input.Output RatiosV5 receives convergent afferents from VI, shown bythe fact that V5 receptive fields are larger than the VIreceptive fields of which they are composed (Zeki,1971). Consequently, a single V5 neuron receives ax-onal afferents from more than one VI cell. The num-ber of initial segments on V5 projection cells cannotexceed the number of V5 cells. Therefore, the numberof inputs to V5 exceeds its outputs. Examples can befound where the reduction in outputs is high. Eachof the A laminae of the cat's lateral geniculate nucleus(LGN) contains roughly 400,000 cells, of which about300,000 are projection cells. The LGN receives slight-ly fewer than 100,000 retinogeniculate axons and morethan 4,000,000 corticogeniculate axons, in additionto afferents from the brainstem reticular formation andthe reticular nucleus of the thalamus (Sherman andKoch, 1990). The input:output ratio is, in this ex-ample, about 14:1. Note that this output reduction islargely due to backward projections from the cortex.

260 Cortical Organization • Friston et al

Entropy and CorrelationsThe application of information theory often takes theform of optimizing a particular aspect of performanceunder a series of constraints. In what follows, thetransfer of information associated with the transfor-mation of neural firing by a small region of cortex isthe object of optimization. In the context of brainlikefunction, the principle of maximum information pres-ervation has an intuitive, predictive, and constructvalidity (Linsker, 1988; Foldiak, 1990) and is relatedto the concept of redundancy reduction (Barlow, 1961;Atick and Redlich, 1990). Linsker has coined the terminfomax in reference to this principle and has dis-cussed its ramifications and precedents (Linsker,1988).

The main constraint under which the principle ofinformation preservation is developed is the reduc-tion or constriction of outputs relative to inputs. Aheuristic argument suggests that if information in aninput space is redistributed over a substantially small-er number of outputs, information transfer will beenhanced by mutual predictability in the inputs. Thisimplies that firing in one input is predictive of (andpredicted by) activity in the remainder; that is inputswill be correlated. Conversely, efficient use of (noise-less) outputs prohibits mutual prediction and re-quires the outputs to be independent or uncorrelated.

Put another way, if there is a constriction in thenumber of available channels, transfer will be moreefficient if less information tries to get through atonce. These heuristic arguments can be illustratedmore formally using information theory: information(/) is the improbability of an event (x) expressed asthe logarithm of the inverse of its probability \p(x)],or

/(*) = - ln(p(x)) . (1)

An event with low information is highly probable, andits occurrence could have been predicted. For a num-ber of continuous events (neural activity across sev-eral axons, x) the average information is referred toas entropy [//(*)]. For an n-dimensional space wherethe probability density of axonal firing is Gaussian,the entropy is given by (Jones, 1979)

H(x) =ln((2xe)-- |P*l)/2, (2)

where px is the covariance matrix (and | pjc| its de-terminant) describing the covariance between neuralfiring. Following transformation, in accord with theprinciple of information preservation, the entropy(average information) of the outputs should be high.The model of cortical processing we use is definedby the following assumptions: (1) stationary multi-variate Gaussian continuous input with zero meanand unit variance, (2) additive uncorrelated orthog-onal Gaussian noise in, and only in, the inputs, and(3) linear transformations under the constraint that2'c,/ = 1, where cv is the coefficient scaling the con-tribution from input / to output j . The last constraintconserves total synaptic contacts a dendritic tree canexpress. For example, if synaptic efficacy is propor-tional to the radii of postsynaptic specializations, then

the total areal extent of all specializations on one treeis unity. For this idealized model, the entropy of theoutputs \H{y)] is arithmetically related to the mutualinformation between inputs and outputs [I(x, y)]. Themutual information reflects the information that yconveys about x (Jones, 1979);

/(*, y) = H(y) - H(z), (3)

where H(z) is the entropy of noise in the inputs. Forunchanging noise characteristics, an increase in H(y)is equivalent to an increase in the mutual information.Optimizing information transfer thus reduces to max-imizing H(y) under the constraints imposed (1) bythe model assumptions and (2) by a high input:outputratio.

The linear transformation that maximizes outputentropy (under the above assumptions) is a principalcomponent transformation (Linsker, 1988; Foldiak,1989; Oja, 1989; Hornik and Kuan, 1992). This trans-formation renders the outputs orthogonal or inde-pendent (not mutually predictive). For a more de-tailed analysis of the role of noise under less restrictiveconstraints than those assumed here, see Linsker(1988) and Atick and Redlich (1990). We use theprincipal component transformation to examine howthe upper limit on H(y) = Hma depends on the co-variance structure of the inputs (Cx) and the input:output ratio (n-.m).

This dependency is illustrated in Figure 1 by mod-eling the input covariance matrix as a Gaussian au-tocovariance matrix:

px(i,f. i - j= b) (4)

As /? gets bigger, the covariance between distant in-puts increases and the inputs exhibit a greater degreeof intercorrelation or coherence. /9 will be referred toas coherence. The upper limit on H(y) is given byEquation 2 where \px\ = lit, and e, are the m largesteigenvalues of px. It is immediately obvious, fromFigure 1, that as soon as an initially incoherent inputbecomes more coherent, Hma increases, therein po-tentially optimizing information transfer. This effectis more sustained with higher input:output ratios. Theupper solid line in Figure 1 corresponds to a ratio of2:1 (on average extrinsic axons from projection cellsbifurcate once). In this case, the optimal input co-herence is realized fairly soon. Subsequent increasesresult in a progressive reduction in output entropy asthe capacity to support high variances in the largenumber of orthogonal outputs fails. For higher ratios,the optimal coherence (largest H^) is much greater.This behavior is a general feature of all monotonicdecreasing autocovariance functions we have exam-ined. Figure 2 illustrates the generality of this behav-ior by relating H^ to the entropy of the inputs for anumber of autocovariance functions. Initially as inputentropy falls (coherence increases), Haax increases toa maximum and then declines monotonically. Thelocation of the maxima depends on the nature of theautocovariance function and input:output functions;however, all are associated with nontrivially low en-tropies (nonzero coherence). The three autocovari-

Cerebral Cortex May/June 1992, V 2 N 3 261

coherenceFigure 1 . Dependency of upper limit on output entropy (//_) on coherence (standard deviation of a Gsussian autocovariance function] of a stationary input This relationship isshown to five cases of increasing inputoutput {am) ratio with n = 64 inputs. H^ = ln((2 x e f - I I e . l / Z where«, are the m largest eigenvalues of px{i,f. / - y - h| =

ance functions depicted in Figure 2 are a Gaussian(Eq. 4), an exponential px(b) = exp( — b/$) |s>0, anda hyperbolic function px(b) = (fo + iy\0<o (see Fig.2 caption for the ranges of /S used).

The use of autocovariance matrices assumes thatthe input covariance pattern [px(h)] is stationary; cross-covariances are a function of distance between inputs(h), not specific locations. This is appropriate giventhat we are modeling an organizational principle thatis invariant over the entire cortex.

In conclusion, there is an optimal coherence thatmaximizes the entropy of the outputs for any input:output ratio. In other words, given the above con-straints, the most informative and balanced neuralactivity in a series of independent, uncorrelated out-puts from an area is associated with low-optimal-en-tropy, correlated activity in the inputs. This is exactlythe arrangement predicted by functional segregation:(1) correlated activity in convergent afferents and (2)uncorrelated activity in divergent efferents. Hebb'srule can be seen as satisfying a special case of thisarrangement—where the input entropy is unspecifiedand the cortical area in question reduces to a singledendritic tree. Oja (1982) has analyzed a model witha single output unit using a local Hebbian connectionstrength modification rule and demonstrated that theunit extracts the principal component with the largesteigenvalue (ft) from a stationary input. The entropy[H(y)] of a unidimensional Gaussian distribution isgiven by (Jones, 1979) H(y) = log(2weft)/2. Conse-quently, output entropy is maximized. Note the an-tisymmetric nature of this conclusion: to comply withthe principle of maximal information preservation—in this context maximization of output entropy—thereis an almost paradoxical requirement that the entropy

of the inputs be significantly less than chance expec-tation.

Examples of Correlated Inputs"The EEG is both a consequence and a sign of cor-related activity in the brain" (Cook, 1991). If neuronsconverging onto cortex all fired independently, thenthe effects on the electrical field outside the craniumwould largely cancel. High- and low-frequency fieldpotential changes imply a strong local correlation.

Example of Independent OutputsThere are divergent projections from VI to V5 andV4. Independent activation of V5 and V4 has beendemonstrated in human subjects using functional im-aging (Zeki et al., 1991). Therefore, firing in VI ef-ferents projecting to V5 can be independent of firingin projections to V4.

Empirical ValidationWe predicted that convergent afferentation in the cor-tex will have a low entropy over a range of spatialdomains. This prediction can be reformulated in termsof the instantaneous measurement of neural activity,predicted to have nontrivial autocovariance, or to bespatially coherent.

There is a substantial amount of empirical evi-dence to suggest that regional cerebral blood flow(rCBF) is coupled to neural discharge rates in corticalafferents (e.g., Fox and Raichle, 1986). Compellingevidence that this coupling operates over small spa-tiotemporal domains is provided by high-resolutionoptical imaging of microcirculatory events in oculardominance columns of monkey cortex during visualstimulation (Frostig et al., 1990). This and other ev-

262 Conical Organization • Fnston et al.

-20

input entropy - H(x) {nats}

Figirs 2 . Semlar 10 Rgure 1, but The input coherence is expressed as entropy [M<r|] and the inputoutput ratio (/r.m) is 5:1 [n *• 50, m — 10). The three lines correspond tothree autocovariance functions of /S: (1) Gaussian [as in Fig 1, e x p l - A y i ^ H , /S = 0-2; dashed line], (2) exponential [exp(-/7//3|. /S = 1-41; dotted line] and (3) hyperbolic[{h + I )* , /3 = - 2 . 5 - 0 ; solid line] All three cases demonstrate that as input entropy falls from incoherence [Hx\ = 70.95] output entropy increases (until a maximum isreached).

idence (Conrad and Klingelhofer, 1989) suggests thatrCBF is coupled to afferent activity over a scale of lessthan 1 mm and less than 1 sec. rCBF is therefore aneurophysiological index of afferent activity that wepredicted would show nontrivial autocovariance overmany millimeters.

The spatial frequencies of multiple realizations ofstationary processes remain unchanged when inte-grated. Consequently, the coherence or autocovari-ance of the stationary component of many instanta-neous rCBF measurements integrated over time willbe the same as any single measurement in isolation.We use this to advantage in the PET technique, whichgives the integrated rCBF over several minutes.

Clearly, actual cortical activity may have many non-stationary components, reflecting regional specificityof function. These nonstationary components are notthe subject of the present analysis. As a first step, wewished to demonstrate coherence as a ubiquitous andregionally invariant characteristic of cortical activity.Provisional work (K. J. Friston, C. D. Frith, R. E. Pas-singham, R. J. Dolan, P. F. Liddle, and R. S. J. Frac-kowiak, unpublished observations) using statisticaltests of sphericity, suggests that it is possible to mea-sure regional differences in entropy.

The spatial resolution of PET is poor, but in a well-behaved way (Glick et al., 1989). The confoundingeffect of poor resolution on estimating autocovariance(coherence) can be accounted for by deconvolutionwith the noise power spectrum (NPS).

MethodsTo remove systematic and nonstationary neurophys-iological components, due to regional variations in

perfusion and anatomical configuration of gyri, weused the difference in activity between two measure-ments of cortical rCBF to estimate its autocovariance.Experimental control was exerted over the physio-logical differences by using two tasks repeated in apairwise fashion. The choice of these tasks was ar-bitrary from the point of view of the present analysis,as regional differences were not an issue. The taskswere chosen because they activate extensive andwidespread cortical regions (Frith et al., 1991).

Data AcquisitionSix normal male volunteers where scanned 12 timesin the same session while performing one of two tasksin an alternating sequence (repeating a heard letterand responding with a word that began with a heardletter). Similarly, the standard (Hoffman) three-di-mensional human brain phantom was scanned 12 timesusing 18F at a concentration of 0.6 jxCi/cc in the graymatter compartment. The total counts per image cor-responded to the human studies. Permission to per-form these studies was obtained from the local ethicalcommittee and Advisory Committee for the Admin-istration of Radioactive Substances of the UK. Scanswere obtained with a CTI (model 953B; CTI, Knox-ville, TN) PET camera as a fully three-dimensionalacquisition. Reconstructed (Townsend et al., 1992)images had a resolution of 5.2 mm (T. J. Spinks, T.Jones, D. L. Bailey, D. W. Townsend, S. Grootnook,P M. Bloomfield, M. C. Galardi, M. E. Casey, B. Sipe,and J. Reed, unpublished observations). The volumeimages contained 128 x 128 x 31 voxels correspond-ing to 2 x 2 x 3.1 mm. "O was administered intra-venously as radiolabeled water infused over 2 min.

Cerebral Cortex May/June 1992, V 2 N 3 263

Figure 3. Picture of the brain showing the extern of conical surface analyzed. Anexample of a fined conical ellipse is supsnmposad on a transverse s e a m

The total counts per voxel during the buildup phaseof radioactivity served as an estimate of rCBF (Foxand Mintun, 1989). The tasks began 20 sec prior todelivery of radiolabeled water.

Data AnalysisThe cortical rim was sampled from the 12 scans fromeach subject and the phantom. This sampling usedan ellipse fitted to the length and width of 20 con-secutive slices (see Fig. 3). Subtracted sequential pairs(Kijewskl and Judy, 1987) were used to estimate therCBF autocovariance function.

The objective of our analysis was to show that thehuman data, but not the phantom data, exhibited non-trivial autocovariance over many millimeters. Follow-ing normalization to zero mean and unit variance, theone dimensional subtracted rCBF data were subjectto Fast Fourier transformation. The transformed rCBFdata were averaged across all cortical ellipses fromone subject and divided by the NPS in frequency space.This corresponds to deconvolution in Cartesian space.The resulting spectral density functions can be seenin Figure 4a for the six subjects and the phantomdata, included for comparison. Inverse Fourier trans-formation of the spectral density functions yieldedthe corresponding autocovariance functions (Cox andMiller, 1980), which because of the initial normal-ization are also the autocorrelation functions (Fig.4b).

We used an empirical estimate of the NPS (poly-nomial fit of the Fourier transform of subtracted se-quential phantom pairs) to ensure a proper estimateof low spatial frequencies. This accounted for two-dimensional aliasing due to pixel sampling (Kijewskiand Judy, 1987).

ResultsFigure 4a shows the normalized spectral density func-tions for the phantom data (solid line circled at thebeginning) and the six subjects. The phantom datareceive equal contributions from all frequencies, con-sistent with uncorrelated white noise these data rep-resent. In contrast, the physiological data have rela-tively large contributions from low spatial frequenciesthat result in monotonic declining autocorrelationsat increasing distances. This coherence is seen in thecorresponding autocorrelation functions in Figure 4b.Again, the phantom autocorrelation function is givenfor comparison and is rendered as a solid line. Au-tocorrelation is evident (in this conservative analysis;

see below) at 5 mm and beyond. This coherence can-not be explained by intrinsic connectivity, which hasa maximal spatial extent of 3 mm (Mountcastle, 1978).

Although the autocovariance is significantly great-er than 0 over large distances [e.g., mean p(8 mm)= 0.0644; r=6 .47 ;p< 0.001; df = 5), the size of theautocorrelations is very small. This is because therCBF process is embedded in large amounts of un-correlated noise associated with the high-resolutionPET technique used. Figure 4b illustrates this pointin that the low-frequency structure in the rCBF data"sits on top o f a substantial amount of white noisedistributed uniformly over all frequencies. We choseto use a high-resolution technique in order to lend astereotactic validity to our sampling of the cortex. Itis possible to extend the analysis and partition theobserved process into an uncorrelated noise processand a residual correlated process corresponding tothe underlying rCBF differences. However, the resultspresented above are sufficient to confirm the hypoth-esis that afferent cortical activity exhibits autocovar-iance.

DiscussionWe have examined the implications that functionalsegregation holds for the arrangement of activity inconvergent afferents and divergent efferents. The ar-rangement implied by functional segregation is pre-cisely that predicted theoretically using ideas frominformation theory. This theoretical analysis led to ahypothesis about the spatial autocorrelations of ac-tivity in convergent afferents in the cortex. We de-signed an experiment to test and confirm this hy-pothesis.

In analogy with the application of information the-ory to continuous channels (e.g., Shannon's contin-uous channel theorem), we have taken an idealizedmodel and reduced the problem to one of finding thelargest H(y) subject to certain constraints (e.g., whenthe input is power limited; see Jones, 1979). The con-straint of particular interest in this formulation derivesnot from the role of noise (Atick and Redlich, 1990)or available power in the inputs but from the topo-graphic arrangement of the connections, namely, areduction in the number of independent channels ateach cortical transformation. Axonal bifurcation hasbeen used as empirical support for this constraint,which implies that the dimensionality of Inputs to acortical region exceeds that of the outputs. Inputs aredefined as extrinsic axonal afferents to the region.Outputs are the initial segments of projection cells.This information theoretic approach suggests that alow optimal input entropy maximizes informationtransfer. Firing rates measured in afferent axons willbe of low entropy when those firing rates are corre-lated or coherent. This would mean, at a symboliclevel, that events captured by neural firing map closetogether if and only if those events were in some waymutually predictive. In other words, probabilistic reg-ularities or invariances in extrapersonal space wouldbe embodied in the spatial convergence of neuralprojections.

284 Cortical Organization • Friston et a].

ate 0.04 0.06 0.08

frequency cycles/mm

0.1 0.12 ai4

10 15 20 25 30 35 40

h = distance {mm}

R g u r e 4 . s, Spectral densny functions for phantom data [solid line acted it beamng) and six subjects [dotted foes). The spectral density functions have been normalized tohsve a mean density of I in spatial frequency bins greater than 0.05 cycles/mm. These functions were derived from a series of corticai ellipses (20 from each subject) at leasi128 poets = 256 mm in length. The density functions have been resolution corrected by deconvokruon with an empirical estimate of the NPS. The central finding is that onlythe physiological data have large amounts of slowly undulating low spatial frequencies, b, the same data but presented as autocorrelation functions following inverse Fouriertransform of the spectral density functions. Autocorrelation for the phantom data [solid line] a near zero at aS distance!. In contrast the rC8f data [dotted lines, dashed fine =mean) show nonzero autocorrelations up to 20 mm.

Spatial coherence was measured in terms of theautocorrelation of a series of subtracted rCBF mea-surements in the cortex. The equivalence betweencross-correlations over time and the autocorrelationof a single observation over space depends on theassumption of stationariness. As predicted, autocor-relation was evident over extensive (10-20 mm) do-mains.

Massive divergence and convergence are a nec-essary consequence of these entropic considerations.As extrinsic efferents (and their collaterals) are un-correlated (high entropy), it is unlikely they will ter-minate in the same cortical area. This implies thatefferents are divergent. Divergence implies conver-gence, and both are features of conical organization(Mesulam, 1990; Zeki, 1990). The classification of

Cerebral Cortex May/June 1992, V 2 N 3 266

convergent versus divergent connections is purely amatter of where the connections being described arereferred. A cortical focus can receive convergent pro-jections from extensive and different areas and sub-areas but can only give rise to divergent connections.No single set of axons can be both convergent withreference to one point and divergent with respect toanother. This is important given the assertion thatthese two classes have opposite entropic tendencies.

The transformation effected by intrinsic connec-tivity is assumed to result in a decorrelation of outputs(Foldiak 1989). This decorrelation results in a highentropy. A high entropy is characteristic of "func-tional segregators." Each millimeter of VI containsall the visual information from a particular retinalpoint that is destined for the cortex. Since it is difficultto imagine that the same signals are relayed in thedivergent and parallel projections, VI was proposedto act as a functional segregator (Zeki, 1975). Both aconvergence of correlated activity and a divergenceof uncorrelated activity are implicit. Correlated visualmotion information converges on V5 from widely dis-tributed VI efferents. Conversely, uncorrelated sub-modality information (e.g., motion, color, depth) isdivergently redistributed to functionally specializedareas from VI. Within a grossly homogeneous func-tional area (e.g., V5), the divergent outputs shouldbe uncorrelated and correspond to a finer but stillorthogonal segregation of submodality information,for example, a segregation into the direction and mag-nitude components of the velocity vector. There isevidence for speed- and direction-invariant cells inV5 (Zeki, 1990).

MechanismsPrincipal component transformation has been usedto estimate the theoretical limit on entropy over moutputs given n inputs and their covariance matrix.This does not imply that intrinsic connectivity per-forms a principal component transformation. How-ever, there are specific proposals that finding the prin-cipal component space is an important theme infeature detection (Linsker, 1988; Oja, 1989; Foldiak,1990; Rubner and Schulten, 1990). Foldiak has de-scribed a (anti-)Hebbian mechanism that effects atransformation of a high (n)-dimensional input intoa lower (m)-dimensional output that spans the samesubspace as the mlargest principal components ofthe input (Foldiak, 1989, 1990). In addition to Hebbian feedforward connections, Foldiak's model de-pends on anti-Hebbian feedback connections be-tween the output "neuron-like units" to keep theoutputs uncorrelated (orthogonal). Durbin andMitchison (1990) have used conical wiring lengthconstraints to model connectivity in the primary visualcortex (VI). They present simulations that are re-markably reminiscent of empirically determined con-figurations. One of the basic tenets of their approachis the requirement that contiguous regions of an ex-ternal parameter space (e.g., position in the visualfield) should be represented close together in theconical sheet The spatiotemporal contiguity of real

events would confer coherence (minimize the entro-py of discharge rates measured in afferent fibers im-pinging on contiguous regions in VI) in retinotopicmaps if and only if there is a topographic preservationof real space-time contiguity relationships of the sonthey suggest.

From the point of view of the theory of neuronalgroup selection (Edelman, 1978, 1987), the organi-zational tendencies discussed above may be relevantat the level of the selective expression of the second-ary repertoire. The convergence of temporally cor-related inputs onto the same dendritic tree can beenvisioned in terms of synaptic consolidation, whichdepends on synaptic discharge and changes in trans-membrane potential. This is because the behavior ofeach axonal input is accessible to the remaining in-puts by (electrotonic) communication through thedendritic processes they all share. We have, however,discussed coherence in terms of spatial domains,which are far more extensive than a single dendritictree. In this case, a different mechanism must be pos-tulated that does not depend on convergence ontoone neuron. Such a mechanism (based on nitric ox-ide) has been proposed (Gaily et al., 1990; Montagueet al., 1991). The effects of a short-lived, rapidly dif-fusible signal on local synaptic plasticity have beensimulated and shown to be able to link the activity ina local volume of tissue, regardless of whether theneurons are directly connected by synapses.

NotesK.J.F. was funded by the Wellcome Trust. We thank PeterFoldiak and the Fellows of the Neurosciences Institute forinspiring discussions, and Semir Zeki for invaluable guid-ance during the development of this work.

Correspondence should be addressed to Dr. Karl J. Fris-ton, The Neurosciences Institute, 1230 York Avenue, NewYork, NY 10021.

ReferencesAtickJJ, RedichAN (1990) Towards a theory of early visual

processing. Neural Comput 2:308-320.Barlow, HB (1961) Possible principles underlying the

transformation of sensory messages. In: Sensory com-munication (Rosenblith WA, ed). Cambridge, MA: MITPress.

Bullier J, Kennedy H (1987) Axonal bifurcation in thevisual systems. Trends Neurosci 10-205-210.

Conrad R, Klingelhofer J (1989) Dynamics of regional ce-rebral blood flow for various visual stimuli. Exp BrainRes 77:437-441.

CookJE (1991) Correlated activity in the CNS: a role onevery timescale? Trends Neurosci 14:397-401.

Cox DR, Miller HD (1980) The theory of stochastic pro-cesses, pp 272-336. New York. Chapman and Hall.

Durbin R, Mitchison G (1990) A dimension reductionframework for understanding cortical maps. Nature 343:644-647.

Edelman GM (1978) Group selection and phasic re entrantsignalling: a theory of higher brain function. In Themindful brain (Edelman GM, Mountcastle VB, eds), pp55-100. Cambridge, MA: MIT Press

Edelman GM (1987) Neural Darwinism, the theory of neu-ronal group selection. New York: Basic.

Foldiak P (1989) Adaptive network for optimal linear fea-ture extraction. In- Proceedings of the IEEE/INNS jointconference on neural networks, pp 401^05 New York:IEEE

286 Cortical Organization • Frlston et al.

Foldiak P (1990) Forming sparse representations by localanti-Hebbian learning. Biol Cybern 64:165-170.

Fox FT, Mintun MA (1989) Non-invasive functional brainmapping by change distribution analysis of averaged PETimages of H2"O tissue activity. J Nucl Med 30:141-149.

Fox PT, Raichle ME (1986) Focal physiological uncoup-ling of cerebral blood flow and oxidative metabolismduring somatosensory stimulation in human subjects. ProcNatl Acad Sci USA 83:1140-1144.

Fries W, Keizer K, Kuyprs HGJM (1985) Large layer VIcells in macaque striate cortex (Meynert cells) project toboth superior colliculus and prestriate visual area V5.Exp Brain Res 58:613-616

Frith CD, Friston KJ, Liddle PF, Frackowiak RSJ (1991)Willed action and the prefrontal cortex in man. Proc RSoc Lond [Biol] 244.241-246.

Frostig RD, Lieke EE, Ts'o DY, Grinvald A (1990) Corticalfunctional architecture and local coupling between neu-ronal activity and the microcirculation revealed by in vivohigh-resolution optical imaging of intrinsic signals. ProcNatl Acad Sci USA 87:6082-6086

Gaily JA, Montague PR, Reeke GN, Edelman GM (1990)The NO hypothesis: possible effects of a short lived, rap-idly diffusible signal in the development and function ofthe nervous system Proc Natl Acad Sci USA 87-3547-3551.

Glick SJ, King MA, Penny BC (1989) Characterization ofthe modulation transfer function of discrete filtered back-projection. IEEE Trans Med Imaging 8:203-213.

Hornik K, Kuan CM (1992) Convergence analysis of localfeature extraction algorithms. Neural Networks 5:229-240.

Jones DS (1979) Elementary information theory, p 152.Oxford: Clarendon.

Kijewski MF, Judy PF (1987) The noise power spectrumof CT images Phys Med Biol 32:565-575.

LinskerR (1988) Self organization in a perceptual networkComputer 21105-117

Livingstone MS, Hubel DH (1984) Anatomy and physiol-ogy of a color system in the primate visual cortex. J Neu-rosci 4:309-356.

Mesulam MM (1990) Large scale neurocognitive networksand distributed processing for attention language andmemory. Ann Neurol 28597-613.

Montague PR, Gaily JA, Edelman GM (1991) Spatial sig-nalling in the development and function of neural con-nections. Cereb Cortex 1:199-220

Mountcastle VB (1978) The mindful brain, pp 7-51 Cam-bridge- MIT Press.

Oja E (1982) A simplified neuron model as a principalcomponent analyzer J Math Biol 15:267-273.

Oja E (1989) Neural networks, principal components, andsubspaces. Int J Neural Systems 1:61-68.

Powell TPS (1981) Certain aspects of the intrinsic orga-nization of the cerebral cortex. In. Brain metabolism andperceptual awareness (Pompelano O, Aimone Marsan C,eds), pp 1-19. New York: Raven.

Rubner J, Schulten K (1990) Development of feature de-tectors by self organization: a network model. Biol Cy-bern 62:193-199.

Shepherd G, Koch C (1990) Introduction to synaptic cir-cuits. In- The synaptic organization of the brain (Shep-herd GM, ed), p 3. London: Oxford UP

Sherman, SM, Koch C (1990) Thalamus. In: The synapticorganization of the brain (Shepherd GM, ed), p 246London: Oxford UP.

Shipp S, Zeki S (1989a) The organization of connectionsbetween areas V5 and VI in the macaque monkey visualcortex EurJ Neurosci 1:309-332.

Shipp S, Zeki S (1989b) The organization of connectionsbetween areas V5 and V2 in the macaque monkey visualcortex. EurJ Neurosci 1:333-354

Townsend DW, Geissbuhler A, Defrise M, Hoffman EJ, SpinksTJ, Bailey D, Gilardi MC, Jones T (1992) Fully three-dimensional reconstruction for a PET camera with re-tractable septa. IEEE Trans Med Imaging, in press.

Zeki S (1971) Convergent input from the striate cortex(area 17) to the cortex of the superior temporal sulcusin the rhesus monkey. Brain Res 28:338-340.

Zeki S (1975) The functional organizations of projectionsfrom striate to prestriate visual cortex in the rhesus mon-key. Cold Spring Harbor Symp Quant Biol 40:591-600.

Zeki S (1990) The motion pathways of the visual cortex.In: Vision- coding and efficiency (Blakemore C, ed), pp321-345. Cambridge: Cambridge UP.

Zeki S, Shipp S (1989) Modular connections between areasV2 and V4 of the macaque monkey visual cortex. Eur JNeurosci 1:494-506.

Zeki S, Watson J, Lueck C, Friston KJ, Kennard C, FrackowiakRSJ (1991) A direct demonstration of functional spe-cialization in human visual cortex. J Neurosci 11:641 —649.

Cerebral Cortex May/June 1992, V 2 N 3 267