Self-Organizing Visual Cortex Model using Homeostatic Plasticity Mechanism

Veldri Kurniawan S0565942

Master of Science

School of Informatics University of Edinburgh

2006

i

Abstract Homeostatic plasticity is regulatory mechanism that modulates both synaptic connections and intrinsic properties of individual neuron in order to maintain stability of neuronal activity. This type of plasticity is considered to be an important complement to Hebbian plasticity, especially during learning and development process of nervous system. In this project, the role of homeostatic plasticity in self-organizing visual cortex model (LISSOM) will be investigated. Two computational model of homeostatic plasticity, gradient rule for intrinsic plasticity proposed by Triesch [27] and homeostatic synaptic scaling proposed by Sullivan and de Sa [26] will be applied to LISSOM model of V1. Several experiments will be conducted to evaluate the self-organization process and the organized map of the model with homeostatic mechanism. The result of this initial exploration indicates that homeostatic plasticity, particularly intrinsic plasticity, can be successfully integrated into LISSOM model of visual cortex to improve some of its behavior and make it more biologically realistic.

ii

Acknowledgement I would like to express my sincere gratitude to my supervisor, Dr. James A. Bednar for his help, guidance, support and inspiration during the course of this project. I would also like to thank Lewis Ng and Tikesh Ramtohul for fruitful discussions on many technical aspects related to this project.

iii

Declaration I declare that this thesis was composed by myself, that the work contained herein is my own except where explicitly stated otherwise in the text, and that this work has not been submitted for any other degree or professional qualification except as specified.

(Veldri Kurniawan)

iv

Table of Contents

Chapter 1 Introduction................................................................................................1 1.1 Objectives ............................................................................................................2 1.2 Document Structure .............................................................................................2

Chapter 2 Background ................................................................................................4 2.1 Biological Basis of Homeostatic Plasticity..........................................................4

2.1.1 Homeostatic Regulation in Biological Systems............................................4 2.1.2 Hebbian Learning and Problem of Stability .................................................5 2.1.3 Evidence of Homeostatic Regulation in Nervous Systems...........................7 2.1.4 Mechanisms of Homeostatic Plasticity.........................................................9

2.2 Computational Models of Homeostatic Plasticity .............................................13 2.2.1 Homeostatic Synaptic Scaling Model of Sullivan and de Sa......................14 2.2.2 Triesch’s Intrinsic Plasticity Learning Rule ...............................................16

2.3 The LISSOM Model ..........................................................................................18 2.3.1 Architecture of the LISSOM Model of V1 .................................................19 2.3.2 Threshold Adaptation in LISSOM..............................................................24 2.3.3 LISSOM without Shrinking Lateral Excitatory Radius..............................26

Chapter 3 Method & Parameters.............................................................................29 3.1 Choice of the Models .........................................................................................29 3.2 Experiment with Intrinsic Plasticity Applied to LISSOM Models....................30

3.2.1 Intrinsic Plasticity in Simplified LISSOM Network...................................30 3.2.2 Sensitivity of Intrinsic Plasticity Parameter Settings..................................33 3.2.3 Intrinsic Plasticity Applied to LISSOM with LGN ON/OFF Channel.......34 3.2.4 Intrinsic Plasticity Applied to LISSOM without Shrinking Lateral Excitatory Radius.................................................................................................35

3.3 Experiment with Homeostatic Synaptic Scaling Applied to LISSOM Models.35

Chapter 4 Results.......................................................................................................37 4.1 Intrinsic Plasticity Applied to Simplified LISSOM Network............................37

4.1.1 Receptive Fields and Orientation Maps......................................................37 4.1.2 Average Firing Rate, Firing Rate Distribution, and Speed of Threshold Adaptation............................................................................................................44

4.2. Sensitivity of Intrinsic Plasticity Parameter Settings........................................48 4.2.1. Initial Threshold Values.............................................................................49 4.2.2. Intrinsic Plasticity Learning Rate...............................................................52

v

4.2.3. Target Average Firing Rate .......................................................................54 4.3. LISSOM with LGN ON/OFF and Intrinsic Plasticity ......................................55 4.4 Intrinsic Plasticity Applied to LISSOM with no Shrinking Lateral Excitatory Radius ......................................................................................................................58 4.5. Homeostatic Synaptic Scaling ..........................................................................61

Chapter 5 Discussion and Future Works.................................................................66 5.1. Discussion .........................................................................................................66

5.1.1. Intrinsic Plasticity and LISSOM Models...................................................66 5.1.2. Synaptic Plasticity and LISSOM Models ..................................................71

5.2. Future Works ....................................................................................................71

Chapter 6 Conclusions...............................................................................................73 Bibliography ...............................................................................................................74

1

Chapter 1 Introduction

Homeostatic plasticity is a recently proposed regulatory mechanism that modulates both synaptic connections between neurons and intrinsic electrical properties of individual neurons in order to maintain stability of neuronal activity in spite of change during learning by Hebbian mechanism. This kind of plasticity is considered to be an important complement of Hebbian plasticity, especially during learning and development process of nervous system. [13, 29, 30, 31, 36] Self-organizing visual cortex model that are going to be used in this project is LISSOM (Laterally Interconnected Synergetically Self-Organizing Map). LISSOM is one of the most biologically realistic, laterally connected self-organizing map model of the visual cortex. This model can demonstrate wide variety of phenomena occurring in visual system including input-driven self-organization, patchy connectivity, ocular dominance, orientation selectivity, tilt-aftereffect, contour integration, preference for faces in newborn, and more. [2, 3, 4, 18, 19, 20, 23] An automatic threshold adaptation is one of the desirable extensions to the LISSOM model for enhancing its reliability and making it more relevant biologically [19, chapter 17]. In current LISSOM model, activation thresholds (the upper and lower bound of neuron’s activation function) should be tuned manually by hand through trial and error. Besides time consuming for the modeler, this process also likely to be subjective. Comparison of experiment results under different conditions is also difficult. Moreover, different types of input pattern will require different threshold settings, depending on how strongly it activates the network. High-contrast abstract patterns might require lower threshold values as opposed to low-contrast natural images which require higher threshold settings. LISSOM model currently use Hebbian learning rule combined with divisive weight normalization to prevent unconstrained synaptic weight growth. While this normalization rule is useful in practice, there is little evidence from biological experiment that confirm the existence of such mechanism. Recent experiments by Turrigiano et. al. [29, 30] suggest that neurons might use a kind of homeostatic plasticity to regulate its firing rate by multiplicatively scaling all synapses strength depending on recent history of inputs. The Topographica cortical map simulator [2] will be used for testing and implementing various homeostatic mechanisms in the LISSOM model. Topographica

2

is a large-scale, general purpose simulator for topographic map formation in the brain. As a testbed for initial testing of various homeostatic plasticity mechanisms, a very simplified model of visual system which consists only of one input layer and one output layer will be set up in Topographica. After the implementations have been verified to be correct, more complete visual models will be tested using the new homeostatic plasticity rules. It is expected that with addition of homeostatic plasticity, the original functionality of LISSOM such as ability to self-organize and form orientation selectivity will still be working well. Several aspect of stability will be evaluated such as sensitivity of threshold setting, stability of neurons activity over long period of time, and long term stability of the weight values. The effect of homeostatic plasticity mechanism (if any) on the original functionality of LISSOM (such as self-organization, orientation and direction selectivity, and other features) will be also investigated.

1.1 Objectives The objective of this project is: 1) to extend the LISSOM model to use automatic threshold adaptation based on homeostatic plasticity mechanism. A mechanism of homeostatic threshold adaptation will be implemented and tested using Topographica simulator [2]. If this mechanism can be successfully implemented, the need of manually tuning the threshold function can be avoided and therefore make the simulator more reliable and more applicable in real situations where the visual system receives inputs from natural scenes. 2) to replace the weight normalization rule used in Hebbian learning process in LISSOM with a form of homeostatic synaptic scaling, which is more relevant biologically. It is hoped that by addition of automatic threshold adaptation and homeostatic synaptic scaling, the resulting system will be less sensitive to threshold settings, have more stability to cope with different inputs, and also more biologically plausible.

1.2 Document Structure Chapter 2 presents the background information relevant to the project concerning the biological evidence and mechanism of homeostatic plasticity that recently found in nervous system, followed by overview of computational model of homeostatic plasticity. Lastly, the architecture of basic LISSOM model of V1 will be described.

3

Chapter 3 describes the method and parameters of the experiments that are conducted to investigate the behavior of LISSOM models with homeostatic plasticity mechanism. It starts by explanation of the reason behind the choice of the models, and then followed by two sections describing detailed experiment with intrinsic plasticity and homeostatic synaptic scaling. Chapter 4 presents the results obtained from the experiments of applying homeostatic plasticity mechanism into LISSOM. The resulting self-organized orientation maps and other related plots that show the detailed properties of the network during self-organization will be examined. Chapter 5 presents a discussion and analysis of the experiment results and suggestions for future works related to this topic. Chapter 6 presents the conclusions.

4

Chapter 2 Background

This chapter presents a literature review of homeostatic plasticity and LISSOM model. The first part explains the biological evidence and mechanism of homeostatic plasticity that recently has been found in nervous system. The second part gives an overview of some computational model of homeostatic plasticity and detailed exposition of the model that are going to be used in this project. The third part explains the basic LISSOM model of V1.

2.1 Biological Basis of Homeostatic Plasticity

2.1.1 Homeostatic Regulation in Biological Systems

Homeostasis is a regulatory mechanism of an open system to maintain its internal equilibrium by means of continuous dynamic adjustments. The term was coined by Walter Cannon in 1932, derived from the Greek words ‘homoios’ which means ‘similar’/‘same’ and ‘stasis’ which means ‘standing still’. He suggested a new term homeostatic instead of using the term equilibria which is commonly used in simple control systems and explains the reason behind this in his seminal book, The Wisdom of the Body (1932):

The constant conditions which are maintained in the body might be termed equilibria. That word, however, has come to have fairly exact meaning as applied to relatively simple physico-chemical states, in closed systems, where known forces are balanced. The coordinated physiological processes which maintain most of the steady states in the organism are so complex and so peculiar to living beings – involving, as they may, the brain and nerves, the heart, lungs, kidneys and spleen, all working cooperatively – that I have suggested a special designation for these states, homeostasis. The word does not imply something set and immobile, a stagnation. It means a condition – a condition which may vary, but which is relatively constant.

Canonn’s insight into the nature of stability behind biological systems originated from his observation that complex self-righting adjustment (self-organization) might be the way used by an organism to regulate its internal mechanism in face of tremendous amount of changes and perturbations. He proposed four tentative general features of homeostasis in his book:

5

1. Constancy in an open system, such as our bodies represent, requires mechanisms that

act to maintain this constancy. 2. Steady-state conditions require that any tendency toward change automatically meets

with factors that resist change. An increase in blood sugar results in thirst as the body attempts to dilute the concentration of sugar in the extracellular fluid.

3. The regulating system that determines the homeostatic state consists of a number of

cooperating mechanisms acting simultaneously or successively. Blood sugar is regulated by insulin, glucagons, and other hormones that control its release from the liver or its uptake by the tissues.

4. Homeostasis does not occur by chance, but is the result of organized self-government.

There are many known examples of homeostatic regulation in living organism such as: regulation of body temperature, regulation of the amounts of water and minerals in the body, regulation of blood glucose level, regulation of blood pressure, and the recently proposed regulation of neural activity. Homeostatic regulation mostly occurs through negative feedback mechanism, in which the system reacts in such a way to reverse the direction of change and thus maintain stability. One examples of negative feedback is Thermoregulation. When body temperature falls, receptors in the skin and hypothalamus register a change and send signals to part of the brain which controls the temperature. The brain will then respond by sending signals to other parts of the body to increase the temperature.

2.1.2 Hebbian Learning and Problem of Stability

Nervous systems are highly plastic. During the development stage and lifetime of an individual, neurons can dramatically change in shape/size, synaptic connections, and their intrinsic properties. Neuron’s plasticity is important for learning and adaptation to the continuously changing environment. The outcome of many years of research in neuroscience suggests that plasticity and learning are facilitated by modification in the number and strength of synaptic connections and also by alternations in the internal properties of neurons (such as ion channels). [13, 29, 30, 31] The most celebrated theory of activity-dependent plasticity, Hebbian Learning, has dominated neuroscience research since its introduction by Donald Hebb in 1949. Hebbian rule states that synaptic strength between pre-synaptic and post-synaptic

6

neurons will be strengthened when both neurons fire in correlation and weakened when they fire out of correlation. This rule is extremely powerful in explaining the phenomena of information storage and learning in neural system. However, because Hebbian plasticity relies on positive feedback mechanism, it is also highly unstable and could drive neurons to become hyper-activated or silenced altogether. Suppose we have a hypothetical pre-synaptic neuron connected with a bunch of synapses to one post-synaptic neuron. Consider the case of positive correlation in which the firing of pre-synaptic neuron caused potentiation of post-synaptic neuron. According to Hebbian rule, synapses between these two neurons should be strengthened, which in turn would allow the pre-synaptic neuron to drive the post-synaptic one more strongly. The increase in post-synaptic activation would tend to strengthen the synapses even more, which lead to even more activation and so on, until it reach maximum saturation point. The reverse case with uncorrelated input is also similar. The synapses strength would be weakened progressively and selectivity of post-synaptic neuron falls to zero. Computational modelers who were working with Hebbian rule in the early times realized that this stability problem can prevent the application of this rule in larger models so they incorporated some kind of normalization rule to limit the growth of synaptic strength within bound. Although simple form of normalization usually works well in practice, there is no biological evidence that support such operation in real organism. This early form of ad-hoc normalization is practically a simplification of more complicated homeostatic regulation that counterbalances Hebbian plasticity [31]. Turrigiano and Nelson [31] give a very helpful illustration of the importance of regulating excitability of neural circuit in feedforward neural network architecture (figure 2.1). Suppose there are several layers of networks connected in hierarchical fashion by feedforward connections from the input layer to several higher layers above. We can view this simplified network topology as resembling connection of neurons in primate visual systems starting from photoreceptors, bipolar cells, ganglion cells, LGN, V1, V2, and so on. Input pattern is received by neurons in the first layer and propagated through synaptic connections to the second layer neurons. From the second layer, the signal is forwarded to the third layer neurons, and so on up to the highest layer. In this simplified model, it is obvious that the gain of signal transmission from one layer to the next should be close to one in order to retain the specificity of the pattern. If the gain is higher than one, the signal will be continuously amplified every time it passed through one layer of neurons, and at the highest layer the firing rate will reach saturation and information about the pattern is lost. If the

7

gain is lower than one, the signal will be weakened over time and fail to reach the highest layer neurons. This example shows that the ability of neurons to maintain stable/constant rate of activity is very important for conserving the information that is being propagated through the network of neurons.

Figure 2.1. This figure illustrates the problem of propagating information in feedforward network. (a) Abstract feedforward network consisting of 5 layers of neurons. (b) The graph shows expected firing rates of the red neuron in each layers when given sinusoidal input. If the activity is subsequently amplified when passing from lower layer to higher layer, in the end it will saturates and the information about the input is lost. (c) In this case, the activity is subsequently weakened in each layer, and fails to propagate to highest layer. (d) When every neuron maintain stable firing rate, the signal can be transmitted up to the highest layer. (adapted from [31])

2.1.3 Evidence of Homeostatic Regulation in Nervous Systems

Recently, an increasing amount of experimental evidence suggests that several different kinds of homeostatic regulation are being exercised by nervous systems in order to maintain neural excitability within a normal physiological range [12, 29, 31]. Nervous System face tremendous amount of perturbations in the early development stage and during the life of an organism, including changes in cell structure and size, fluctuations in synaptic strengths, and innervations. In face of such distress, neurons can still maintain stable level of excitation by regulating several diverse properties such as synaptic size, synaptic strength, synaptic number, and ion channels that controls neuronal excitability [13, 12, 31]. The results of many experiments suggest that the ability of neurons to maintain their excitation levels might implies the capability of monitoring their own internal activity and transduce this information into regulated changes in synaptic and ion channel

8

properties. This kind of monitoring and feedback mechanism is thought to be similar with other homeostatic regulation found in other biological systems and may provide a basis for understanding how stability in neural system is maintained [12]. Experimental evidence of homeostatic regulation has been found in systems ranging from invertebrate neuromuscular junction (NMJ) to synapses in the vertebrate cortex. During each of these experiments, the neurons were artificially perturbed to alter the innvervation, synaptic function, and cell growth. In the recent study which involves advanced genetic manipulations, it has been shown that homeostatic regulation plays role in Drosophila neuromuscular junction to keep neuromuscular transmission relatively constant in the midst of altered synaptic properties. In this experiment, the number of post-synaptic receptors of Drosophila NMJ is impaired or reduced genetically. By doing this, we expect abnormal muscle depolarization, but contrary to the expectation, the muscle can still depolarize normally because impaired post-synaptic receptor is precisely compensated by increased presynaptic transmitter release. A similar homeostatic regulation is also observed at neuromuscular junction of mouse with genetically modified post-synaptic acetylcholine receptor. The reduction of post-synaptic receptor is precisely compensated by increase in transmitter release so the muscle can still function normally. [12, 29, 31] Another evidence of homeostatic response was observed in lobster stomatogastric ganglion (STG) during in vitro experiment. In its natural place, STG neurons fire in bursts due to both synaptic inputs and intrinsic conductance. Remarkably, when STG neurons are isolated from their synaptic circuit and placed in a cell culture, they can regenerate their bursting pattern in absence of any synaptic input by coordinating changes in its voltage-gated channel densities. This homeostatic adaptation can be stopped by supplying rhythmic pattern similar with neuron’s normal synaptic inputs or by preventing Ca2+ entry to the neuron. Several theoretical and experimental works suggest that chemical signals such as Ca2+ play important role in balancing intrinsic electrical properties of neurons in homeostatic way. [12, 29] Many experimental results also confirm existence of homeostatic regulation in central nervous system (CNS). Neurons in CNS are much more complex compared to NMJ or STG neurons. These neurons receive inputs from thousands of other neurons and carry out fluctuating signal over short time scale to transfer information. Over longer time scale these neurons should also regulate themselves homeostatically to prevent runaway excitation or die out. One evidence of this regulation is found in the

9

experiment of blocking some inputs to neurons in invertebrate networks and vertebrate spinal cord. Initially the neurons cease to have spontaneous activity but over time the activity recovers to normal state. Another experiment with cortical pyramidal neurons using cortical cultures also suggests that these neurons can maintain a set-point firing rate in the face of changing synaptic input. Similarly, regulation of firing rates was also found when Kir potassium channel was expressed in individual cultured hippocampal neurons. This channel create a hyperpolarizing current that lowers firing rates in the beginning, but over time firing rates return to normal state even with continued expression of the channel. This result demonstrates that homeostatic regulation of firing rate can take place at the level of individual neuron. [31]

2.1.4 Mechanisms of Homeostatic Plasticity

The exact mechanisms of homeostatic regulation in nervous systems remain to be investigated in details. Current evidences also indicate the possibility that various different types of regulation might actually taking place at the same time. Homeostatic regulation not only occurs at the level of individual neuron but also involve complex intercellular signaling. Some hypotheses even suggest that it may also take place at larger circuit or system level. [12] Several different mechanisms that promote homeostatic plasticity in central nervous system have been proposed in recent years. These mechanisms include multiplicative synaptic scaling, the regulation of intrinsic excitability of neurons, the BCM model, and the change in the brain derived neurotrophic factor (BDNF) [13, 14, 29]. Experiments on these mechanisms indicate that neurons have an ability to utilize their own activity as a feedback signal to modify intrinsic excitability and total synaptic strength with the aim of maintaining long term stability. However, the exact activity that is being regulated is still not clearly defined. It could be average firing rate, average calcium concentration, or some statistical measure of network activity (such as maximization of information transfer) [30]. 2.1.4.1 Multiplicative Synaptic Scaling One way to maintain neuron stability is using synaptic scaling mechanism that preserves the average neurons activity level by multiplicatively increasing or decreasing the strength of all synaptic connections, depending on previous activities. When the neuron firing rate increases, the strength of all excitatory connections will

10

be scaled down, and conversely when the neuron firing rate fall, they will be scaled up [13, 29, 30]. Experimental evidences of synaptic scaling can be found in a work on neuromuscular junction and also from cultured networks of central neurons. More recent experiments also found similar behavior in living brain. [13] One important difference between synaptic scaling with much studied form plasticity such as LTP/LTD is the speed of change. LTP/LTD can induce change in synaptic strengths quite rapidly, while synaptic scaling is much slower, requiring hours or days of modified activity to produce measurable changes. This also shows that neurons integrate average activity over long period of time, instead of responding to moment-to-moment fluctuations. If homeostatic regulation occurs very rapidly, it would destroy signal variations that are used for information transmission. Conversely, if it is too slow, it will not have enough power to counterbalance destabilizing effect caused by other type of plasticity. The exact rate of this balance is still unknown. [13, 30, 31] The other important distinction between LTP and synaptic scaling is that instead of modifying specific-synapse individually, synaptic scaling will scale all synaptic connections up or down by the same multiplicative factor. This characteristic ensures that modification of synapses will still preserve the relative difference between individual synaptic inputs, which means that information learned by LTP/LDP plasticity such as Hebbian rule will be kept intact while at the same time the neurons can maintain stable firing rate. Therefore we can see that Hebbian plasticity and synaptic plasticity works together complementary to encode information and regulating overall neurons activity within acceptable limit [13, 29, 30]. The figure below illustrates the effect of synaptic scaling on the number of receptors at each synapses.

Figure 2.2. Illustration of multiplicative scaling effect on the number of receptor in synapses. The left picture shows two synapses with equal strength. When one presynaptic neuron is potentiated, the corresponding synapse in postsynaptic terminal will have an increase in the number of receptors which result in increased firing rate. After multiplicative scaling (right picture), the firing rate returns to

11

normal again because removal of some AMPA receptors in postsynaptic terminal. However, the relative difference between the two synapses is preserved. (adapted from [31])

How can a neuron regulate its own excitability ? We might suspect that in order to do this task, neuron should have some mechanism to monitor its own history of activation and regulate the number of ion channels accordingly. But where is this activity monitor located. One possible answer is that activity monitors are located in the cell soma and they are capable to integrate action-potential activity over time. This central location would allow genetic transcriptional response at the nucleus during homeostatic signaling. The signal could then be transmitted to all synapses. The second possible answer suggests that homeostatic mechanism could be locally managed post-synaptically at each synapse. Every synapse would regulate its own excitability using information about sum total excitation received through action potential back propagation. [12] 2.1.4.2 Intrinsic Plasticity Another form of homeostatic plasticity which can promote stability in neurons activity is Intrinsic Plasticity. Many experimental evidences suggest that somehow neurons can also regulate their intrinsic electro-chemical properties (which govern the way they integrate synaptic inputs and produce spikes) according to previous history of their activations. These intrinsic properties are influenced by the amount of intrinsic ion channels and not by channels associated with synaptic terminal. Changing the quantity and distribution of these channels can influence the neural excitability, synaptic integration, firing rate and firing pattern, and synaptic plasticity. Experimental evidences of intrinsic plasticity have been found in the culture of neocortical cells and also demonstrated in the functioning of cerebellar granule cells (neurons that coordinate movements). [13, 31, 36] Molecular mechanism of intrinsic plasticity might be related to chemical signaling process (such as intracellular Ca2+ concentration) that regulate the balance of neuron’s outward and inward conductances depending on the level of excitation (figure 2.3). When the neuron activity is higher than the target activity, Ca2+ flows into the cell and raises intracellular Ca2+ concentration which in turn will change the proportions of ion-channels and adjust conductances to reduce the activity level. On the other hand, when the neuron activity is low, intracellular Ca2+ concentration is reduced and the proportions of ion-channels and conductances will be regulated to increase the level of activity. [29]

12

Figure 2.3. Schematic showing regularization of internal Ca2+ concentration depending on the activity of the neuron (adapted from [29])

Figure 2.4. Intrinsic plasticity effect can be seen as shifting the F/I curve to the left or right depending on the neuron’s level of activity. (figure adapted from [13])

The figure above shows the typical F/I curve which relates the synaptic input current with the neuron’s firing rate. This figure illustrates that intrinsic plasticity works by changing the way the input is transformed into firing rate, instead of modifying synaptic connections. The curve in bold shows the initial condition of the neuron. In this condition, if the given input is too low, the neuron will not fire, and if it is too high it will reach saturation. The neuron will only be selective in the middle range between these two states. When the average neuron’s activity over long period of time became very low, intrinsic plasticity mechanism will increase the neuron’s excitability and effectively translate the F/I curve to the left. Conversely, when the average neuron’s activity became too high, plasticity mechanism will make the neuron less excitable and effectively translate the F/I curve to the right. The shifting of F/I curve will prevent the neuron to fall into complete silent or complete saturation, even when the input became very weak or very strong. [13] 2.1.4.3 Unifying Framework It would be very interesting to see one unifying framework which explains every kinds of homeostatic plasticity in one general basic mechanism. Results from current experiments have shown that both synaptic scaling and intrinsic plasticity seems to be governed by common underlying subcellular pathways which is the activity-

13

dependent brain-derived neurotrophic factor (BNDF) [14]. However, detailed mechanism of how BNDF operates is still not well understood. Another interesting question to be asked is about connection between homeostatic plasticity and Hebbian plasticity in deeper subcellular level. Whether both of them are governed by the same biochemical mechanism such as BDNF or not is still an open question. [13] Further issues concerning homeostatic plasticity is the question about what activities are exactly being regulated by this mechanism. Current experiments suggest that the number of regulated activities seems to be very diverse. Usual form of homeostatic plasticity deals with regulation of firing rates and maintaining synaptic strengths within bounds. However there are some evidences which show that other kinds of activity are also being regulated, such as the regulation of firing pattern in stomatogastric ganglion (STG) of decapod crustaceans [31]. Another experiment on organotypic rat spinal culture also demonstrates that regulation is not only present in individual neuron but also take place in larger network-wide activity. [13]

2.2 Computational Models of Homeostatic Plasticity Several theoretical and computational models of homeostatic plasticity mechanisms have been proposed by some researchers based on the results and evidences from biological experiments. Most of the models that exist in current literature are very detailed low level molecular models. Some of them explain the phenomenon of homeostatic in the level of single neuron using voltage-gated ion-channel model [24] and some others use integrate-and-fire neurons model [17, 32, 34]. For the purpose of this project, we need more abstract homeostatic plasticity models which can be applied to rate-based neurons model. Quite recently, several models like this have been emerged. Sullivan and de Sa, in their recent papers [25, 26], proposed one possible implementation of homeostatic synaptic scaling (together with Hebbian learning) in one dimensional Self-Organizing Maps neural network. They applied multiplicative synaptic scaling by replacing normally used weight normalization scheme in Hebbian rule with normalization based on the average activity of the neurons over period of time. The result of the experiment is quite promising as the network is able to self-organize and also regulate Hebbian learning from overrun at the same time. Moreover,

14

they also reported that the network can still maintain stability of firing rate in the face of network growth and neurons die-out. Another interesting model which implements intrinsic plasticity was recently proposed by Triesch [27]. Basically, the author use information theory and apply the hypothesis that neurons might use intrinsic plasticity to obtain an exponential distribution in firing rates and therefore maximize the entropy. Maximization of entropy was thought to be beneficial because it allows neurons to process maximum amount of information using minimum amount of energy. Triesch develop a gradient descent rule for adjusting the shape and position of the sigmoid F/I curve. The gradient rule is derived by measuring Kullback-Leiber divergence between distribution of current firing rate and desired exponentially distributed firing rate. This model was applied to a continuous activation model neuron and proven to be successful in producing approximately exponential firing rate distribution. In another publication [28], Triesch reported that precursor of this intrinsic plasticity model which uses simpler proportional-control-law rules also works similarly. More recently, Triesch and Butko [6, 7] presented the result of applying this model together with SOM and Hebbian Learning to learn V1-like maps in visual cortex. Model of homeostatic plasticity based on well known BCM (Bienenstock, Cooper, Munro) model have also begin to emerge. BCM model was originally proposed to supplement Hebbian learning with competition mechanism, but experiments show that it can also promote stability in neurons while learning using Hebbian rule. Stability in BCM model is achieved by sliding the threshold of depression/potentiation curve depending upon the history of neuron’s activity. More recently, a model called UCM (Unified Calcium Model) has been proposed by Shouval, et. al. [21] as the common mechanism of various bi-directional synaptic plasticity. A neuron that uses this model together with BCM has been shown to develop input selectivity and stability. [8, 34, 35].

2.2.1 Homeostatic Synaptic Scaling Model of Sullivan and de Sa

Sullivan and de Sa [25, 26] proposed an alternative normalization rule based on homeostatic synaptic scaling to replace commonly used weight normalization in Hebbian learning. In order to test this rule, they set up linear SOM (Self Organizing Map) network with Hebbian rule for learning and homeostatic weight normalization, then trained it with series of random Gaussian input patterns. The input and output

15

neurons of SOM are arranged in ring arrangement to eliminate edge effect. At each iterations, the input vector, x , is presented to the network by multiplication with feedforward connection weights, WFF. This feedforward activity is then passed through Mexican hat shape convolution kernel to emulate the effect of lateral connections.

( )FFy f g W x= ∗ (2.1)

Where f(x) = max (0, x), g is Mexican hat shape kernel, and y is the output vector. In

this version of SOM, lateral connection is assumed to be fixed but feedforward connections are adapted with Hebbian learning rule.

1t t t tij ij j iw w x yα−= + (2.2)

Where α is learning rate, xj is presynaptic activity, yi is postsynaptic activity, and tijw is weight value from input neuron j to output neuron i before weight normalization.

Homeostatic normalization based on recent activity of the neurons is used for limiting the growth of the weights.

, target1

target, 1

t tij avg it t

ij i Nti

w A Aw ActivityNorm

AActivityNormβ+

⎛ ⎞−= = + ⎜ ⎟⎜ ⎟

⎝ ⎠ (2.3)

Atarget is preferred target activity level for the neurons, Aavg,i is the average of recent activity of neuron i, and Nβ is homeostatic learning rate. Aavg,i is computed as a

running average of the recent activity of the neuron.

1, ,(1 )t t t

avg i c i c avg iA y Aβ β −= + − (2.4)

The parameter Cβ determines the speed over which previous activity of the neuron is

averaged. This normalization rule can be seen as a simplified abstraction of some internal homeostatic regulation process involving some intercellular chemicals such as calcium concentration. Since the detailed mechanism of this molecular process is not yet known, this model uses the running average firing rate and desired target firing rate directly as a substitute.

16

A linear SOM network with 150 inputs and 15 outputs was used to test this homeostatic rule [26]. After the network was trained using randomly positioned 1-D Gaussian pattern, the map was computed by presenting every possible input to the network and selecting one neuron with the highest activation as the winner for each input. The resulting self-organized map is good because it forms regular topographic ordering of the input position (figure 2.5, left). The average activity of the neurons keeps oscillating around the target value during early training iterations but slowly dampens over time and finally converged to predefined average firing rate (figure 2.5, right).

Figure 2.5. The resulting self-organized map and average neuron activities during self-organization. (adapted from [26])

Sullivan and de Sa also performed an experiment of changing the network architecture in the middle of learning process to simulate synapses proliferation during cortical development [26]. Initially, they started with 75 input neurons and trained the network until average activity is settling down. After that, another 75 input neurons are added to the network and the learning process continues until the activity reach stable point. Then 75 neurons are removed again. During this abrupt modification, they showed that the network can still retain the continuity of the map and homeostatic mechanism can regulate the firing rate within bound.

2.2.2 Triesch’s Intrinsic Plasticity Learning Rule

It has been shown in biological experiment that neurons in visual cortex exhibit sparse exponential firing rate distribution when presented with natural scenes [1]. Exponential distribution of firing rate is believed to be an important aspect of the neurons in order to achieve maximum information processing capacity with limited

17

amount of energy. The reason behind this is because exponential distribution has the highest entropy among all distribution of a non-negative random variable with a fixed mean. Initial experiment conducted using Hodgkin-Huxley model neuron suggests that intrinsic plasticity mechanism might plays an important role in achieving approximately exponential distribution firing rate [24]. In that experiment, unsupervised learning rule that adapts membrane conductance of voltage gated channels to match the desired exponential firing rate distribution was used. More recently, Triesch proposed similar learning rule that can be applied to more abstract rate-based model neuron. The learning rule was initially formulated as proportional control laws based on low order moments of a neuron’s firing rate [28]. Later on, a gradient rule based on Kullback-Leiber divergence (KL-divergence) was derived, and proved to be similar with the previous learning rule [27]. Triesch’s intrinsic plasticity learning rule works by directly adapting the neuron’s activation function to achieve approximately exponential firing rate distribution. The activation function used in this rule is the sigmoid function:

1( )1 exp( ( ))

y g xax b

= =+ − +

(2.5)

y is the neuron’s output (firing rate) and x is total synaptic current received by the neuron. a determines the slope/gradient of the sigmoid and b determines its position. The distribution of firing rate, ( )yf y is related to the distribution of x, ( )xf x , by

equation: ( )( ) x

yf xf y y

x

=∂∂

(2.6)

The gradient rule was constructed by utilizing the KL-divergence between the current firing rate distribution (denoted by ( )yf y ) and the desired exponential distribution,

exp (1/ )exp( / )f yµ µ= − :

18

exp( )

( || ) ( ) log(1/ )exp( / )

( ) log( ( )) ( ) log

1( ) ( ) log

yy y

y y y

f yD d f f f y dy

y

yf y f y dy f y dy

H y E y

µ µ

µµ

µµ

⎛ ⎞≡ = ⎜ ⎟−⎝ ⎠

⎛ ⎞= − − −⎜ ⎟

⎝ ⎠

= − + +

∫

∫ ∫ (2.7)

Where D is KL distance, H(y) is the entropy of y, E(y) is its expected value, andµ is

desired average firing rate level. In order to bring the current firing rate distribution as close as possible to the exponential distribution, the value of D should be minimized. This can be achieved by constructing a stochastic gradient descent rule that slowly adapts the parameters of sigmoid function, a and b. By taking partial derivatives of D with respect to a and b, the following rule was obtained (the detailed derivations is available in [27]).

2

2

1 1 12

1 11 2

a x xy xya

b y y

ηµ µ

ηµ µ

⎛ ⎞⎛ ⎞∆ = + − + +⎜ ⎟⎜ ⎟

⎝ ⎠⎝ ⎠⎛ ⎞⎛ ⎞

∆ = − + +⎜ ⎟⎜ ⎟⎝ ⎠⎝ ⎠

(2.8)

Where η is intrinsic plasticity learning rate.

Application of this rule to single model neuron was proven to be successful in bringing the neuron’s firing rate close to exponential distribution [27]. The rule also has been successfully applied to solve the “bars” problem, which is a standard non-linear ICA problem [6, 28]. On larger network scale, this rule has been applied to model the self-organization process of V1 oriented maps using SOM-like network and Hebbian learning. It was shown that after being trained with natural image patches, the network can form oriented, bandpass, and localized receptive fields that ordinarily found in V1 maps [6, 7].

2.3 The LISSOM Model LISSOM (Laterally interconnected synergetically self-organizing map) is one of the most biologically realistic computational map model of visual cortex that has been proven to be successful in modeling the structures and processes that take place in V1 and several higher visual areas [18, 19, 20, 23]. This model has been successfully

19

used to demonstrate wide variety of phenomena occurring in visual cortex including input-driven self-organization, patchy lateral connectivity, ocular dominance, orientation selectivity, tilt-aftereffect, contour integration, preference for faces in newborn, and more. [3, 4, 10, 19, 20]. The basic component of LISSOM model is a two-dimensional sheet of neural units that represent the vertical columns of the cortex. Each unit in the sheet is an abstraction of six layers of neurons inside the vertical column of real biological cortex. Such an abstraction is reasonable because neurons in a column generally respond to similar input. Each cortical unit receives input from neurons in LGN ON/OFF channels and the LGN neurons in turn receive inputs from retina. The cortical neurons are connected with lateral excitatory and lateral inhibitory connections that are adapted throughout self-organization. These lateral connections play an important role in the development of orientation map and in mediating functions like illusions and perceptual grouping. Each unit in LISSOM model is modeled after rate-based neuron model. The response of each neural unit is computed by taking a weighted sum of their input and squashing it through a sigmoid function. The afferent and lateral connections of the cortical units are all adapted through Hebbian learning with divisive normalization. [18, 19, 20, 23]

2.3.1 Architecture of the LISSOM Model of V1

For the purpose of experiments conducted in this project, basic LISSOM model of V1 will be used. Figure 2.6 shows the architecture of basic LISSOM model which consists of one retinal receptors, LGN ON/OFF channels, and V1. The retina is represented as RxR array of photoreceptor cells that receive grayscale inputs. LGN ON and LGN OFF consist of LxL array of neurons corresponding to ON/OFF processing in retinal ganglion cells and the LGN. In real biological visual pathway, LGN ON and OFF are not always physically separated, however they are divided into two separate sheets for simplicity. Each LGN unit receives input from a local anatomical receptive field in the retina so the topographic organization of the LGN is the same as the retina. V1 is NxN array of cortical neurons that receive afferent input from LGN ON/OFF and also have reciprocal excitatory and inhibitory lateral connections with other neurons. [19]

20

Figure 2.6. Architecture of basic LISSOM model. The model is a hierarchy of two dimensional array of

neural units, starting from the retinal receptors at the bottom, two sheets that represent LGN ON and OFF channels, and finally a single sheet representing V1.

The LGN ON/OFF neurons receive input from retina through afferent connections. The strength of this afferent connections are not adapted through any learning, instead they are set permanently according to the standard difference-of-Gaussian (DoG) model of LGN receptive fields. The weights for the ON-center units are computed by equation:

2 2 2 2

2 2

, 2 2 2 2

2 2

( ) ( ) ( ) ( )exp exp

( ) ( ) ( ) ( )exp exp

c c c c

c sxy ab

c c c cuv uv

c s

x x y y x x y y

Lu x v y u x v y

σ σ

σ σ

⎛ ⎞ ⎛ ⎞− + − − + −− −⎜ ⎟ ⎜ ⎟

⎝ ⎠ ⎝ ⎠= −⎛ ⎞ ⎛ ⎞− + − − + −− −⎜ ⎟ ⎜ ⎟

⎝ ⎠ ⎝ ⎠∑ ∑

(2.9)

where Lxy,ab is the weight from retinal receptor (x,y) in the receptive field of an ON-center neuron (a,b) with center (xc,yc). cσ defines the width of the central Gaussian

and sσ the width of the surround Gaussian. The OFF-center weights are the negative

of the ON-center weights. Using this weights setting, the ON-center of LGN units will respond to light areas surrounded by dark, and the OFF-center units to dark areas surrounded by light [19]. After receiving input from retina, LGN units compute their response as a squashed weighted sum of the total received activation:

,ab L xy xy abxy

Lξ σ γ χ⎛ ⎞

= ⎜ ⎟⎜ ⎟⎝ ⎠∑ (2.10)

where abξ is the response of LGN ON/OFF unit (a,b) , xyχ is the activation of retinal

unit (x,y) in the receptive field of (a,b), Lxy,ab is the afferent weight from (x,y) to (a,b),

21

and Lγ is a constant scaling factor. σ is a piecewise-linear squashing function (figure

2.7) :

0( ) ( ) /( )

1

l

l u l l u

u

ss s s

s

θσ θ θ θ θ θ

θ

≤⎧⎪= − − < <⎨⎪ ≥⎩

(2.11)

Figure 2.7. Piecewise-linear squashing function that approximates the sigmoid activation function.

V1 units are connected to the LGN ON/OFF units by excitatory afferent connections. Besides afferent connections, V1 neurons also have short range lateral excitatory connections and long range lateral inhibitory connections to other neurons. All of these connections are initialized with normalized random value in the beginning and then modified by learning from the inputs presented to the network. The initial response of V1 neurons is computed by taking weighted sum of activations received from LGN and passing it through sigmoid function:

, ,

(0) ( )

ij A ab ab ij ab ab ijab ON ab OFF

ij ij

s A A

s

γ ξ ξ

η σ

∈ ∈

⎛ ⎞= +⎜ ⎟

⎝ ⎠

=

∑ ∑ (2.12)

where (0)ijη is initial response of V1 neuron (i,j), sij is afferent activation of V1

neuron (i,j), abξ is the activation of LGN ON/OFF neuron (a,b) in the receptive field

of V1 neuron (i,j), Aab,ij is the afferent weight, and Aγ is a constant scaling factor.

After receiving afferent input, V1 neurons propagate the activations through the lateral excitatory and inhibitory connections for several time steps until the activity settles down. At each time step, the neuron combines the afferent activation with lateral excitation and inhibition:

22

, ,( ) ( 1) ( 1)ij ij E kl kl ij I kl kl ijkl kl

t s t E t Iη σ γ η γ η⎛ ⎞

= + − − −⎜ ⎟⎝ ⎠

∑ ∑ (2.13)

where ( 1)kl tη − is the activity of another V1 neuron (k,l) in previous time step, Ekl,ij is

the excitatory lateral connection weight connecting neuron (i,j) and (k,l), Ikl,ij is the inhibitory connection weight, Eγ and Iγ are scaling factor that determine the strength

of excitatory and inhibitory lateral interactions. The initial activity of V1 starts out diffuse but slowly converges into a small number of stable focused patches of activity, or activity bubbles. After the activity settles, the afferent and lateral connection weights of V1 neurons are modified according to Hebbian rule with divisive postsynaptic normalization:

( ),'

,,

pq ij pq ijpq ij

uv ij uv ijuv

w Xw

w Xα η

α η

+=

+∑ (2.14)

where wpq,ij is the current connection weight from (p,q) to (i,j), ',pw ijw is the new

connection weight, α is the learning rate for each type of connection, Xpq is the presynaptic activity after settling, and ijη is the activity of neuron (i,j) after settling.

After the map begins to organize, some lateral inhibitory connection weights become very weak and they can be periodically removed to model connection death in biology, resulting in patchy lateral connectivity similar to that observed in animal visual cortex. Hebbian learning rule facilitates the cortical neurons to learn correlations in the input patterns and develop representation of the correlation structures in the afferent and lateral weights value. Divisive normalization is necessary for Hebbian rule to prevent the weights value from increasing out of bound. One aims of this project is to replace this normalization with more biologically plausible regulation based on homeostatic synaptic scaling. Several parameters like lateral excitation, sigmoid curve, and the learning parameters are gradually adjusted during self-organization to enhance the resulting self-organized map. This adjustment is currently done manually and it is hoped that some of this manual adjustment could be replaced with homeostatic mechanism. For the purpose of this project, the network will be trained with oriented Gaussian pattern with random position and random orientation, using the following equation:

23

2 2

, , , ,2 2

[( )cos( ) ( )sin( )] [( )sin( ) ( )cos( )]maxexp c k c k c k c k

xy k a b

x x y y x x y yφ φ φ φχ

σ σ

⎛ ⎞− − − − + −= − −⎜ ⎟⎜ ⎟

⎝ ⎠ (2.15)

Where xyχ is the activity of retinal receptor (x,y), aσ and bσ determine the width

along the major and minor axes of the Gaussian, and φ is the orientation.

After being trained for several thousands iterations (usually between 10000 to 20000 iterations), the network develop an orientation map which is similar to the maps observed in animal visual cortex (figure 2.8). This orientation map was obtained by presenting the sine grating with various orientations and frequencies and then measuring the activity of the neuron that most strongly respond to particular orientation. If we look at the self-organized afferent weights, some neurons develop Gabor-shaped RFs that are strongly selective to particular orientation and the some others develop spherical RFs that are not orientation selective (figure 2.9.a). Lateral inhibitory connections are patchy and connecting neurons with similar orientation preference, consistent with neurobiological experimental result (figure 2.9.b).

Figure 2.8. Orientation preference map and orientation selectivity of LISSOM network trained with oriented Gaussian pattern. Every neuron in the map is colored according to the orientation that it prefers. Nearby neurons generally prefers similar orientations, forming groups called iso-orientation patches. The map contains all the features found in real animal maps, such as pinwheel (marked by white circle): points around which orientation changes continuously, linear zones (long rectangle): straight lines along which orientations changes continuously, fractures (small square): sharp transition between two very different orientation, and saddle point (marked with bowtie): a long patch where one orientation is nearly bisected by another. Orientation selectivity measures how closely the input must match the neuron’s preferred orientation for it to respond (coded by grayscale, with white means highly selective, and black means low selecvitiy). (adapted from [19])

24

Figure 2.9. Self-organized afferent weights and lateral inhibitory weights of LISSOM network trained with oriented Gaussian pattern (adapted from [19]).

2.3.2 Threshold Adaptation in LISSOM

In LISSOM model, the global activation threshold should be adapted gradually to enable the development of smooth orientation map and increasing selectivity of the neurons to particular orientation. In the beginning, the threshold is set within a certain range so that all the neurons can easily response to the given input pattern and Hebbian learning can associate the activity of input and output to change the synaptic weights accordingly. During the course of self-organization, the activation threshold should be gradually increased so that the neurons become harder to fire and retain its selectivity to certain orientation that was previously learned. If the threshold is kept unmoving, later presentations of input pattern will disturb the learned weights value so the neurons cannot learn the association with specific orientation and form good receptive field. With the intention to observe the effect of self-organization without threshold adaptation, the simplified LISSOM model (with one retinal sheet and one V1 layer) was trained with fixed threshold setting. After 20000 iterations, we get the following orientation map and afferent weights.

25

Figure 2.10. orientation map and afferent weights of LISSOM without threshold adaptation.

Figure 2.11. Average firing rates of several neurons of LISSOM without threshold adaptation.

It can be seen clearly that the network failed to organize properly. The orientation map looks totally unordered and the receptive fields of most neurons are not selective to particular orientation. If we plot the average firing rate of several neurons, we also notice that the neurons tend to increase the activity level in the beginning and leveled up after about 5000 iterations. This happened because with fix threshold setting the neuron could not limit the increase in excitability due to Hebbian learning, so instead of becoming more selective to particular orientation, it will end up preferring several orientations at once. The preceding example illustrates the importance of adjusting the neuron’s excitability in the development of orientation map. In typical LISSOM models, the threshold is changed several times by manually setting upper and lower bound values at certain predefined iterations. This process is usually done by trial and error, based on previous experience of the modeler. Figure 2.12 shows the plot of threshold settings that are usually used in typical LISSOM models.

26

Figure 2.12. The piecewise-linear activation function during several iterations of learning.

2.3.3 LISSOM without Shrinking Lateral Excitatory Radius

LISSOM models have been very successful in simulating the self-organization of orientation map in primary visual cortex. However, there are still some discrepancies when we compare the self-organization process in LISSOM and the development of orientation map in real animal visual cortex. Measurements in ferret visual cortex during development, reveals that the structure of the orientation map is already established in the early stage of development and remains stable throughout self-organization [9]. On the other hand, in LISSOM model, the structure of the orientation map is repeatedly changing during the first several thousands iterations, and the orientation column spacing also shrinks dramatically as the map develops (figure 2.13). Reduction of orientation column spacing is not observed in the development of animal orientation map.

100 iterations 200 iterations 500 iterations 1000 iterations

2000 iterations 4000 iterations 6000 iterations 20000 iterations

Figure 2.13 Self-organization of orientation map in LISSOM model. It can be observed that the map changes its structure several times during the first 4000 iterations and column spacing shrinks

considerably in the first 1000 iterations.

27

The main cause of this problem was speculated to be the adjustment of lateral excitatory radius and reduction of learning rate in the beginning of self-organization. Initial attempts have been made to solve this problem, using LISSOM model with fixed lateral excitatory radius and faster learning rate reduction. The orientation map can still develop using this method and the map formation is smoother, more stable, and closer to biology (figure 2.14). However, new problem arise with this approach. Some neurons fail to organize properly and their afferent weights still look random (figure 2.15). This might happen because the learning rate was reduced very fast in the beginning and these neurons did not receive enough activation while the learning rate was still high. They also miss the opportunity of learning during later iterations because the activation threshold was already too high, reducing their excitability. It is hoped that homeostatic plasticity mechanism might helps in this case by regulating individual neurons excitability depending on the activations they receive. So the neurons that fail to learn in the beginning can still have an opportunity to learn later because they still retain their high excitability.

Figure 2.14. Orientation map obtained from LISSOM model without shrinking lateral excitatory radius and faster learning rate reduction. White circles indicate areas where groups of neurons fail to organize properly.

28

Figure 2.15. Afferent RFs and retinotopic mapping of LISSOM model without shrinking lateral excitatory radius. It can be noticed that RFs of some neurons are still random looking and the retinotopic mapping clearly shows several areas with discontinuities.

29

Chapter 3 Method & Parameters

This chapter describes the method and parameters of various experiments that are conducted to investigate the behavior of LISSOM models with homeostatic plasticity mechanism. It is hoped that with inclusion of homeostatic adaptation several aspects of LISSOM based models can be improved and makes it more biologically plausible. Two models of homeostatic plasticity, intrinsic plasticity model of Triesch [27, 28] and homeostatic synaptic scaling model of Sullivan & de Sa will be implemented and applied to various LISSOM models of visual cortex self-organization. The result of these experiments will be presented in the next chapter.

3.1 Choice of the Models In current computational neuroscience literature, there are only few proposed models of homeostatic plasticity mechanism that are appropriate for rate-based neural models. Most computational models of homeostatic plasticity explain the phenomena in higher level of details involving Hodgkin-Huxley models or spiking neuron models. Although there is a variant of LISSOM model that is based on spiking neurons (SLISSOM [10]) which was primarily developed to investigate segmentation and binding, most LISSOM based models are still using rate-based neurons, mainly for speed and simplicity. Several models of homeostatic plasticity mechanism might be applicable to rate-based LISSOM models. These models are Triesch intrinsic plasticity [27, 28], Sullivan and de Sa homeostatic synaptic scaling [25, 26], variants of BCM rules, and threshold scaling using plastic facilitation function [15, 16, 33]. It is very tempting to try implementing and applying all these different models into LISSOM and analyzes its effect. However, given limited time, I only choose the first two models to be implemented and analyzed in details. The main reason behind this choice is because these two models have been proven to be working reasonably well in SOM based models (which are precursor of LISSOM) [6, 7, 25], so there is higher possibility that these models will also work well with LISSOM. Moreover, these two models represent two primary aspects of homeostatic plasticity that are currently hypothesized, which are intrinsic plasticity [36] and synaptic scaling [29, 30]. Intrinsic plasticity mechanism might be a promising alternative to replace manual

30

threshold adaptation in LISSOM models and homeostatic synaptic scaling might be an alternative to replace divisive normalization in Hebbian learning.

3.2 Experiment with Intrinsic Plasticity Applied to LISSOM Models This experiment is conducted to study and analyze the effect of applying intrinsic plasticity mechanism proposed by Triesch [27] (described in section 2.2.2) to the LISSOM models. It is hoped that implementation of intrinsic plasticity can eliminate the need of manual threshold adjustment and improve several aspect of LISSOM models. Four different experiments are conducted to test this homeostatic mechanism. The first experiment is carried out to study the basic behavior of simplified LISSOM model with intrinsic plasticity in quite details. Several features of self-organized network and changes during self-organization will be carefully examined. This first experiment is also intended to find out whether LISSOM with intrinsic plasticity can perform as well as (or better than) the one with manual threshold adaptation. In the second experiment, sensitivity of intrinsic plasticity parameters will be examined by running several simulations with different parameter settings and analyze the result of self-organized network. The third experiment is devised to test intrinsic plasticity with more complete LISSOM network with LGN ON/OFF channels. In the forth experiment, intrinsic plasticity will be applied to LISSOM network without shrinking lateral excitatory boundary, which are thought to be more realistic in simulating self-organizing process during the development of visual cortex. The following subsections explain the methods and parameter settings for all these experiments.

3.2.1 Intrinsic Plasticity in Simplified LISSOM Network

A simplified LISSOM model with one retinal sheet and one V1 layer will be used in this experiment (figure 3.1). The retina density is 24x24 and the V1 density is 84x84 neurons. Afferent connections from retina to V1 and lateral excitatory/inhibitory connections of V1 neurons are all adapted through Hebbian learning. Divisive normalization is used to constraint the growth of the weights value. In later experiment, homeostatic weight normalization will be tried together with intrinsic plasticity to make the model closer to biology. The network is trained using elongated Gaussian spot (figure 3.2) with random position and random orientation, presented to the retina one pattern at each time step.

31

Figure 3.1. Simplified LISSOM model with one retina sheet and V1 layer.

Figure 3.2. Example of elongated Gaussian spot presented to the retina as training pattern.

For analysis and comparison, two slightly different LISSOM networks are implemented in Topographica. The first network is LISSOM with manual threshold adaptation and the second one is LISSOM with automatic threshold adaptation based on intrinsic plasticity model of Triesch [27, 28]. Parameters setting (such as learning rate, input patterns, and adaptation of lateral connections) for both models are identical except their activation functions. The first network is the typical LISSOM implementation that uses piecewise linear approximation of sigmoid activation function to control the gain and implement saturation behavior of V1 neurons (figure 2.7; equation 2.11). There is only one globally shared threshold function for every V1 neuron. The upper and lower bound of this threshold is adjusted manually several times during self-organization to ensure smooth global organization and well-tuned receptive field. Using single activation function for every neuron might not be very realistic biologically but it is practically effective and simpler to implement. The second network is LISSOM with intrinsic plasticity that maintain individual sigmoid activation function (equation 2.5) for each V1 neuron. These sigmoid functions are adapted automatically based on recent activity of the neurons using gradient descent algorithm to maximize entropy / information transfer [27]. Using nonlinear sigmoid functions for individual neurons instead of one global linear activation function might impact the speed of simulation since exponentiation is

32

slower than division. The gradient rule that update the sigmoid threshold also involves quite complex arithmetic operations that might consume additional computation time. However, based on recent evidence of homeostatic regulation in neuron’s intrinsic plasticity, keeping track separate activation function for each neuron is more biologically realistic and might also give additional benefits to the whole process of self-organization. Extra parameters are also needed for this gradient rule (equation 2.8). Separate experiments have been carried out (by trying different parameter settings systematically) to find out approximately the best value of these parameters for this particular model. The parameter settings for this experiment are: initial sigmoid threshold (a=13, b=-4), homeostatic learning rate (η =0.0002), target average firing

rate of individual neuron (µ =0.01). The initial sigmoid threshold is chosen such that

initially V1 neurons will respond quite strongly to the input pattern and enable rapid learning in the beginning. The position and slope of initial sigmoid threshold is similar with initial threshold setting of the piecewise linear in the first model. The homeostatic learning rate is set to be much slower than Hebbian learning, consistent with biological observation. Although the exact speed of homeostatic adaptation is still unknown, this value works quite well. Target average firing rate is set based on measurement of the average firing rate of the first model after the network stabilized. Sensitivity of the model to these homeostatic parameters is explored in the next experiment. After running the simulation for both networks until reaching convergence (about 20000 iterations), several aspects of them can be analyzed by observing some properties that emerge after self-organization such as the orientation maps, shape of receptive fields, retinotopic mapping, and lateral inhibitory connections. The similarities and differences of these properties between the two models will be discussed and compared. Another aspects of the models which are more numerical such as average firing rate of individual neuron, distribution of firing rate, activity pattern of individual neuron, and the speed of threshold adaptation will be studied by plotting time-series graphs and analyzing its pattern. The average firing rate will not be computed using all previous activities of the neuron; instead only recent activities of the neurons are used to calculate the running average, using the following equation:

1(1 )t t ti i iY y Yα α −= + −

33

t

iY is the running average of neuron i at time t; tiy is the firing rate of neuron i at time t,

and α determines the speed over which previous activity of the neuron is averaged. For all experiments in this report, 0.0003α = is used. This equation is used because homeostatic adaptation is hypothesized to be related to recent past activities of the neuron. So instead of averaging all activities from the beginning to the end, the average is computed by taking into account only recent activities in specific time window.

3.2.2 Sensitivity of Intrinsic Plasticity Parameter Settings

The second experiment is performed to test the effect of choosing different parameter values for intrinsic plasticity model to the learned self-organized LISSOM network. The simplified LISSOM model with intrinsic plasticity threshold adaptation that is described in the first experiment will be used again in this experiment. There are three different parameters that control the behavior of intrinsic plasticity mechanism (equation 2.8) : the initial position and slope of sigmoid function (a and b), the homeostatic adaptation rate (η ), and the desired average firing rate ( µ ). As a

baseline for comparison, the values in the previous experiment will be used (a=13, b=-4, η =0.0002, µ =0.01). Separate experiment has demonstrated that using this

parameter setting for intrinsic plasticity can produce good self-organized LISSOM map. At first, using fixed homeostatic adaptation rate (η =0.0002) and fixed target average

firing rates (µ =0.01), several simulations will be carried out using different initial

position and slope of sigmoid activation function, and then the result of self-organized map and all its properties will be observed. Secondly, the initial sigmoid curve and target average firing rates are held fixed (a=13, b=-4, µ =0.01), but homeostatic

adaptation rate will be varied. Finally, using fixed initial sigmoid curve and homeostatic adaptation rate, several values of target average firing rates will be tried. Table 3.1 lists all parameter values that will be tried in the simulations.

simulation # a b η µ

1 15 -3 0.0002 0.01 2 18 -3 0.0002 0.01 3 10 -4 0.0002 0.01 4 13 -6 0.0002 0.01

34

5 13 -4 0.0001 0.01 6 13 -4 0.0004 0.01 7 13 -4 0.001 0.01 8 13 -4 0.0002 0.005 9 13 -4 0.0002 0.02

Table 3.1. Different intrinsic plasticity parameter settings what will be used in simulation.

3.2.3 Intrinsic Plasticity Applied to LISSOM with LGN ON/OFF Channel

In the third experiment, intrinsic plasticity will be applied to more complete LISSOM model with LGN ON/OFF layers in-between retina and V1, as shown in figure 3.3. Intrinsic plasticity is applied only to V1 neurons, not to LGN neurons. In reality, perhaps all neurons, including retina and LGN neurons are regulated by some kind of homeostatic mechanism, but this fact is ignored in this experiment for simplicity. After being trained for about 20000 iterations using oriented Gaussian patterns, the result of the network with homeostatic mechanism will be analyzed and compared with result of the network with manual threshold adaptation.

Figure 3.3. LISSOM model with LGN ON and LGN OFF sheets

Parameter setting for this experiment is similar with the first experiment. The size of retina sheet and LGN On/Off is 24x24, and the size of V1 is 84x84 neurons. Using higher density for V1 will allow better visualization of the orientation map. Values for intrinsic plasticity parameters are exactly the same with the first experiment (a=13, b=-4, η =0.0002, µ =0.01). It is expected that using the same parameter values the

network can still organize properly.

35

3.2.4 Intrinsic Plasticity Applied to LISSOM without Shrinking Lateral Excitatory Radius

LISSOM models have been quite successful in simulating the self-organization of visual cortex map. However, as described in section 2.3.4, the detailed development of orientation map in this model is rather different from development of orientation map in animal. The difference was thought to be caused by shrinking lateral excitatory radius in the beginning of self-organization. Initial experiments have been conducted to simulate LISSOM with fixed lateral excitatory radius. Although the orientation map can still form, there are some neurons that fail to learn because they didn’t get enough activation in the beginning and the global activation threshold become too high afterwards. In this experiment, intrinsic plasticity will be applied to simplified LISSOM model without shrinking lateral excitatory radius. It is hoped that with intrinsic plasticity the process of orientation map formation will be more similar to animal map development. The parameters for this simulation (in Topographica) are given in table below.

Parameter Value Retina density 24 x 24 V1 density 84 x 84 Afferent learning rate Initial value: 0.9590; after 100 iterations: 0.548;

after 1000 iterations: 0.274; after 3000 iterations: 0.137 Lateral excitatory radius 0.03 Lateral excitatory learning rate 0 Lateral inhibitory radius 0.244 Lateral inhibitory learning rate 1.7871 Input pattern Oriented gaussian: scale=1.0, size=0.0936, aspect_ratio=4.0 Intrinsic plasticity learning rate 0.002 Initial sigmoid position a = 13, b = -4 Intrinsic plasticity target average firing rate

0.01

Table 3.2. Parameter for simulation of LISSOM with intrinsic plasticity and no shrinking excitatory radius

3.3 Experiment with Homeostatic Synaptic Scaling Applied to LISSOM Models In this experiment, homeostatic synaptic scaling model of Sullivan and de Sa [25, 26] will be implemented and applied to LISSOM model. This synaptic scaling mechanism will replace the divisive normalization that is used in Hebbian learning to constraint

36

the growth of synaptic weights. It is hoped that with homeostatic synaptic scaling the network can still organize properly and good orientation map can be developed. The simplified LISSOM network that was described in section 3.2.1 will be used again in this experiment. For initial experiment, the simulation will not use any threshold function. Only simple bounding function to constraint the firing rate within 0.0 (minimum value) to 1.0 (maximum value) will be applied, similar to what has been done in [26]. If this approach cannot work properly, manually adjusted piecewise-linear threshold will be tried to regulate the neurons activation. In the last experiment, homeostatic synaptic scaling will be applied together with intrinsic plasticity mechanism to the LISSOM network. It will be observed whether the network can still organize well in this case. The network will be trained for several thousands iterations, and then several properties like orientation map and receptive fields will be observed and compared with the standard LISSOM network. For observing the stability of this algorithm, the average firing rate and the sum of weights value of one projection will be plotted in time-series graph. The parameters for LISSOM is exactly the same with the first experiment. According to equation 2.3 and 2.4, the synaptic scaling itself needs three additional parameters, which are Nβ (homeostatic learning rate), cβ (control the speed of averaging the

neurons firing rate), and Atarget (desired target average firing rate). Several different values will be tried for these parameters and the resulting self-organized network will be evaluated. Further analysis such as sensitivity of these parameters will be carried out if the basic result of this algorithm on LISSOM network is demonstrated to be good enough.

37

Chapter 4 Results

4.1 Intrinsic Plasticity Applied to Simplified LISSOM Network This section presents the result of experiment that was described in section 3.2.1. The experiment was conducted to study the effect of applying intrinsic plasticity mechanism to the simplified LISSOM model. Two networks are compared in this experiment, one is LISSOM with manual activation threshold adjustment and the other is LISSOM with intrinsic plasticity.

4.1.1 Receptive Fields and Orientation Maps

In this section we will compare the orientation map and receptive fields for LISSOM model with manual threshold adjustment and model with intrinsic plasticity. After being trained for 20000 iterations, several properties of the network can be visualized graphically in Topographica. One of this property is orientation preference map. This map is computed by presenting the sine grating with different orientations and phases to the network and measuring the activity of V1 neurons to this input. Each neuron will be assigned particular color that represent particular orientation that activate it most strongly. It can be observed from figure 4.1 that the network with intrinsic plasticity can develop properly organized orientation map with visual appearance and features (such as pinwheel, linear zones, saddle point, and fractures) similar to the original model using manual threshold adaptation. The map with intrinsic plasticity also has larger column spacing and looks more similar to real orientation map found in animal visual cortex [5]. Orientation selectivity also improves considerably with intrinsic plasticity. Most of the neurons are highly selective to specific orientation with some exception on the neurons in the upper-right and bottom-right corners.

38

(a) Orientation map of LISSOM with manual threshold adjustment

(b) Orientation map of LISSOM with intrinsic plasticity

Figure 4.1. This figure shows the orientation map and selectivity of LISSOM with (a) manual threshold adaptation and (b) with intrinsic plasticity, after 20000 training iterations. Orientation preference map show the preference of individual neuron to particular orientation. Each orientation is represented by different color coding. Orientation selectivity map shows the degree of selectivity of specific neuron to particular orientation. The grayscale from black to white represent increasing selectivity (i.e. black: non selective, white: highly selective). Orientation preference & Selectivity is combination of orientation preference map and orientation selectivity. Certain features that exist in typical orientation map also appear in map with intrinsic plasticity, such as linear zones (long rectangles), fractures (square), pinwheel (circle), and saddle point (marked with bowtie).

With manual threshold adaptation, almost every neuron in the corners fails to organize properly therefore we saw random noise on the orientation preference map in this area (most noticeable on top-right and bottom-left corner of figure 4.1.a). This happens because these corner neurons receive less activations in the beginning, so while they are still learning an association with some oriented pattern in later iterations, the activation threshold might be already too high so they cannot learn much and ends up to be selective to some random orientation.

39

If we observe both orientation maps closely, it is apparent that there are some discontinuities around the borderline. This happens because the oriented Gaussians spot that are used as input pattern did not cover the neurons around the borders as much as neurons in the middle area. Therefore the neurons around the border receive less training input than other neurons. This unpleasant artifact can be easily removed by extending the coverage area of the oriented Gaussian input (i.e. by increasing the upper and lower bound of random number generator for x & y position of Gaussian input pattern from +/- 0.5 to +/- 0.65 in Topographica). However, in this experiment, this fact might also help to discern the benefit of using intrinsic plasticity. In the orientation and selectivity map of the model using manual threshold adaptation, we can observe some neurons around the border (especially neurons in the corners) that are completely unselective and did not organize properly, whereas in the model using intrinsic plasticity, these neurons are still selective even though they receive very few inputs. Using homeostatic mechanism, when the neurons receive less input, their intrinsic excitability will be lowered to achieve homeostatic average firing rate so it can still operate effectively. This is beneficial because there will be less neurons that are useless and die out due to receiving less input during the development. In the next section this observation will be verified more closely by plotting the graph of average firing rate and threshold value of these neurons.

(a) RFs and lateral inhibitory weights of LISSOM with manual threshold adjustment

40

(b) RFs and lateral inhibitory weights of LISSOM with intrinsic plasticity

Figure 4.2. This figure shows the afferent RFs (left) and lateral inhibitory connections (right) of LISSOM with manual threshold adaptation and with intrinsic plasticity, after 20000 training iterations. The afferent RFs show the pattern of synaptic weights that connect V1 neurons to retina neurons. Each little square represent synaptic weights of one V1 neuron. The weight value is coded by grayscale from black (zero value) to white (high value). The lateral inhibitory plots show the pattern of lateral inhibitory connections between V1 neurons. Each small square represent lateral connection weights of one V1 neuron to its neighbor’s neurons (represented as single pixel in that square). The colors represent orientation preference of the neurons that are laterally connected to the current neuron. The intensity/brightness of the color represents the strength of connection (higher intensity means higher strength).

The above figure compares the afferent weights of V1 neurons in the network with manual threshold adaptation and network with intrinsic plasticity. Qualitatively, the afferent receptive fields of both networks do not differ much. The final afferent RFs of the network with intrinsic plasticity are Gabor-shaped and strongly selective to particular orientation. Lateral connections of both networks are patchy and tend to follow the RF shape, connecting neurons with similar orientation preference, similar to result found in biology [22]. One obvious difference is that neurons with intrinsic plasticity have slightly broader receptive field size and less unselective RFs. This might happen because with this model of intrinsic plasticity, the neurons try to achieve a desired average firing rate and exponential firing rate distribution at the same time. In order to achieve exponential firing rate, the frequency of weak firing should be more often than strong firing. With broader receptive field each neuron has more chance to be weakly activated by patterns that are not aligned to its receptive field orientation.

41

(a)

(b)

Figure 4.3. This figure shows the retinotopic organization of the orientation map for LISSOM with (a) manual threshold adaptation and (b) with intrinsic plasticity, after 20000 training iterations. This map is computed by taking the center of gravity of the afferent weights of each V1 neurons and connecting it with the center of gravity of neighboring neurons.

The above figure shows the retinotopic organization of the orientation map which is computed by taking the center of gravity of the afferent weights of the neuron and connecting it to the center of gravity neighboring neurons with lines. There are some distortions in this mapping because orientation map represent both position and orientation smoothly across the same surface. The retinotopic map of the network with intrinsic plasticity (figure 4.3.b) covers the whole input space uniformly which is good indication that the network has learned to represent all positions in the input space properly. Moreover, there are fewer distortions in this map with intrinsic plasticity which indicates that the network has managed to learn more accurate mapping than the network without intrinsic plasticity. The figure below shows the detailed self-organization process of LISSOM with intrinsic plasticity and LISSOM without intrinsic plasticity. It is apparent that with intrinsic plasticity the development of the map is more stable. The pattern of the orientation map did not change much during self-organization.

42

Iterations LISSOM With Intrinsic Plasticity LISSOM Without Intrinsic Plasticity

100

200

300

500

1000

43

2000

4000

6000

10000

20000

Figure 4.4. Comparison of orientation map self-organization process between LISSOM with intrinsic plasticity and LISSOM with manual threshold adaptation. It is apparent that with intrinsic plasticity the development of the map is more stable and the structure of orientation map did not change much during self-organization.

44

4.1.2 Average Firing Rate, Firing Rate Distribution, and Speed of Threshold Adaptation

In order to gain better insight into the mechanism of intrinsic plasticity, the behavior of individual neurons is analyzed by plotting the graph of the firing rates, average firing rates, distribution of firing rates, and the speed homeostatic threshold adaptation. Figure 4.5 show the average firing rates of several neurons on different position of V1 sheet during self-organization from 0 to 20000 iterations.

(a) average firing rates of several neurons in LISSOM with manual threshold adaptation.

(b) average firing rates of several neurons in LISSOM with intrinsic plasticity.

Figure 4.5 Average firing rates of neurons in V1 (Blue: neuron in the top-left corner position, Red: neuron in the center of V1, Green & Cyan: neurons halfway between the corner and the center)

The average firing rates curves of the neurons in both networks follow similar trends. Initially, the average firing rates is quite high (ranging from 0.04 to 0.10) because the activation threshold is low in the beginning to enable the neurons to learn as much as possible from the given inputs. As the network self-organize and learns better associations with input patterns, slowly the activation threshold is increased so the neurons becomes more selective to specific orientation and activated less often than

45

before. So gradually, the average firing rates also decrease until reaching stable value around 0.01. This plot also shows that intrinsic plasticity can perform as good as manual threshold adjustment in slowly regulating the average firing rate of the neurons to reach desired value. An interesting difference can be observed in the average firing rates curves of the neuron in the top-left corner (in blue color). As described in section 4.1.1, this neuron receives fewer inputs compared to other neurons so it also activated less. In the network with manual threshold adjustment (figure 4.5.a), the average firing rate of this neuron is slowly rising for the first 500 iterations (after it receives some input patterns) but then it decays rapidly and silenced out after 5000 iterations. This happens because the activation threshold is adjusted globally so the activity of all neurons will be decreased over time with the same speed regardless of individual neuron’s activity history. Therefore the neuron that receives less activation in the beginning will be even harder to be activated later and finally it fails to learn meaningful association with the inputs and sometimes even becomes completely silenced. This situation can be improved using intrinsic plasticity for individual neuron (figure 4.5.b, blue curve). The same neuron can now slowly increase its activity (by decreasing its own sigmoid activation threshold) to achieve the target average firing rate. After only about 500 iterations this neuron can reach reasonable average firing rate (around 0.03) so it can learn correlations with input pattern more properly.

(a) Firing activity of single neuron in network without intrinsic plasticity

(b) Firing activity of single neuron in network with intrinsic plasticity

Figure 4.6. Firing activity of single neuron (located in the center of V1) of LISSOM with manual threshold adaptation (a), and LISSOM with intrinsic plasticity (b) during 20000 training iterations.

46

Figure 4.6 shows the plots of firing pattern for single neuron located in the center of V1 for the network with manual threshold adjustment and the network with intrinsic plasticity. It is interesting to find out that although both networks have similar average firing rate curves (figure 4.5), the pattern of firing of individual neuron is comparatively different. Without intrinsic plasticity (figure 4.6.a), the neuron’s firing pattern is dense and most activities are dominated by higher firing rate level (between 0.5 – 1.0). In the beginning of self-organization (the first 5000 iterations), the neuron is activated more often because the activation threshold is lower and learning rate is high. After 5000 iterations, the activity become sparser and the peak activity level is also slightly reduced. Different firing behavior is observed in the network with intrinsic plasticity. The neuron’s activity is already sparse since the beginning, and most activities are dominated by lower firing rate level (below 0.1). The difference can be seen more clearly by plotting the histogram of output of single neuron before and after passing through threshold function (figure 4.7). Before passing through the threshold function, the neuron’s output is already exponentially distributed. This might happen because oriented Gaussian is used as the training pattern. With other training pattern such as natural images this statistics might be different. After passing through piecewise-linear threshold function (figure 4.7.a, right), the distribution of firing rates becomes almost flat with an exception of single peak around 0, which means that every possible value of firing rate occurs equally often. Using intrinsic plasticity, the neuron retains the exponential distribution of firing rates (figure 4.7.b, right), with most activations dominated by small firing rate activity (below 0.05, with an exception of small peak around 1.0). This shows that model of intrinsic plasticity of Triesch [27] can work well on LISSOM model in regulating the neurons activity to achieve sparse and exponentially distributed firing rates.

47

(a) Histogram showing distribution of outputs from single V1 neuron before thresholding (left) and after thresholding (right) for the network with manually adjusted piecewise-linear threshold.

(b) Histogram showing distribution of outputs from single neuron before thresholding (left) and after

thresholding (right) for the network with intrinsic plasticity.

Figure 4.7. The plots compare the histogram of output from single neuron before and after passing through threshold function for the network with manual threshold adjustment (a) and network with intrinsic plasticity (b). The histogram is computed using 100 bins using data collected during 20000 iterations.

Intrinsic plasticity adapts the position and gradient of the sigmoid threshold function (variable a and b in equation 2.5) adaptively depending on recent past activity of the neuron during self-organization process. The speed of threshold adaptation can be observed by plotting the change of variable a and b during the course of learning (figure 4.8). The value of a, which controls the gradient, is gradually decreasing following linear path, which means that the gradient of sigmoid is slowly increasing. The value of b, which controls the position of sigmoid, is decreasing rapidly in the first 500 iterations. This signifies that the curve is translated to the right pretty quickly in the beginning to reduce the activation of the neurons to sparser regime (figure 4.9).

48

Figure 4.8. The change of sigmoid threshold parameter (a and b) during the course of self-organization (from 0 to 20000 iterations). Intrinsic plasticity mechanism adjust this value adaptively based on the activity of the neurons. It can be observed that the value of a is linearly decreasing from 13 to 10 while the value of b is decreasing rapidly in the first 500 iterations and then leveled up at -6.

Figure 4.9. Plot of sigmoid threshold adaptation during 20000 iterations. The curve shifts very rapidly in the first 1000 iterations and then constantly increases the slope afterwards.

4.2. Sensitivity of Intrinsic Plasticity Parameter Settings In order to simulate realistic self-organization process of visual cortex map, several parameters in LISSOM network such as Hebbian learning rate and lateral connection strengths should be tuned properly. With inclusion of intrinsic plasticity mechanism, the number of parameters that should be specified is growing. Now we should determine four additional parameters: the initial sigmoid curve (a, b), the homeostatic learning rate (η ), and the target average firing rate (µ ).

This experiment was performed to observe the sensitivity of intrinsic plasticity parameter settings to the resulting self-organized LISSOM map. Simplified LISSOM network that was used in the last experiment was simulated several times with different values of intrinsic plasticity parameters and after several thousands iterations, the resulting map is observed.

49

4.2.1. Initial Threshold Values

Intrinsic plasticity mechanism adjusts the sigmoid activation function of individual neuron automatically, based on the history of their past activities. The sigmoid activation function (equation 2.5) is determined by two variables a and b. a controls the slope of the curve and b controls its position. Initially, this value should be specified by the modeler. Choosing different initial values for a and b might affect the result of self-organization. Four different combinations of initial a and b values were tested in this simulations (figure 4.10). All simulations use the same homeostatic adaptation rate (η =0.0002) and target average firing rate ( µ =0.01). The initial

threshold value that was used in previous experiment (a=13, b=-4) will be used as a baseline for comparison.

Figure 4.10. The plot shows the initial sigmoid activation curves that are going to be tried in this

experiment. The blue curve is the sigmoid activation function that was used in the first experiment. This function will be used as a reference point for comparison.

No. Initial

Threshold Results and Comments

1 a=15, b=-3 (red curve, figure 4.10)

This initial threshold value is set to be lower than the baseline threshold. Because the threshold is low, initially, V1 neurons will be highly active, but intrinsic plasticity mechanism can still bring down the activation level at proper time, so in the end the network can still develop good orientation map. The figure below is the orientation map after 15000 iterations.

50

2 a=18, b=-3

(green curve, figure 4.10)

The curve is located at the same position with the previous one but has smaller gradient. The network failed to develop good orientation map using this initial threshold because V1 activity is very high in the beginning and homeostatic adaptation cannot bring down the neurons activity to the target level (0.01) on time. After 20000 iterations, the neurons average firing rate is still quite high (0.03). Perhaps better map can be obtained if using faster homeostatic adaptation rate. The figure below shows the map after 20000 iterations.

3 a=10, b=-4

(black curve, figure 4.10)

This threshold is located at the same position with the baseline threshold but has considerably larger gradient. The network totally failed to develop good orientation map using this initial threshold. During the training process, the threshold did not adapt too much because V1 activity has already reach near the target value. The value of a did not change at all, and b was only slightly decreasing from -4 to -5.5 for the first 2000 iterations and then leveled up. After 10000 iterations the following orientation map is obtained.

4 a=13, b=-6

(purple curve, figure 4.10)

This curve has the same gradient with the baseline curve but positioned slightly farther to the right. Using this threshold, initial activity of V1 was very low, but slowly the homeostatic mechanism shifted the threshold to the left and restored V1 activity to higher level so Hebbian learning can operate more effectively. After 20000 iterations, good orientation map was developed.

51

An interesting difference to be noticed is the pattern of average firing rates of V1 neurons during self-organization (the figure below this text). The average firing rates is low in the beginning (around 0.0005 – 0.0008), almost constant for the first 2000 iterations, then increasing quite rapidly for the next 6000 iterations. After 8000 iterations, it begins to decrease slowly and reaching the stable level. We observe that this trend of average firing rates is completely different from the experiment reported in section 4.1. In that experiment, the average firing rates was very high in the beginning and then slowly decreasing until reaching the target value. The neurons are less excitable in the beginning, so intrinsic plasticity mechanism will lower the activation threshold slowly. As learning progressed, the neurons activity is gradually increasing and after 5000 iterations it reached high firing rate level. Because the firing rate is larger than target value, intrinsic plasticity mechanism will bring them down gradually to the desired level of activity by increasing the slope of the sigmoid function.

The figure below shows the pattern of activity of single V1 neuron. Initially the activity is low because the activation threshold is high, but gradually intrinsic plasticity increase the neurons excitability so it can slowly restore its activity.

52

4.2.2. Intrinsic Plasticity Learning Rate

The speed of intrinsic plasticity learning rate might also affect the self-organization process of LISSOM models. In this section, different values of learning rate will be tried using simplified LISSOM network that was used previously. All experiments will use the same initial threshold value (a=13, b=-4) and same target average firing rate (µ =0.01). η =0.0002 will be used as a baseline for comparison. The intrinsic

plasticity learning rate (η ) is held fixed throughout the self-organization.

No. Intrinsic Plasticity

Learning Rate Results and Comments

1 η =0.0001 The network can still develop good map using learning rate which is 2 times slower than the baseline value. The figure below shows the orientation map after 10000 iterations.

Interestingly, although good orientation map can still form using slower learning rate, the step-by-step formation process of the orientation map is rather different than using faster learning rate. With slower learning rate, the orientation map substantially reorganizes itself during early stages of self-organization, just like the self-organization process of the network with manual threshold tuning shown in section 4.1.3. The figures below show several snapshots of the map during the first 4000 iterations:

Iterations Orientation map and orientation selectivity

500

53

1000

2000

4000

2 η =0.00002 Using 10 times slower adaptation rate, orientation map can still form but the quality is not very good. There are lots of fractures and discontinuities. The figure below shows the orientation map after 20000 iterations.

3 η =0.0004 With 2 times faster adaptation rate, the network develop some patterns in

the orientation map but the patterns look a bit strange and unusual. The following figure shows the orientation map after 15000 iterations.

54

4 η =0.001 With 5 times faster learning rate, the network totally failed to develop good orientation map. The orientation selectivity is low and most RFs become spherical-shaped. This happens because when intrinsic plasticity is adapting too fast, the neuron’s excitability will be reduced very rapidly in the beginning of self-organization. Hence, the neurons will have very low activation and as a consequence they will not be able to learn properly through subsequent Hebbian learning. The following figure shows orientation map and afferent RFs after 10000 iterations.

4.2.3. Target Average Firing Rate

The following results shows the orientation map obtained using different value of target average firing rate. All experiments will use the same initial threshold value (a=13, b=-4) and same learning rate (η=0.0002).

No. Target Average

Firing Rate Results and Comments

1 µ =0.005 With lower target average firing rate, the network fails to develop good orientation map and the orientation selectivity also very low.

2 µ =0.02 With twice higher target average firing rate, the network also fails to

develop good orientation map.

55

4.3. LISSOM with LGN ON/OFF and Intrinsic Plasticity Previously, several experiments using simplified LISSOM network have shown that intrinsic plasticity mechanism can be applied successfully to regulate the excitability of V1 neurons during self-organization. In order to test whether this mechanism can still work in more complex network, a LISSOM network with LGN ON/OFF layer will be used. The description of the network is given in section 3.2.3. After being trained for 20000 iterations using oriented Gaussian pattern with random positions and random orientations, the following results are obtained. Figure 4.11 shows the final orientation map and orientation selectivity of the network with manual threshold adaptation and network with intrinsic plasticity. Essentially, both networks developed good orientation maps that are similar to real maps measured in animal visual cortex. They contain structures such as linear zones, pinwheel, fractures, and saddle point. If we observe the detailed structure of the maps more closely, we can find several distinct features between them. The orientation column spacing of the first map is narrower than orientation column spacing of the second map. The transitions between different orientations also appear more smoothly in the second map. Measurements of orientation map in animal visual cortex also produce maps with wide column spacing [5], so in certain sense, the second map is more similar to real animal maps than the first one.

56

(a) Orientation map of the network with manual threshold adaptation

(b) Orientation map of the network with intrinsic plasticity

Figure 4.11. Orientation map and orientation selectivity of V1 neurons in LISSOM with LGN ON/OFF

layer, trained with oriented Gaussian input pattern for 20000 iterations. The upper figure is the result using manually adjusted activation threshold and the lower figure is the result using intrinsic plasticity.

Features like pinwheel, linear zones, fractures, and saddle point can be found on both maps.

Another interesting difference can be observed when we plot the average firing rate of V1 neurons for both networks (figure 4.12). The average firing rate of neurons with manual threshold adjustment rapidly drops after about 4000 iterations and then reached constant level at very low value (around 0.0001 – 0.0002) in 15000 iterations. This might be caused by inappropriate threshold setting or the speed of threshold adjustment was too fast. Although the network can still produce good orientation map, the excitability of V1 neurons become very low, so when an input pattern is presented to the trained network, the response of V1 neurons will be very weak (figure 4.13.a). This problem does not happen when intrinsic plasticity is applied. The neurons can still maintain stable homeostatic firing rate level and V1 response to the input pattern is strong (figure 4.13.b). This might be very beneficial, especially when V1 is connected to the network in higher layer such as V2 and V4, because the signal that

57

propagates from V1 will be strong enough to activate neurons in higher layers and the information will not be lost in the midst of transmission.

(a) Average firing rate of neurons in V1 in LISSOM with manual threshold adaptation.

(b) Average firing rate of neurons in V1 in LISSOM with intrinsic plasticity.

Figure 4.12. This plot shows the average firing rate of several V1 neurons in LISSOM with LGN ON/OFF for period of 20000 iterations.

(a) network with manual threshold adaptation

(b) network with intrinsic plasticity

Figure 4.13. Activity of retina, LGN Off, LGN On, and V1 when presented with oriented line pattern. Both networks were previously trained for 20000 iterations with oriented Gaussian inputs. V1 activity of the first network (a) is substantially weaker than V1 activity of the network with intrinsic plasticity

(b).

58

4.4 Intrinsic Plasticity Applied to LISSOM with no Shrinking Lateral Excitatory Radius Intrinsic plasticity was applied to LISSOM model with no shrinking lateral excitatory radius (described in section 3.2.4). For this simulation, the intrinsic plasticity learning rate was set 10 times faster (η =0.002) than the learning rate used in previous

experiments because the Hebbian learning rate was reduced almost 10 times faster in the beginning of self-organization. This indicates that the speed of intrinsic plasticity adaptation is directly related to Hebbian learning rate adaptation. It is apparent that after applying intrinsic plasticity, better orientation map and selectivity is obtained and there is no more neuron that fails to learn (figure 4.14 & 4.15). The orientation map development is relatively stable (figure 4.16). In the first 1000 iterations, pattern of oriented patches emerges smoothly without any noticeable jumps and the pattern does not change dramatically afterward. The patches that are red/orange in the beginning are still mostly red/orange in the end. Some patches that have green/blue color often change to other colors later. We can also observe that the orientation column spacing shrinks a little bit from 2000 to 5000 iterations but not significantly.

Figure 4.14. Final orientation map and orientation selectivity obtained from simulation of LISSOM network with intrinsic plasticity and no shrinking excitatory radius (after 25000 iterations)

59

Figure 4.15. Afferent receptive fields and retinotopic map obtained from simulation of LISSOM network with intrinsic plasticity and no shrinking excitatory radius (after 25000 iterations). It can be observed that all neurons could develop ring-shaped or spherical-shaped RFs. Smooth retinotopic map also indicate that the network has properly organized.

Iterations Orientation Map

50

150

300

60

500

1000

2000

5000

10000

15000

61

20000

25000

Figure 4.16. Snapshots of orientation map and orientation selectivity taken at specific iterations during

self-organization of LISSOM network with intrinsic plasticity and no shrinking lateral excitatory radius.

4.5. Homeostatic Synaptic Scaling This section presents the result of experiment that was described in section 3.3. In these experiments, homeostatic synaptic scaling model proposed by Sullivan and de Sa [25, 26] was implemented to constraint the growth of synaptic weights that undergo changes through Hebbian learning in LISSOM network. The experiments were divided intro three parts. In the first part, synaptic scaling was applied to LISSOM network with fixed activation threshold, in the second part, it was applied to network manual threshold adjustment, and in the last part it was tried in combination with intrinsic plasticity. With the aim to test whether synaptic scaling can works with fixed activation threshold setting, several simulations were carried out using simplified LISSOM network that uses fixed linear threshold (with lower bound=0.0 and upper bound=1.0). Different combinations of homeostatic learning rate and target average firing rate were tried, but all of them did not give satisfying result. Figure 4.17 shows an example of orientation map and afferent weights pattern obtained from one of the simulation. We can observe that there are still some regular structures in the orientation map but the shape of the afferent RFs is completely disorganized. With the speculation that perhaps homeostatic learning rate and target average firing rate should be reduced gradually throughout the self-organization process (to match the speed of Hebbian learning that is also reduced gradually), several additional

62

experiments were performed, but there is not much improvement either. This result indicates that it is difficult to apply synaptic scaling into LISSOM if we use fixed activation function.

Figure 4.17. Orientation map and afferent RFs of simplified LISSOM network with homeostatic synaptic scaling. The network did not use any threshold function to regulate the neuron’s output. The neuron’s activation function is replaced by simple bounding function to restrict the firing rate within

range 0.0 – 1.0. (synaptic scaling parameters: Nβ =0.1, Cβ =0.003, target average firing rate=0.05)

In the second trial, more simulations were performed to test if the result can be improved by manually adapting the activation threshold throughout self-organization. Several different combinations of homeostatic learning rate ( Nβ ) and target average

firing rate (Atarget) were tried but the network always fail to develop good orientation map. Figure 4.18 shows an example of the result obtained. The orientation map is totally disordered and most of the RFs are spherical and unselective to any orientation. If we observe the change of average firing rate during self-organization (figure 4.5), we notice that it is not constant but slowly decreasing over time. With this insight, several simulations were performed with gradually decreasing target average firing rate (Atarget). The result using this approach is slightly better than the result of previous experiment (figure 4.19). It can be observed that some structures begin to emerge in the orientation map and several elongated shapes appeared in the afferent RFs. Inspired by this slightly better result, another set of experiments were carried out using slowly decreasing homeostatic learning rate ( Nβ ) together with decreasing

target average firing rate. It is reasonable to decrease homeostatic learning rate as self-organization progresses in order to match Hebbian learning rate which is also gradually decreased. Several different scheduled adjustments of homeostatic learning rate were tried without success. However, after numerous trials and errors (by trying different learning rate adjustment that minimizes the oscillations of the weights and average firing rates) better orientation map could be obtained (figure 4.20). The structure of the map looks better now, however there are still lots of fractures and

63

discontinuities in some areas. The orientation selectivity also increased considerably, and most neurons have elongated-shaped RFs. Although slightly better orientation map could be obtained using this approach, it is very difficult to manually tune the homeostatic learning rate in this way. The stability of afferent weights value was also observed by plotting the absolute sum of afferent weights value of single V1 neuron (figure 4.21). Initially, the weights value oscillates quite strongly, alternating within range 0.5 to 2.0. Gradually, the oscillations become weaker but the frequency did not change much. After 20000 iterations, the frequency of oscillation begins to drop because at this time the homeostatic learning rate was reduced significantly.

Figure 4.18. Orientation map and afferent RFs of simplified LISSOM network with manual threshold adjustment and homeostatic synaptic scaling after 13000 iterations. (using constant synaptic scaling

parameters: Nβ =0.01, Cβ =0.005, target average firing rate=0.03)

Figure 4.19. Orientation map and afferent RFs of simplified LISSOM network with manual threshold adjustment and homeostatic synaptic scaling after 20000 iterations. (synaptic scaling parameters:

Nβ =0.01, Cβ =0.01, Atarget=0.05 for the first 1000 iterations, after 1000 iterations Atarget=0.03, after

2500 iterations Atarget=0.01)

64

Figure 4.20. Orientation map and afferent RFs of simplified LISSOM network with manual threshold adjustment and homeostatic synaptic scaling after 30000 iterations. (using gradually adjusted synaptic

scaling parameters: Cβ =0.0085; Nβ =0.0085 for the first 500 iterations, after 500

iterations Nβ =0.0075, after 2000 iterations Nβ =0.0065, after 4000 iterations Nβ =0.005, after 20000

iterations Nβ =0.003; Atarget=0.05 for the first 1000 iterations, after 1000 iterations Atarget=0.025, after

2000 iterations Atarget=0.02, after 4000 iterations Atarget=0.015, after 8000 iterations Atarget=0.01)

Figure 4.21. This plot shows the absolute sum of afferent weights value of one neuron (in the center of V1) during 30000 iterations. The plot was obtained from the same simulation that produced the result shown in figure 4.20.

It is interesting to find out whether LISSOM can still develop good orientation map if homeostatic synaptic scaling is applied together with Triesch’s intrinsic plasticity mechanism. After trying several different parameter settings, finally pretty good looking orientation map could be obtained (figure 4.22). However, this map still looks a bit strange compared to normal maps. There are lots of fractures and transition between different orientations is not as smooth as ordinary maps. The orientation selectivity is low and the shape of the afferent RFs also looks rather blurred. Figure 4.23 shows the stability of weight value during self-organization. It can be observed that initially there are some large oscillations but after 5000 iterations the weights become quite stable. Compared to previous experiment (figure 4.23) the weights value is more stable in this case.

65

Although the result is not as good as expected, this experiment shows that orientation selective RFs and structured orientation map could still emerge from LISSOM model with combination of intrinsic plasticity and homeostatic synaptic scaling.

Figure 4.22. Orientation map and afferent RFs of simplified LISSOM network with intrinsic plasticity and homeostatic synaptic scaling after 38000 iterations of training. Intrinsic plasticity parameters are: a = 13, b= -4, learning rate=0.0002, target average firing rate=0.01. Synaptic scaling parameters are adjusted

gradually: Cβ =0.0085; Nβ =0.0085 for the first 500 iterations, after 500 iterations Nβ =0.0075, after

2000 iterations Cβ =0.0065, after 4000 iterations Cβ =0.005, after 8000 iterations Nβ =0.003, after

20000 iterations Cβ =0.002; Atarget=0.04 for the first 1000 iterations, after 1000 iterations Atarget=0.035,

after 2000 iterations Atarget=0.03, after 3000 iterations Atarget=0.025, after 4000 iterations Atarget=0.020, after 5000 iterations Atarget=0.015, after 6000 iterations Atarget=0.013, after 7000 iterations Atarget=0.011, after 8000 iterations Atarget=0.01.

Figure 4.23. Plot of the absolute sum of weights value of the center V1 neuron during 38000 iterations.

66

Chapter 5 Discussion and Future Works

5.1. Discussion

5.1.1. Intrinsic Plasticity and LISSOM Models

Regularization of neuron’s excitability plays an important role in the self-organization process of LISSOM models. In typical models, neuron’s excitability is adapted by gradually adjusting the piecewise-linear threshold function during self-organization. Initial investigation presented in section 4.1 indicates that intrinsic plasticity mechanism can be successfully integrated into LISSOM model to replace the manual threshold adjustment and regulate the neuron’s excitability more robustly. The plots of V1 neurons activity and distribution of firing rate shown in figure 4.5 and 4.6 also verify that the neurons with intrinsic plasticity can produce sparser firing activity and achieve approximately exponential distribution of firing rate. This is consistent with observations conducted in biological visual cortex when presented with natural scenes [1]. The models with intrinsic plasticity can develop good orientation map that has smooth transition between different orientations and contain features similar to maps found in animal visual cortex (figure 4.1.b, 4.11.b, 4.14). Moreover, compared to the maps with manual threshold tuning, increased selectivity was observed in the maps with intrinsic plasticity. Orientation selectivity was increased primarily because each neuron maintains its own separate activation function so it can regulate its own excitability depending on the amount of input received. The neurons that receive less input will have lower activation threshold and vice versa. On the other hand, the neurons with globally tuned threshold function should regulate their excitability uniformly no matter how much input they receive, therefore some of the neurons might not be very selectivity because they didn’t have much chance to learn while the threshold was low. The use of global threshold adaptation implies that all neurons should learn and adapt at the same speed throughout the self-organization process. This assumption might be too restrictive and limiting, because once the neurons lag behind their counterpart in the beginning of learning stage, they will never recover their learning ability and lost their plasticity forever. Intrinsic plasticity can increase the neurons selectivity to single orientation because each neuron can now adapt their

67

activation threshold individually independent of the others. So the neurons which did not have chance to learn in the beginning can still learn an association with input pattern in later stage. This might be beneficial in terms of information maximization because every neuron will be utilized up to its maximum capacity so it can still contribute to the whole network [1, 24, 27]. Moreover, the ability to regulate individual neuron’s excitability also proven to be effective in solving the problem found in LISSOM with no shrinking excitatory radius described in section 4.4. Another distinct feature of the maps with intrinsic plasticity compared to the old maps is the size of orientation column spacing (the thickness of orientation-selective patches). The maps with intrinsic plasticity tend to have wider orientation column spacing (i.e. thicker orientation-selective patches). Measurements from animal visual cortex also reveal similar maps with wide orientation column spacing [5]. Consequently, it can be said that the maps with intrinsic plasticity is more closely resemble real animal maps than the map with manual threshold adjustment. The exact reason of why the map with intrinsic plasticity could have wider column spacing is remain to be investigated further. Comparison of detailed step-by-step formation of the orientation map presented in section 4.1.3 reveals another benefit of using intrinsic plasticity. For the first 500 iterations, there is not much noticeable difference in the map of both networks. Some oriented column patterns begin to emerge on both maps and orientation selectivity also increasing. During the next iterations, the map formation process differs greatly, leading to different final orientation map. With intrinsic plasticity, the structure of the orientation map is relatively stable and does not fluctuate greatly during self-organization. Large orientation-selective patches appear in the beginning 1000 iterations and then gradually shrink into smaller patches with slightly different orientations. The orientation selectivity of the neurons is also increasing constantly with the largest rise in the first 2000 iterations. Different pattern of self-organization is observed in the network without intrinsic plasticity. During the first 5000 iterations, the orientation map reorganizes several times into substantially different pattern before it begins to settle down and form quite stable pattern. This difference might be related to the speed and timing of threshold adaptation. As can be seen in figure 4.7 and 4.8, the speed of threshold adaptation using intrinsic plasticity is not linear. For the first 1000 iterations, the threshold was translated rapidly to the right in order to counteract the rapid increase of activity due to high Hebbian learning rate, and thus substantially reducing the neuron’s excitability. This is reflected in the formation of orientation map for the first 1000 iterations, in which

68

large orientation-selective patches appear and orientation selectivity increases rapidly. At this stage, most neurons have already learn an association with specific orientation, and because the neuron’s excitability is not as high as before, subsequent associative learning did not change the neuron’s orientation preference dramatically. Hence, during the next iterations relatively smooth transitions can be observed in the formation of the orientation map. In order to test whether using slower learning rate would result in unstable transitions of the orientation map, simulations with several times slower learning rate was performed. As reported in section 4.2.2, with slower intrinsic plasticity learning rate, the network indeed went through substantial reorganization during the first several thousands iterations, similar to the network with manual threshold adjustment. This result shows that smooth transitions during self-organization did not miraculously appear just by applying intrinsic plasticity into LISSOM network, but it also depends on careful choice of intrinsic plasticity learning rate (and also initial threshold position). Results reported in section 4.2.1 indicate that initial threshold setting of intrinsic plasticity also affect the process of self-organization. If we set the initial threshold too low or too high, the network might fail to organize properly. The experiments also point out that the choice of initial gradient/slope of the sigmoid (determined by variable a) greatly affects the resulting map more than the choice of initial position of the sigmoid (variable b). An interesting behavior can be observed in the fourth experiment reported in section 4.2.1. In this experiment, the initial threshold is positioned slightly farther to the right (using b=-6 instead of b=-4, and the initial gradient value, a=13, is not changed). This means that initially V1 neurons are less excitable and have very weak activity because of high threshold value. Surprisingly, even with very weak initial V1 activity, the network can still learn correlations in the input patterns and develop good orientation map. Initially, the average firing rate of V1 neurons are very low (below 0.01, as can be seen in the plot in section 4.2.1). During the first 2000 iterations, intrinsic plasticity mechanism slowly restores the neurons excitability and at the same time the neurons also learns quickly from input patterns because the Hebbian learning rate was high at that moment. Gradually the activity becomes stronger and reached quite high level of average firing rate (around 0.02) after 6000 iterations, before they slowly decreasing again to reach the target value of 0.01. This pattern of neurons activity is different with the normal pattern of activity in typical LISSOM model, in which the neurons have very high activity in the beginning and gradually decreasing as self-organization progresses. The relevance of this result to real biological development of cortical neurons is not clear, because to our knowledge there is no definite literature that

69

report the exact pattern of neurons activity during the development, prenatally and postnatally. On the other hand, this result indicates that with intrinsic plasticity mechanism, self-organization of cortical neurons is not very sensitive to initial neuron’s excitability. Another intrinsic plasticity parameter that should be set correctly is the target average firing rate. The result of simulations reported in section 4.2.3 shows that this parameter affects the result of self-organization greatly. The network failed to develop good map when the setting of this value was higher or lower than 0.01. This value cannot be set arbitrarily because it is constrained by input activity. The statistics of input pattern and normalization of synaptic weights performed after applying Hebbian rule might also determine the possible range of average activity. During input presentations, the oriented Gaussian input patterns with fixed intensity (scale) cover the whole retina space uniformly, so the strength of overall activations coming from retina to each V1 neuron is roughly the same. The afferent connection weights from retina to each V1 neuron is adapted through Hebbian learning and normalized to 1. Because of this normalization, the activation of V1 neurons (before passing through threshold function) is constrained to be within certain range, proportional to the strength of the inputs coming from retina. Therefore, when the activation threshold is moved to increase or decrease the average activity of V1 neurons during the learning process, the previously learned receptive field will also be disrupted. To illustrate this point more clearly, suppose that we have a trained network that already form a good orientation map and receptive fields. The average activity of V1 neurons in this network is 0.01. Then, we retrain this network using Hebbian learning and intrinsic plasticity with twice higher target average activity (0.02) for several iterations. Because the target average activity is higher than current average activity, intrinsic plasticity mechanism will increase the neuron’s excitability by shifting the sigmoid threshold to the left, so the neurons can have more activation. This also means that the neurons can be easily activated by other oriented patterns different from their RFs orientation. Hebbian learning is still operating here, so when the neurons output activity increase, the connection weights also become strongly adapted to new input patterns. The weights value is normalized to 1, so the contribution of new pattern to the weights value is significant. In this way, some of previously learned receptive fields will be blurs out forming spherical unselective RFs. Higher average activity can be achieved but also destroying the map organization at the same time. From this observation, we might speculate that in order to control the target average activity arbitrarily, we should adjust both the activation threshold and the weights normalization value. One possibility is to use homeostatic synaptic scaling

70

mechanism to replace the divisive normalization. However, because of limited time, the sensitivity of the network with intrinsic plasticity and synaptic scaling to the value of target average firing rate has not been tried. This issue is also related to the problem of training the network using input with different scale (intensity). Every time we train the network using input with different scale, we should use different initial threshold and target average activity for the intrinsic plasticity mechanism. Several attempts have been made to find an algorithm which can automatically find the proper initial value of this parameter for any input scales, however none of them is proven to be good enough. Even though it is rather difficult to set the parameters of intrinsic plasticity so it can work properly, other experiments reported in section 4.3 and 4.4 have shown that this mechanism is really beneficial to improve several aspects of LISSOM models. Better orientation map could be obtained when intrinsic plasticity was applied to LISSOM with LGN ON/OFF channel (section 4.3). Moreover, in that experiment it has been shown that even though the network with manual threshold adaptation could develop good orientation map, the excitability of V1 neurons might become very low after self-organization. This problem might happen because the threshold was not carefully tuned. On the other hand, the network with intrinsic plasticity could maintain stable firing rate level throughout self-organization process, so in the end, V1 neurons can still respond strongly to given input pattern. Maintaining strong response and stable average activity is important, especially when we consider transmitting the activations to higher layers like V2 or V4. If the signal that propagates from V1 is strong enough then it will not be lost in the midst of transmission to higher layers. It has been observed in real biological experiment that the structure of the orientation map in ferret primary visual cortex is remarkably stable during the development stage from the earliest age until maturation. This suggests that orientation map in animals is already established in very early stage of the development and remain stable over time [9]. As described in section 2.3.4 and section 4.4, although the orientation map produced by LISSOM model is similar with the map measured in animal, the detailed self-organization process of the orientation map in LISSOM is quite different with the development of the map in animals. In the beginning of self-organization, the orientation map of LISSOM model repeatedly reorganizes itself and the orientation column spacing shrinks dramatically as the map develops. Initial attempts have been made to improve this by keeping the lateral excitatory radius fixed and speed up the Hebbian learning rate reduction. Orientation map still

71

develop using this approach, however some neurons fail to learn and their afferent weights are still random because they didn’t get enough activation in the beginning and the global activation threshold become too high afterwards. Results presented in section 4.4 shows that after applying intrinsic plasticity to this network, better orientation map was obtained and there are no more dead neurons. This improvement could be gained partly because intrinsic plasticity mechanism adjusts the activation for each neuron separately. The development of the orientation map is also relatively stable. Pattern of oriented patches appears smoothly without any noticeable jumps and the pattern does not change dramatically afterward. This result is, to a certain extent, looks similar to the development of the map in animal.

5.1.2. Synaptic Plasticity and LISSOM Models

The results reported in section 4.5 have shown that, to some extent, it is possible to apply homeostatic synaptic scaling mechanism of Sullivan and de Sa [25, 26] to replace the divisive normalization used in LISSOM model. Nevertheless, it is very difficult to tune the parameters of the algorithm. If the homeostatic learning rate is too high, the weights value will decrease very quickly. If the learning rate is too slow, the weights value will become very large and the network becomes saturated. Pretty good result could be obtained after gradually adjusting the learning rate and target average firing rate during the course of self-organization. However, the resulting orientation map still does not look as good as usual. The last experiment demonstrates that with the proper parameter settings, it is possible to combine both intrinsic plasticity and homeostatic synaptic scaling in LISSOM visual cortex model, even though the result is not as good as when using intrinsic plasticity and divisive normalization.

5.2. Future Works From the results discussed above it is clear that there are still lots of issues concerning homeostatic plasticity in LISSOM models that needs further investigations. In this project, only two computational models (intrinsic plasticity and homeostatic synaptic scaling) that were implemented and applied to LISSOM model. There are other models such as BCM [8, 34, 35] (including some of its variants) and STDP (Spike Timing Dependent Plasticity) [17, 32] that can be implemented in the future. STDP will need spiking neuron model, so in principle it should be possible to integrate this mechanism into SLISSOM model (Spiking LISSOM) [10].

72

Intrinsic plasticity mechanism of Triesch [27] has been shown to be applicable to replace manual threshold adjustment in LISSOM and could improve some aspect of the model. This mechanism was only applied to V1 neurons, but in principle, intrinsic plasticity can be also applied to other neurons including retinal receptors and LGN neurons. It is possible that by using this kind of regulation on retina or LGN, the ability of the network to deal with variability in input scale (intensity) can be improved considerably. This ability is important, especially when using natural images as training pattern. Currently, LISSOM model trained with natural images does not develop the orientation map as good as the model trained with artificial pattern. Better result should be possible if some kind of homeostatic regulation is applied when training the model using natural images. The role of intrinsic plasticity in other LISSOM models such as model with two retinal sheets, model of color, ocular dominance, disparity, and direction selectivity also remains to be investigated. It has been shown that orientation map self-organization in LISSOM model with intrinsic plasticity and no shrinking lateral excitatory radius looks very similar to the process of map development in animal visual cortex. This result was obtained when the model was trained using oriented Gaussian. More realistic training pattern such as combination of retinal waves and natural images (to simulate prenatal and postnatal training pattern in animal) should be tried to produce more biologically convincing result. One difficulty of applying intrinsic plasticity to LISSOM model is choosing the right parameter values. It might be possible to devise an algorithm that adapts some of these parameters automatically based on other values that reflect the state of the network. For example intrinsic plasticity learning rate could be adjusted depending on how fast the network connections weights changes and target average activity could be determined from the statistics of inputs coming to the neurons. Initial experiments shows that it is possible to integrate homeostatic synaptic scaling of Sullivan and de Sa [25] into LISSOM model. However, it is even more difficult to tune the parameters so that the network could produce good orientation map. The homeostatic learning rate and target average activity should be decreased gradually during the course of self-organization. Currently, it was done manually by trial and error. It might be possible in the future works to regulate these parameters automatically by another learning process.

73

Chapter 6 Conclusions

The experiments conducted in this project have demonstrated that homeostatic plasticity mechanism, particularly intrinsic plasticity, can be successfully integrated into LISSOM model of visual cortex to make it more biologically realistic and also improves some of its behavior. Two computational model of homeostatic plasticity, intrinsic plasticity mechanism proposed by Triesch [27] and homeostatic synaptic scaling proposed by Sullivan and de Sa [25, 26] have been implemented and applied to basic LISSOM model. Several experiments were conducted to test the resulting self-organized map and the behavior of the network under this homeostatic mechanism. The orientation map that is more closely resemble the map measured in animal visual cortex was obtained after applying intrinsic plasticity mechanism into LISSOM model. Moreover, this mechanism also proven to be effective in solving the problem of dead-neurons that fails to organize properly in LISSOM model without shrinking lateral excitatory radius. The use of intrinsic plasticity also eliminates the need of manually tuning the activation threshold and therefore makes the model more reliable and more biologically plausible. Quite extensive experiments have been performed to test the sensitivity of intrinsic plasticity parameters setting to the resulting self-organized map. It has been found that parameter settings affect the process of self-organization quite significantly and it is rather difficult to find the optimal parameters that work well with specific model. Moreover, in LISSOM model, certain values such as Hebbian learning rate is not constant but gradually adjusted during self-organization. This might indicate that perhaps intrinsic plasticity learning rate should also be adapted regularly to balance the speed of Hebbian learning. It might be possible to devise another kind of regularization rule that automatically adjust intrinsic plasticity parameters based on current state of the network. Initial experiments with homeostatic synaptic scaling in LISSOM model indicate that it is very difficult to apply this algorithm directly because there are too many fluctuations in the weights values during self-organization. The fluctuations can be slightly reduced by adapting the homeostatic parameters gradually, but this process is currently done manually and based on trial and error. In order to integrate it into LISSOM more fully, some kind of automatic adjustment of the parameters should be invented.

74

Bibliography [1] Baddeley, R., et. al., Responses of Neurons in Primary and Inferior Temporal Visual Cortices to Natural Scenes. Proc. Roy. Soc. of London. B, 264:1775-1783, 1998. [2] Bednar J. A., Choe Y., De Paula J., Miikkulainen R., Provost J., and Tversky T., Modeling cortical maps with Topographica. Neurocomputing: 1129-1135, Presented at the 2003 Computational Neuroscience Meeting, 2004. - http://www.topographica.org [3] Bednar J. A. and Miikkulainen R., Self-organization of spatiotemporal receptive fields and laterally connected direction and orientation maps. Neurocomputing, 52-54:473–480, 2003. [4] Bednar J. A., and Miikkulainen R., Tilt Aftereffects in a Self-Organizing Model of the Primary Visual Cortex. Neural Computation, 12:1721–1740, 2000. [5] Blasdel, G. G. Orientation selectivity, preference, and continuity in monkey striate cortex. Journal of Neuroscience, 12:3139—3161, 1992 [6] Butko N. and Triesch J., Exploring the Role of Intrinsic Plasticity for the Learning of Sensory Representations. ESANN 2006 proceedings - 14th European Symposium on Artificial Neural Networks Bruges, 2005. [7] Butko N. and Triesch J., V1 maps from sparse neurons with intrinsic plasticity. 12th Joint Symposium on Neural Computation, 2005. [8] Castellani G. C., Quinlan E. M., Cooper L. N., Shouval H. Z., A biophysical model of bidirectional synaptic plasticity: Dependence on AMPA and NMDA receptors. PNAS vol. 98, no 22: 12772-12777, 2001. [9] Chapman B., et. al., Development of Orientation Preference Maps in Ferret Primary Visual Cortex. The Journal of Neuroscience, 16(20):6443–6453, 1996. [10] Choe Y. and Miikkulainen R. Contour Integration and Segmentation with Self-Organized Lateral Connections, Biological Cybernetics 90:75-88, 2004.

75

[11] Davis, G.W., Homeostatic Control of Neural Activity: From Phenomenology to Molecular Design, Annual Review Neuroscience, 29:307-323, 2006 [12] Davis, G.W., Bezprozvanny, I., Maintaining The Stability of Neural Function: A Homeostatic Hypothesis, Annual Review Physiology, 63:847-69, 2001 [13] Desai N., Homeostatic plasticity in the CNS: synaptic and intrinsic forms. Journal of Physiology – Paris 97:391-402, 2003. [14] Desai N., Rutherford L. C., Turrigiano G. G., BDNF regulates the intrinsic excitability of cortical neurons. Learning & Memory 6:284-291, 1999. [15] Di Paolo E. A., Homeostatic adaptation to inversion of the visual field and other sensorimotor disruptions. From Animals to Animats 6. Proceedings of the VI International Conference on Simulation of Adaptive Behavior: 440-449. [16] Di Paolo, E. A., Organismically-inspired robotics: homeostatic adaptation and teleology beyond the closed sensorimotor loop, Dynamical systems approach to embodiment and sociality, 2003. [17] Gutig R., et. al., Learning Input Correlations through Nonlinear Temporally Asymmetric Hebbian Plasticity, The Journal of Neuroscience, 23(9):3697-3714, 2003. [18] Miikkulainen, R. Self-organizing process based on lateral inhibition and synaptic resource redistribution. In Kohonen, T., et. al., editors, Proceedings of the 1991 International Conference on Artificial Neural Networks, 415–420, 1991 [19] Miikkulainen R., Bednar J. A., Choe Y., and Sirosh J., Computational maps in the visual cortex. Springer, Berlin, 2005. [20] Miikkulainen, R., Bednar, J. A., Choe, Y., and Sirosh, J. Self-organization, plasticity, and low-level visual phenomena in a laterally connected map model of the primary visual cortex. In Goldstone, R. L., et. al., editors, Perceptual Learning, vol. 36 of Psychology of Learning and Motivation, 257–308, 1997 [21] Shouval H. Z., Bear M. F., Cooper L. N., A unified model of NMDA receptor-dependent bidirectional synaptic plasticity. Proc Natl Acad Sci USA, 2002.

76

[22] Sincich, L. C., and Blasdel, G. G. Oriented axon projections in primary visual cortex of the monkey. The Journal of Neuroscience, 21:4416–4426, 2001 [23] Sirosh J., Miikkulainen R., and Bednar J.A. Self-organization of orientation maps, lateral connections, and dynamic receptive fields in the primary visual cortex. In Lateral Interactions in the cortex: Structure and Function, UTCS Neural NetRes Grp. Electronic book, ISBN 0-9647060-0-8, 1996. [24] Stemmler, M., Koch, C., How Voltage-Dependent Conductances can Adapat to Maximize the Information Encoded by Neuronal Firing Rate. Nature Neuroscience, 2:521-527, 1999. [25] Sullivan T. and de Sa. V. R., A Self-Organizing Map with homeostatic synaptic scaling. 14th Annual Computational Neuroscience Meeting, CNS, 2005. [26] Sullivan T. and de Sa. V. R., Homeostatic synaptic scaling in Self-Organizing Maps. Workshop on Self-Organizing Maps, Paris, 2005. [27] Triesch J., A gradient rule for the plasticity of a neuron's intrinsic excitability. ICANN 2005, LNCS 3696:65-70, 2005. [28] Triesch J., Synergies between intrinsic and synaptic plasticity in individual model neurons. Advances in Neural Information Processing Systems 17. MIT Press, 2005. [29] Turrigiano, G. G., Homeostatic plasticity in neuronal networks: the more things change, the more they stay the same. Trends in Neurosciences 22: 221-227, 1999. [30] Turrigiano G. G. and Nelson S., Hebb and homeostasis in neuronal plasticity. Current Opinion in Neurobiology 10:358-364, 2000. [31] Turrigiano G. G. and Nelson S., Homeostatic Plasticity in the Developing Nervous System. Nature Review: Neuroscience, Vol. 5, 2004. [32] van Rossum M. C. W., Bi, G. Q., Turrigiano, G. G., Stable Hebbian Learning from Spike Timing-Dependent Plasticity, The Journal of Neuroscience, 20(23):8812-8821, 2000.

77

[33] Williams H. and Noble J., Homeostatic plasticity improves signal propagation in Continuous-time Recurrent Neural Networks. Sixth International Workshop on Information Processing in Cells and Tissues, IPCAT, 2005. [34] Yeung L. C., Harel Z. S., Blais B. S., Cooper L. N., Synaptic homeostatis and input selectivity follow from calcium-dependent plasticity model. PNAS vol. 101, no 41:14943-14948, 2004. [35] Yeung L. C., Blais B. S., Cooper L. N., Shouval H.Z., Metaplasticity and the Unified Calcium Model lead to input selectivity in spiking neurons. [36] Zhang W. and Linden D. J., The Other Side of the Engram: Experience-Dependent Changes in Neuronal Intrinsic Excitability. Nature Reviews Neuroscience, 4:885-900, 2003.