Upload
ngoanh
View
215
Download
0
Embed Size (px)
Citation preview
James Mnatzaganian & Ross Reinhardt
Neuromorphic Computing OverviewTheory of Neuromorphic ArchitecturesAdvantages and Challenges of Neuromorphic ArchitecturesNeuromorphic Architectural AttemptsMemristor OverviewMemristor FunctionalityMemristive-based Neuromorphic Architectures
11/13/2012 2
Neuromorphic ArchitecturesNeuromorphic Architectures
11/13/2012 3
Modeled from neural circuitry ◦ Artificial neural networks
Abandons Von Neumann architecture◦ Integrates processor with memory
High degree of parallel processing
High efficiency
11/13/2012 4
Biological Model◦ Neuron: Processes received information◦ Axon: Sends outgoing signal to other neurons◦ Synapse: Connection between neurons
Remembers previous state and weight of path◦ Dendrites: Receives incoming signals from other neurons
Information Flow◦ Neurons triggered at a voltage threshold◦ Operate on voltage spikes rather than constant V◦ Input through dendrites, output through axon◦ Learning achieved by synapse memory◦ Synapses adjust weight based on previous states
[1]
11/13/2012 5
Hardware Implementationo Goal: Emulate biological neural networks
[2]
11/13/2012 6
High SpeedParallel ProcessingLow PowerAdaptive Architecture o Can “learn” and operate on incomplete dataRecognition of patterns & matching taskso Difficult for Von Neumann architecturesIntegrates processor and memoryo Eliminates Von Neumann bottleneckHigh densities achievable using memristor-based technologies
11/13/2012 7
Interconnection with existing architecturesComplex design and managementThermal managementEach neuron is independento Topologyo Error checkingDistributed processing and storageSimulated Neuromorphic Architectureso Not true neural networkso High simulation overhead on current technology = low
performanceo Cannot be run in real-time on Von Neumann architectureso Can only simulate small portions of a neural networko Must be run on supercomputers = high power consumption
11/13/2012 8
IBM SyNAPSESystems of Neuromorphic Adaptive Plastic Scalable Electronicso IBM attempt to build scalable architecture similar to the
mammalian braino DARPA funded 2009o Failed to meet goals
[3]
11/13/2012 9
Architectureo One 2x3mm coreo 1024 Axonso 256 Neuronso 262,144 programmable or
65,356 learning synapseso CMOS Technology
[4]
45 nm technology node~500 Transistors / synapseHigh transistor count & layout area Not Scalable to biological levels
11/13/2012 10
Von Neumann bottleneck◦ Separated memory from CPU◦ Single order execution◦ Minimal parallelizability
Dawn of a new era – The memristor◦ Low power◦ High parallelizability◦ Biologically-based
11/13/2012 11
MemristorsMemristors
11/13/2012 12
Theorized by Leon Chua in 1971Demonstrated by HP Labs in 2008“Missing element”
[7]
[9]
Nonlinear two-terminal deviceResistor with a memory element
11/13/2012 13
[8]
[8]
Resistance is a function of applied voltage & timeResistance is based off oxygen vacancies
11/13/2012 14
Nonlinearity is a result of M and i dependency on qMemristor’s state is dependent upon past and present events
[14]
11/13/2012 15
Synapses are used by neurons to pass signals to cellsSTDP – Spike Timing-Dependent Plasticity◦ Spiking from pre- to post-synaptic neuron◦ Determines synaptic weightSynaptic weight – Strength of neuron connectionExcitatory synapses increase membrane voltageInhibitory synapses decrease membrane voltageChanging of weights allows biological systems to learn
[15]11/13/2012 16
Memristance ≈ Synaptic weightCrossbar array◦ Memristors connected at each crosspoint
connecting CMOS pre- and post-neurons◦ Up to 1010 synapses / cm2 (100 nm pitch)
[11]
11/13/2012 17
Memristors + Neuromorphic Memristors + Neuromorphic ComputingComputing
11/13/2012 18
Memristor-Based SynapseSTDP-Based Neuromorphic ArchitectureMemristors + Spin DevicesmrFPGA
11/13/2012 19
Biological model approachMemristor Training Circuit◦ Self-adaptive to environment◦ Real-time synaptic weight adjustmentMulti-synapse training schemeSelf-training mode◦ Dynamic write time
[12]
11/13/2012 20
STDP – Spike Timing Dependent Plasticity◦ Biological process that adjusts the strength of
connections between neuronsChange in weights is a function of excitatory and inhibitory synapsesAnalog architecture utilizing the crossbar structure
[13]
11/13/2012 21
Theoretical Designo Lateral spin valves & Memristorso Proposed by Intel, 2012o May theoretically decrease power consumption by 15-300x compared to CMOSo Terminal voltages in mV = lower powero Still much less efficient than real neurons
[5]
MTJo Magnetic Tunnel Junctiono Two magnetic poles separated by thin insulator
• Nanomagnetso Poles shift due to external magnetic fields
• “Spin” = neuronal spiking• Low power consumption
11/13/2012 22
Memristor-based FPGA Proposed 2011Replaces SRAM interconnectso From 6 transistors to 1 memristoro Current interconnects are 90% of
area, 80% delay, 85% of power consumption
Help close the performance gap between FPGA and ASIC Greater performance with lower development costs
[16]11/13/2012 23
[16]
CMOS scaling will eventually halt◦ Hardware-based improvements neededMemristive-based neuromorphic architectures could be the keyActive research by numerous companies and universities“It’s only a matter of time...”A.I. ???
11/13/2012 24
Memristors act like synapsesA biological model consists of neurons connected via synapsesNeuromorphic architectures are designed to mimic the brainMemristor-based neuromorphic architectures prove to be the most realizable option
11/13/2012 25
[1] Kandel E.R., Schwartz, J.H., Jessell, T.M. 2000. Principles of Neural Science, 4th ed., McGraw-Hill, New York.[2] http://www.darpa.mil/Our_Work/DSO/Programs/Systems_of_Neuromorphic_Adaptive_Plastic_Scalable_Electronics_%28SYNAPSE%29.aspx [3] Mosher, Dave. New Chip Borrows Brain’s Computing Tricks. <http://www.wired.com/wiredscience/2011/08/ibm-synapse-cognitive-computer>. 2011. [4] John V. Arthur, Paul A. Merolla, Filipp Akopyan, Rodrigo Alvarez-Icaza, Andrew Cassidy, Shyamal Chandra, Steven K. Esser, Nabil Imam, William Risk, Daniel
Rubin, Rajit Manohar, and Dharmendra S. Modha, Building Block of a Programmable Neuromorphic Substrate: A Digital Neurosynaptic Core, International Joint Conference on Neural Networks, 2012. Available: http://www.modha.org/papers/IJCNN%202012.pdf
[5] M. Sharad, C. Augustine, G. Panagopolous and K. Roy, " Proposal for Neuromorphic Hardware using Spin Devices", arXiv:1206.3227v4[6] https://www.fp7-nanotec.eu/content/session-7-neuromorphic-computing (Grollier, 2012)[7] Rinky B P. Innovative Blood. Memristor - The Missing Circuit Element . <http://innovativeblood.blogspot.com/2011/02/memristor-missing-circuit-
element.html>. 2012.[8] Williams, Stanley. How We Found the Missing Memristor. IEEE Spectrum. 2008.[9] Kvatinsky, Shahar. Logic Design With Memristors. Technion – Israel Institute of Technology. 2012.[10] Borhetii, Julien, et. al. Memeristive switches enable stateful logic operations via material implication. nature vol 464. doi:10.1038/nature08940. 2010.
<http://www.nature.com/nature/journal/v464/n7290/pdf/nature08940.pdf>.[11] Hyun Jo, Sung, et. al. Nanoscale Memristor Device as Synapse in Neuromorphic Systems. Nano letters doi: 10.1021/nl904092h. 2010.
<http://pubs.acs.org/doi/pdf/10.1021/nl904092h>.[12] Hui Wang; Hai Li; Pino, R.E.; , "Memristor-based synapse design and training scheme for neuromorphic computing architecture," Neural Networks (IJCNN),
The 2012 International Joint Conference on , vol., no., pp.1-5, 10-15 June 2012. doi: 10.1109/IJCNN.2012.6252577. URL: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6252577&isnumber=6252360
[13] Ebong, I.; Deshpande, D.; Yilmaz, Y.; Mazumder, P.; , "Multi-purpose neuro-architecture with memristors," Nanotechnology (IEEE-NANO), 2011 11th IEEE Conference on , vol., no., pp.431-435, 15-18 Aug. 2011. doi: 10.1109/NANO.2011.6144522. URL: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6144522&isnumber=6144287
[14] Massimiliano Verscae. (2011, August 9). “Fuzzy logic and memristive hardware,” neurdo. Available: http://www.neurdon.com/2011/08/09/development-of-massively-parallel-fuzzy-adaptive-neural-algorithms-for-efficient-implementation-on-memristive-hardware/
[15] Robert Stufflebeam. "Neurons, Synapses, Action Potentials, and Neurotranmission." Consoritum on Congnitive Science Instruction. <http://www.mind.ilstu.edu/curriculum/neurons_intro/neurons_intro.php>. 2008.
[16] Cong, J.; Xiao, B.;, "mrFPGA: A Novel FPGA Architecture with Memristor-Based Reconfiguration," Nanotechnology (IEEE-NANO), 2011 IEEE International Symposium on Nanoscale Architectures
11/13/2012 26
BACKUPBACKUP
11/13/2012 27
Synaptic Plasticityo Excitatory & Inhibitory synapses
o Action potential (spike) at an excitatory synapse strengthens a connection, while spikes at inhibitory synapses lower the probability that a post-synaptic cell will spike
o Weight of synapse changes in response to inputs forming new pathwayso Responsible for network reorganization and learning
11/13/2012 28
[9]
IMPLY – pIMPq == (NOTp)ORqCombinatorial logicPerforms & stores logicReduced area
MAGIC – Memristor Aided LoGICLogic inside memorySeparate input and output memristors
Hybrid – Memristors + transistorsComputational onlyFast for Boolean logicCMOS compatibility
Many types of memristor-based logic• Fast computational memory• Logic + memory• Linear / nonlinear / somewhere in-between
11/13/2012 29