Upload
others
View
6
Download
0
Embed Size (px)
Citation preview
1
Jacques-Olivier KleinIEF, Univ. Paris Sud/ CNRS
Weisheng ZhaoEmbedded Computing Lab, CEA LIST
Nanocomponent based neuromimetic memoires
Workshop: Innovative Memory Technologies, 24-06-200 9, Grenoble, France
2
Plan
• Why Nano Neuromimetic Memory?
• What are its main characteristics?
• Learning vs. Programming
• An implementation example
• Challenges
3
Von Neumann architecture and CMOS technology
•Stored program computer (Turing)
•Predetermined algorithms define capabilities (software)
•Electronics implementation of Boolean operators
•Based on well defined reproducible states
•“Bottleneck” : Throughout between the ALU and memory
•Low fault tolerance
•High speed
•Low power (limited by leakage power)
•Low mismatch and process Variation (difficult)
•Low cost (not true now!)
•High density (limited by atomic distance scale)
F. Wanlass, US Patent 3356858, 1967Memory hierarchy
CMOS technology
4
Nanotechnologies: Next big thing after CMOS
IBM, 2008
Moore’s law will continue?Von Neumann will continue?
•Nanotube
•Nanowire
•Graphene
•Memristor
•Domain Wall
•Molecular
•……
5
Why Neural Network?
New computing architectures are required!
Nanotechnology
• Small size (some nano meters)
• Self-assembly fabrication
Significant variation and high defect ratio!
Sample I Sample II
Multiple Carbon nanotube stripes
field-effect transistor (CNTFET)
Nanotubes
D
S
D
S
Q. Cao et al., Nature, 2008
G G
6
Why Neuromimetic Memoires? Nanotechnology
•Maximally Parallel computing (*)
•Based on the elements with large diversity (*)
•Extremely Low power
•Learning capabilities (e.g. environment)
•High fault tolerance (*)
•“Bottleneck” : Neuromimetic memories: synapse
Courtesy K.Meier
Based on CMOS technology, one synapse requires at least eight transistors
-K.Boahen, Stanford, Neural Computation, 2007
Neural Computation
7
Nanocomponent based Neuromimetic Memoires?
Artificial neural network
Biological neural network
•Historical memory
•Analog levels
•Extremely high density(104 synapses per neuron), human brain: 1010/cm2
Synapse
8
Plan
• Why Nano Neuromimetic Memory?
Nanotechnology Neural computation architecture
• What are its main characteristics?
• Learning vs. Programming
• An implementation example
• Challenges
9
I
VT+
VT-
V
RON
ROFF
VT+
VT-
V
Main characteristics required?
•Historical memory
•Multi levels (analog)
•Voltage or current threshold (control)
•Non-volatile
State 0State 1State 2State 3
Static behaviors Dynamic behaviors
10
Memristor
Strukov et al., Nature 453 (2008)
G. Snider et al., Nanoarchi (2008)
L.Chua et al., IEEE TCT 18 (1971)
W: State variable
11
Nanoscale Synaptic transistor
-5V +7V
I-I-
Lai et al., Nano Letter (2008)
The conductance can be configured to arbitrary states dynamically and reversibly by applying a series of Vg pulses with different amplitude, polarity and duration
12
Plan
• Why Neuromimetic Memoires?
• What are their main characteristics?
• Learning vs. Programming
• An implementation example
• Challenges
13
Learning vs. programming
Before
After
N
11
11
--11
N
For all the patterns
Convergence
??
??
??
14
Nano-Neuro-inspired Architectures
Learning circuit based on Memristor and CNTFET
He et al. EL 2008
Desired output
Actual output
Input
)( jjiij XYXW −×=∆ α
Delta learning rule
B. Widrow et al., 1960
2 CNTFET+1 Memristor
Learning pulses
15
Electrical simulation of one learning example
1- Learning pulses2 - Neuron output
3 - Expected output
For learning step = 1 to 6For input pattern = 000 to 111
For column = 1 to 8 (2x3 inputs + 2 thresholds)
16
Plan
• Why Neuromimetic Memoires?
• What are their main characteristics?
• Learning rules vs. Programming
• An implementation example
• Challenges
17
X1+ X1- X2+ X2- V+ V-
Inputs
Neural threshold
∆
V4 X4
Y4Fb4
∆
V5 X5
Y5Fb5
Learning driven by a feedback voltage
V
+VT / HZ / -VT
VT
Nano-Neuro-inspired Architectures
Desired output
Actual output
V+>VT
V-<-VT
Post Synaptic potential
18
Cases included
V
V V
V V
• Cases 2 and 4 : Patent FR0705532, 2007
• Other cases: Patent PCT/FR/050943, 2009
1
32
4 5
Muiti-Wall CNT Optical Gate-CNTFET
CBRAMMemristor
Nano-Neuro-inspired Architectures
J. Borghetti et al., Advanced Materials 2006
S. Dietrich et al., IEEE JSSCC 2007
PrototypeOG-CNTFET: Synapse
CMOS: Neuron
V
V
6
7
OxRAM ?
Phase change Nanowire
Y. Jung et al,. Nano letters(2008)
Baek, IEDM 2004
19
One example
Learning of 7 functions linearly-separable for 3 inputs
Good results
Errors
Distribution of the weights
Journal of Vacuum Science & Technology, 20 (6)
Nano-Neuro-inspired Architectures
V2
MWCNT
56 synapses
20
Plan
• Why Neuromimetic Memoires?
• What are their main characteristics?
• Learning vs. Programming
• An implementation example
• Challenges
21
Challenges: Joint efforts are required
• Nanofabrication
• Circuit design
• Learning rules
• Architecture and system (we are here)
• Prototyping
• Neuroscience (not well understood)
22
Thank you very much!
23
Learning rules: Spiking Time Dependant Plasticity ( STDP)
i j
t
Bi and Poo J. Neurosci. 1998Yang et al., Nature Neuroscience 2008