12
3/20/2013 1 Biologically Inspired Algorithms for Micro and Nano System Design Prof. Pinaki Mazumder, Univ. of Michigan Ann Arbor, MI 48105 Outline of the Talk Evolutionary Approach to Distributed VLSI Layout Synthesis Self Healing of VLSI Chips by Neurally Self-Healing of VLSI Chips by Neurally Inspired Hardware Methods • Neuromorphic Nanoarchitecture using Cellular Nonlinear Network Model Reinforcement Learning Hardware Biologically Inspired Computing Biologically Inspired Computing Neural Networks (Connectionism) Evolutionary Computation (Survival of the fittest) Genetic Genetic Social Systems Memb- rane Artificial I Wetware Computing DNA Molecular MODELS MODELS TECHNOLOGIES TECHNOLOGIES Multilayer Perceptrons Hopfield Networks Cellular Neural Networks Nanoarchitectures Self-assembled Quantum Dot Array for image processing, video motion detection, & spatial & temporal filtering (Gabor, VTF) Self-healing, Self-repairable VLSI Chips Genetic Programming Genetic Algorithms Grammatical Evolution VLSI Cell Placement Chip Wire Routing Chip Floorplanning Chip Testing Logic Synthesis Swarm Ant Immune Systems My research employs BIC in Micro & Nano Systems Design Reinforcement Learning Chips with Action-Critic Model to perform Adaptive Dynamic Programming 500,000,000 MOSFETs Year Transistors 4004 1971 2.25 K 8080 1974 5 K 8086 1978 29 K 286 1982 120 K Intel386™ 1985 275 K Intel486™ 1989 1.1 M Pentium® 1993 3.1 M Pentium® II 1997 7.5 M Pentium® 4 2000 42 M Itanium® 2002 220 M Itanium® 2 2003 410 M Massive Massive Constrained Constrained Permutation Permutation Problem Problem with over with over 250,000 250,000 Elements. Elements. CAD Design Environment Changed from Uni-processor (Vax) N k f W k i (NW) Network of Workstations (NOW) Network of Workstations (NOW) Connected by High Connected by High-speed LAN speed LAN to Network of Workstations (NoW) 1. High Communications costs through Local Area Network (LAN) 2. Distributed Cohesive Search-space Solver for VLSI Chip Layouts 3. Linear Speed-up and High-quality Chip Layouts. 4. Increasing Chip Complexity & Complex Search-space. e Objective Function Simulated annealing uses a single configuration Simulated annealing uses a single configuration Why Genetic Algorithms are Better Suited for NoW Platform than Stochastic Algorithms Global Optimum Global Optimum Local Optimum Local Optimum Solution Configurations Cost of th GA uses population of configurations Adaptive Parallel Fast Distributed Local Optimum Stochastic Sequential Slow Single-processor Global Optimum Lesson #1: GA’s are Suited for Distributed Optimization

Outline of the Talk - web.eecs.umich.eduweb.eecs.umich.edu/~mazum/BIC-Mazumder.pdf · Biologically Inspired Algorithms for Micro and Nano System Design Prof. Pinaki Mazumder, Univ

  • Upload
    others

  • View
    1

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Outline of the Talk - web.eecs.umich.eduweb.eecs.umich.edu/~mazum/BIC-Mazumder.pdf · Biologically Inspired Algorithms for Micro and Nano System Design Prof. Pinaki Mazumder, Univ

3/20/2013

1

Biologically Inspired Algorithms for Micro and Nano System Design

Prof. Pinaki Mazumder, Univ. of Michigan

Ann Arbor, MI 48105

Outline of the Talk

• Evolutionary Approach to Distributed VLSI Layout Synthesis

• Self Healing of VLSI Chips by Neurally• Self-Healing of VLSI Chips by NeurallyInspired Hardware Methods

• Neuromorphic Nanoarchitecture using Cellular Nonlinear Network Model

• Reinforcement Learning Hardware

Biologically Inspired Computing Biologically Inspired Computing

Neural Networks(Connectionism)

Evolutionary Computation

(Survival of the fittest)

…Genetic Genetic

Social Systems Memb-

rane

ArtificialI

WetwareComputing

DNA

Molecular

MODELSMODELS TECHNOLOGIESTECHNOLOGIES

MultilayerPerceptrons

HopfieldNetworks

CellularNeural Networks

NanoarchitecturesSelf-assembled Quantum Dot Arrayfor image processing,video motion detection,& spatial & temporal filtering (Gabor, VTF)

Self-healing,Self-repairableVLSI Chips

Genetic Programming

Genetic Algorithms

Grammatical Evolution

• VLSI Cell Placement• Chip Wire Routing• Chip Floorplanning• Chip Testing• Logic Synthesis

Swarm Ant

Immune Systems

My research employs BIC in Micro & Nano Systems Design

Reinforcement Learning Chipswith Action-CriticModel to performAdaptive DynamicProgramming

500,000,000 MOSFETs

Year Transistors

4004 1971 2.25 K

8080 1974 5 K

8086 1978 29 K

286 1982 120 K

Intel386™ 1985 275 K

Intel486™ 1989 1.1 M

Pentium® 1993 3.1 M

Pentium® II 1997 7.5 M

Pentium® 4 2000 42 M

Itanium® 2002 220 M

Itanium® 2 2003 410 M

Massive Massive ConstrainedConstrainedPermutationPermutationProblem Problem with over with over 250,000250,000Elements.Elements.

CAD Design EnvironmentChanged from Uni-processor (Vax)

N k f W k i (N W)

Network of Workstations (NOW)Network of Workstations (NOW)Connected by HighConnected by High--speed LANspeed LAN

to Network of Workstations (NoW)

1. High Communications costs throughLocal Area Network (LAN)

2. Distributed Cohesive Search-spaceSolver for VLSI Chip Layouts

3. Linear Speed-up and High-quality Chip Layouts.

4. Increasing Chip Complexity & ComplexSearch-space.

e Obj

ective

Fun

ction Simulated annealing uses a single configurationSimulated annealing uses a single configuration

Why Genetic Algorithms are Better Suited for NoW Platform than Stochastic Algorithms

Global OptimumGlobal Optimum

Local OptimumLocal Optimum

Solution Configurations

Cost

of

th

GA uses population of configurations

• Adaptive• Parallel• Fast• Distributed• Local Optimum

• Stochastic• Sequential• Slow• Single-processor• Global Optimum

Lesson #1:GA’s areSuited forDistributedOptimization

Page 2: Outline of the Talk - web.eecs.umich.eduweb.eecs.umich.edu/~mazum/BIC-Mazumder.pdf · Biologically Inspired Algorithms for Micro and Nano System Design Prof. Pinaki Mazumder, Univ

3/20/2013

2

Adjustments of crossover, inversion, mutation rates; population size; selection criteriapopulation size; selection criteria

Lesson #2: GA’s have Many Parameters Requiring a Meta Optimization Process

Infeasible crossover

Feasible crossovers

Lesson #3: O(N ) Time is Required to Generate Feasible Crossover Operations.Lesson #4: GA is Fast, but Tends to Optimize LocallyLesson #4: GA is Fast, but Tends to Optimize Locallyfor Problems with Very Large Search Space. for Problems with Very Large Search Space.

Multi-Dimensional Search Space forMacro-Cell Placement with Translation,Mirroring, Rotation, etc. Operations.

Lesson #5: GA is Effective for Many RealWorld Problems that Require MULTI-DIMENSIONAL CROSSOVER OPERATORS

Page 3: Outline of the Talk - web.eecs.umich.eduweb.eecs.umich.edu/~mazum/BIC-Mazumder.pdf · Biologically Inspired Algorithms for Micro and Nano System Design Prof. Pinaki Mazumder, Univ

3/20/2013

3

IncrementalIncrementaliimprovementmprovementis is required required towards thetowards theeend nd of Search of Search for for Global Global MinimaMinima

Lesson #6: “GA + SA” Performs Better than Pure GA

CAD Design EnvironmentChanged from Uni-processor (Vax)To Network of Workstations (NOW)

Network of Workstations (Network of Workstations (NoWNoW))Connected by HighConnected by High--speed LANspeed LAN

1. High Communications costs throughLocal Area Network (LAN)

2. Distributed Cohesive Search-spaceSolver for VLSI Chip Layouts

3. Linear Speed-up and High-quality Chip Layouts.

4. Increasing Chip Complexity & ComplexSearch-space.

CHILDREN PROCESSORSCHILDREN PROCESSORSCONTROLLER PROCESSORCONTROLLER PROCESSOR CHILDREN PROCESSORSCHILDREN PROCESSORS

Communication Time v.Epoch Length

Lesson #7: GA’s are Suited for Distributed Optimization

e Obj

ective

Fun

ction

GA uses population of configurations

Simulated annealing uses a single configuration

Why Genetic Algorithms are Better Suited for NOW than Stochastic Algorithms

Solution Configurations

Cost

of

th

• Adaptive• Parallel• Fast• Distributed• Local Optimum

• Stochastic• Sequential• Slow• Single-processor• Global Optimum

Why GA’s are Suited for Distributed Optimization?

Page 4: Outline of the Talk - web.eecs.umich.eduweb.eecs.umich.edu/~mazum/BIC-Mazumder.pdf · Biologically Inspired Algorithms for Micro and Nano System Design Prof. Pinaki Mazumder, Univ

3/20/2013

4

VARIOUS ASPECTS OF GA’S FOR VLSIDESIGN, TEST AND LAYOUT OPTIMIZATION

Opportunities for Future Research:

Mathematical Modeling of Genetic Process

Simulated annealing uses Markov Chain togive polynomial time solution.

Probability analysis & Empirical modelingis needed for sequential and distributedGenetic optimization Genetic optimization.

OPEN PROBLEM #1:OPEN PROBLEM #1: Markov Chain can be used to model the Simulated Annealing by representing each solution configuration by a State in the Markov Chain and by using the probability of an Incremental transformation as the Transition Probability between different states.

It will lead to a Markov Chain of Length, L such that at the end one can obtain near Global Optimal solution. The Length, L can be controlled by selecting suitable Annealing Parameters and Inner Loop stopping criteria up to 4 or 5 Variables only.

However, the Genetic Algorithm applies Crossover that Causes Multiple Changes in the Chromosome. It cannot be represented by the Markov Chain model. A p ybetter way to apply very rigorous Probability Modeling Technique that will simultaneously optimize parameters such as Crossover rate, Mutation rate, Inversion rate, Population Size, etc.

OPEN PROBLEM #2:OPEN PROBLEM #2: Distributed Genetic Algorithm will require a more complex Mathematical Modeling to compute the Epoch rate, Search cohesion, Speedup, etc.

OPEN PROBLEM #3:OPEN PROBLEM #3: Are GA’s suited for Constrained Combinatorial Optimization like in VLSI layouts? How to devise clever CrossoverOperators for such cases? Are there advantages of MultidimensionalCrossover operators in Multivariate Optimizations?

Outline of the Talk

• Evolutionary Approach to Distributed VLSI Layout Synthesis

• Self-Healing of VLSI Chips by NeurallyInspired Hardware MethodsInspired Hardware Methods

• Neuromorphic Nanoarchitecture using Cellular Nonlinear Network Model

• Reinforcement Learning Hardware

Biologically Inspired Computing Biologically Inspired Computing

Neural Networks(Connectionism)

Evolutionary Computation

(Survival of the fittest)

…Genetic Genetic

Social Systems Memb-

rane

ArtificialI

WetwareComputing

DNA

Molecular

MODELSMODELS TECHNOLOGIESTECHNOLOGIES

MultilayerPerceptrons

HopfieldNetworks

CellularNeural Networks

NanoarchitecturesSelf-assembled Quantum Dot Arrayfor image processing,video motion detection,& spatial & temporal filtering (Gabor, VTF)

Self-healing,Self-repairableVLSI Chips

Genetic Programming

Genetic Algorithms

Grammatical Evolution

• VLSI Cell Placement• Chip Wire Routing• Chip Floorplanning• Chip Testing• Logic Synthesis

Swarm Ant

Immune Systems

My research employs BIC in Micro & Nano Systems Design

Reinforcement Learning Chipswith Action-CriticModel to performAdaptive DynamicProgramming

Manufacturing Yield Reduces DramaticallyAs Chips BecomeLarger. Self Healing Is Needed to ImproveYield and Reduce ChipManufacturing Cost.

Year Transistors

4004 1971 2.25 K

8080 1974 5 K

8086 1978 29 K

286 1982 120 K

Intel386™ 1985 275 K

Intel486™ 1989 1.1 M

Pentium® 1993 3.1 M

Itanium® 2002 220 M

Itanium® 2 2003 410 M

Objective: Show the concept of self-healing by restructuring a megabit Memory array of RAM

Page 5: Outline of the Talk - web.eecs.umich.eduweb.eecs.umich.edu/~mazum/BIC-Mazumder.pdf · Biologically Inspired Algorithms for Micro and Nano System Design Prof. Pinaki Mazumder, Univ

3/20/2013

5

L2Data

L2Data

L2 Tag

Router Mem1Mem0

N

S

E

W

I/O

Spare Rows &ColumnsAreNeededFor ReplacingBad cells.

Given an NxN memory array which can haveGiven an NxN memory array which can havean arbitrary defect pattern, can we repair thean arbitrary defect pattern, can we repair thememory by using at most p rows and q columns? memory by using at most p rows and q columns? NPNP--Complete Problem (Restricted Node Cover)Complete Problem (Restricted Node Cover)

Data Data

EV68 CoreP0123

P4567

Faulty Row

Faulty Column

Faulty Cell

1024 x 1024 = 1 Mb Memory Array1024 x 1024 = 1 Mb Memory Arraywith 4 spare rows and 4 columnswith 4 spare rows and 4 columns

Defect PatternDefect Pattern

Spare Rows

Spare ColumnsSpare Columns

CompactedCompactedDefect PatternDefect Pattern

Lyapunov EnergyLyapunov EnergyFunctionFunction

•• THE ENDTHE END

Page 6: Outline of the Talk - web.eecs.umich.eduweb.eecs.umich.edu/~mazum/BIC-Mazumder.pdf · Biologically Inspired Algorithms for Micro and Nano System Design Prof. Pinaki Mazumder, Univ

3/20/2013

6

COMPARISON BETWN. NUROMORPHIC COMPARISON BETWN. NUROMORPHIC v. SOFTWARE TECHNIQUESv. SOFTWARE TECHNIQUES

Only 10,000 transistors are needed to healOnly 10,000 transistors are needed to heal1 GB of RAM array1 GB of RAM array

Manufacturing Yield

Faulty Column

Faulty Row

100%

20%

With BISR

WithoutBISR

Manufacturing Yield

1995

Neural-inspired Maximum Matching Problem on a Bipartite Graph.

Outline of the Talk

• Evolutionary Approach to Distributed VLSI Layout Synthesis

• Self-Healing of VLSI Chips by NeurallyInspired Hardware MethodsInspired Hardware Methods

• Neuromorphic Nanoarchitecture using Cellular Nonlinear Network Model

• Reinforcement Learning Hardware

Biologically Inspired Computing Biologically Inspired Computing

Neural Networks(Connectionism)

Evolutionary Computation

(Survival of the fittest)

…Genetic Genetic

Social Systems Memb-

rane

ArtificialI

WetwareComputing

DNA

Molecular

MODELSMODELS TECHNOLOGIESTECHNOLOGIES

MultilayerPerceptrons

HopfieldNetworks

CellularNeural Networks

NanoarchitecturesSelf-assembled Quantum Dot Arrayfor image processing,video motion detection,& spatial & temporal filtering (Gabor, VTF)

Self-healing,Self-repairableVLSI Chips

Genetic Programming

Genetic Algorithms

Grammatical Evolution

• VLSI Cell Placement• Chip Wire Routing• Chip Floorplanning• Chip Testing• Logic Synthesis

Swarm Ant

Immune Systems

My research employs BIC in Micro & Nano Systems Design

NANONANO BICBIC

QISQIS

Page 7: Outline of the Talk - web.eecs.umich.eduweb.eecs.umich.edu/~mazum/BIC-Mazumder.pdf · Biologically Inspired Algorithms for Micro and Nano System Design Prof. Pinaki Mazumder, Univ

3/20/2013

7

CNN Cell StructureCNN Cell Structure

dtB

A

f (.)

ijdx

dtijx

ijy

y

z

iju

u

Input U State X Output Y

B

A

f(x)= yij

S1(i,j)

C(i,j)

Sphere of Activity Sphere of Activity

IBUAXtXhdt

tdXC

jiNlkCklklij

ij

r

),(),(

)())(()(A: Feedback TemplateA: Feedback Template

B: FeedB: Feed--forward Templateforward Template

VLSI Implementation – CMOS CNN Nucleus

CNNCNNnucleus

LCCULCCULocal communication

and control unit

LAMLAMLocal

analogmemory

LLMLLMLocallogic

memory

LAOULAOULocal analog

output unit

LLULLULocal logic unit

CNNCNNnucleus

LCCULCCULocal communication

and control unit

LAMLAMLocal

analogmemory

LLMLLMLocallogic

memory

LAOULAOULocal analog

output unit

LLULLULocal logic unit

VP+

Vdd

Mn2

Mp2

Mp1Mp15

Mn5

Mp5

Mp4

Mp3

Mn3 Mn6

Mp6

Mn8

Mp8

Mp7

Mn9

Mp9

Mn11

Mp11

Mp10Mp14

Mp12

Mn12

Vpa

Mn11

Mp11

Mp10Mp14

Mp12

Mn12

VsVp1

Mp13

Mn14

Mn11

Mp11

Mp10Mp14

Mp12

Mn12

Vs

Mp13

Iout1 Ioutn

1st feedback weight output stage (A Template)

Nth feedback weight output stage

Vss

Mn15Mn1

Cx

Mn4 Mn7 Mn14 Mn10 Mn14 Mn10

Mp13

Mn13

Mn10

Vp1

Mn13

Current InverterIntegrator

Source: L. Ravezzi, G.F. Dalla Betta, G. SettiCellular Neural Networks and Their Applications, 1998. (CNNA 1998). Proceedings of the 1998 5th IEEE International Workshop on , April 1998 Pages:253- 258

VLSI Implementation VLSI Implementation –– QMOS QMOS CNN Nucleus for HCCDCNN Nucleus for HCCD

Tocell(i,j+1) as its yw

yw

Mw

ye

Mc

clka clkb

Toll(i j 1)

Mcc

CNNCNNnucleus

LCCULCCULocal communication

and control unit

LAMLAMLocal

analogmemory

LLMLLMLocallogic

memory

LAOULAOULocal analog

output unit

LLULLULocal logic unit

CNNCNNnucleus

LCCULCCULocal communication

and control unit

LAMLAMLocal

analogmemory

LLMLLMLocallogic

memory

LAOULAOULocal analog

output unit

LLULLULocal logic unitQuantum Tunneling Device

Me

cell(i,j-1) as its ye

Initial Condition

Dark : 1White : 0

4 transistors & 4 RTDs for one cell

CNN-based Image ProcessingEdge Extraction-128x128

0.5 n 1 nGray scale Input

pFC 1

1.5 n 2 n

mAI

mABmAA

1

010

141

010

000

000

000

The structure of the synaptic The structure of the synaptic Network between the base Network between the base cell and neighboring cells.cell and neighboring cells.

Roadmap

Year Transistors

4004 1971 2.25 K

8080 1974 5 K

Generation #1 Generation #5

Exponential Economic Growth

The Law of Accelerating ReturnsBy Ray Kurzweil

#2 #3 #4

MolecularMolecularSwitchSwitch

QuantumQuantumComputingComputing

QD’s

8080 1974 5 K

8086 1978 29 K

286 1982 120 K

Intel386™ 1985 275 K

Intel486™ 1989 1.1 M

Pentium® 1993 3.1 M

Pentium® II 1997 7.5 M

Pentium® 4 2000 42 M

Itanium® 2002 220 M

Itanium® 2 2003 410 M

ECONOMIC GROWTH HINGES ONECONOMIC GROWTH HINGES ONBOOSTING OF EMT RESEARCHBOOSTING OF EMT RESEARCH

The Trajectory ofExponential Growth

Technology Grows Exponentially

* Ultimate Device Size: 10-40 Å* Three-Dimensional Integration* Exponential Processing Power* Molecular & Quantum Devices

AI-basedNanobot

Technology Does NotChange Linearly

100 Billion Fold Growth Between 1900-2000 WillOccur in the Twenty First Century in 20 Years!!

Molecular DiodeUniv. of Chicago

Page 8: Outline of the Talk - web.eecs.umich.eduweb.eecs.umich.edu/~mazum/BIC-Mazumder.pdf · Biologically Inspired Algorithms for Micro and Nano System Design Prof. Pinaki Mazumder, Univ

3/20/2013

8

Nanoarchitectures with 0-D RTD & Self-Assembled Quantum Dots

Quantum DotsHaving 5 nm diameter

Scalability

Self-assembled metal/molecule/metal nanostructures for nanoscale devices

PyramidalQuantum Dots usingGaAs

Quantum Dot ArrayFig. 2 Schematic view of the pyramid-shape In0.5Ga0.5As quantum dot.

2040

6080

100

20

40

60

80100

-0.2

-0.1

0.0

0.1

0.2

Y Axi

s

X Axis

22--D Wave Function With D Wave Function With 21 meV Eigen Energy 21 meV Eigen Energy

Quantum Dot ArrayQuantum Dot Array

2 2 2

2 22 *( , | )

( , , ) ( , | ) ( , | )LC

x y zm x y

V x y z x y z E x y z

)()|,(),,( zzyxzyx

22

2

0 1 0 12 0( , ) ( , ), ' ' , ' '

'

( ) ( ) ( )

( ) ( ) ( ) ( )

dz k z z

dz

dC z z C z z

dz

Multiscale Multiscale Simulation:Simulation:Quantum DeviceQuantum Device

Multiscale Multiscale Simulation:Simulation:Quantum DeviceQuantum Device

dEEfEfETe

IeV RLtunnel

)()(

0.0 0.5 1.0 1.5 2.00.000

0.002

0.004

0.006

0.008

0.010

0.012

Cu

rren

t (

A)

Applied Voltage (V)

Quantum Device Quantum Device Modeling &Modeling &Quantum SpiceQuantum Spice

Quantum Device Quantum Device Modeling &Modeling &Quantum SpiceQuantum Spice

Negative Differential Resistance of C60 & BPDN Molecules Negative Differential Resistance of C60 & BPDN Molecules

0 1tot i i i ii

H V c c hc c

Tight-binding Hamiltonian model:

2

, ,mols l l sI V V

bipyridyl-dinitro oligophenylene-ethylene dithiol (BPDN)

Courtesy: I. I. Oleynik & M. A. Kozhushner

Quantum Dot Array

0.10.2

0.3

0.4

0.50.1

0.2

0.3

0.40.5

0.0

0.5

1.0

V(f

m,f

n)/

I BR

f mfn

nm ff

RInzmzV

21221212

coscos

),( ZZ--transform of a QD pairtransform of a QD pair ==

Connecting Resistance Defines the Connecting Resistance Defines the Mutual Voltage between a pair of QD’sMutual Voltage between a pair of QD’s

Nanoscale Nonlinear Circuit TheoryNanoscale Nonlinear Circuit Theory

Edge Detection by QDA

0 ns 2 ns1 ns

Gray Gray BinaryBinaryConversionConversion

V l L D (VLD)

5 ns 6 ns

Vertical Line Detection (VLD)

6 ns0 ns

Image Processor is being fabricated and tested under a NIRT projectImage Processor is being fabricated and tested under a NIRT project

Page 9: Outline of the Talk - web.eecs.umich.eduweb.eecs.umich.edu/~mazum/BIC-Mazumder.pdf · Biologically Inspired Algorithms for Micro and Nano System Design Prof. Pinaki Mazumder, Univ

3/20/2013

9

15 pA

Nanoarchitucture for Motion EstimationNanoarchitucture for Motion Estimation

Quantum Dot

Quantum Dot basedCellular NonlinearNetwork with Motion Estimation Capability

Quantum Dotsare underneaththe top metalparticles

.;/;/;/;/

.0 and ;0

])()[(2)(2)(21

12222

rCRrbRraRrbRra

Cbaba

ffbafbajfbafbaH

NNSSWWEE

NSWE

tyyyxxxyNSxWE

HSPICE Result of QD VTF for v = 0

To get filtered output, S value should be above 1.45 for high value =0.75 (RTD mid voltage)+0.7(diode threshold voltage)

V = +2 pixel/sec

Input Images Filtered Output

V = +1 pixel/sec

V = 0 pixel/sec

V = -1 pixel/sec

V = -2 pixel/sec

3-D Self-Assembled Architectures

Raenssler Polytechnique Inst Los Alamos National Laboratory

Applications of 3-D Self-Assembled Architectures (Random Boolean Networks)

• Genetic Regulatory Networks (Kauffman 1993,Gershenson 2005)

Understanding of disease treatmentGenomic interaction and data miningGenomic interaction and data mining

• Evolutionary Computing & Evolvable Hardware (JPL)• Artificial Neural Networks (Huepe & Aldana, 2002)• Social Modeling (Shelling 1971)• Robotics (Quick, et al. 2003) • Cellular Automata (Wuensche and Lesser, 1992)• Percolation Theory (Stauffer, 1985)• Biologically Inspired Computing (Swarm, Ant forage, …)

Outline of the Talk

• Evolutionary Approach to Distributed VLSI Layout Synthesis

• Self-Healing of VLSI Chips by NeurallyInspired Hardware MethodsInspired Hardware Methods

• Neuromorphic Nanoarchitecture using Cellular Nonlinear Network Model

• Reinforcement Learning Hardware

Page 10: Outline of the Talk - web.eecs.umich.eduweb.eecs.umich.edu/~mazum/BIC-Mazumder.pdf · Biologically Inspired Algorithms for Micro and Nano System Design Prof. Pinaki Mazumder, Univ

3/20/2013

10

Biologically Inspired Computing Biologically Inspired Computing

Neural Networks(Connectionism)

Evolutionary Computation

(Survival of the fittest)

…Genetic Genetic

Social Systems Memb-

rane

ArtificialI

WetwareComputing

DNA

Molecular

MODELSMODELS TECHNOLOGIESTECHNOLOGIES

MultilayerPerceptrons

HopfieldNetworks

CellularNeural Networks

NanoarchitecturesSelf-assembled Quantum Dot Arrayfor image processing,video motion detection,& spatial & temporal filtering (Gabor, VTF)

Self-healing,Self-repairableVLSI Chips

Genetic Programming

Genetic Algorithms

Grammatical Evolution

• VLSI Cell Placement• Chip Wire Routing• Chip Floorplanning• Chip Testing• Logic Synthesis

Swarm Ant

Immune Systems

My research employs BIC in Micro & Nano Systems Design

Reinforcement Learning Chipswith Action-CriticModel to performAdaptive DynamicProgramming

The Holy Grail of Computing The Holy Grail of Computing ----Can we spill human brain on Silicon?Can we spill human brain on Silicon?

Current CapabilityCurrent Capability

The ExponentialThe ExponentialGrowth DrivesGrowth Drivesthe Societalthe SocietalEconomicsEconomics

1 GW digital1 GW digital

Facets of Facets of NeuromorphicNeuromorphic or Brainor Brain--like Computinglike Computing::

SelfSelf--Healing (Robust)Healing (Robust) Spike LearningSpike LearningCognition (Visual, Cognition (Visual,

Auditory, Tactile) Huge MemoryAuditory, Tactile) Huge Memory

1 GW digital 1 GW digital computercomputer

< 50 W Brain< 50 W Brain

Excitatory synapse

Dendritic TreeEVOLUTION OFEVOLUTION OFNEUROMORPHICNEUROMORPHICCOMPUTING:COMPUTING:

Perceptron (’60)Perceptron (’60)(10 E 1 neurons)(10 E 1 neurons)

Neural Net (’80)Neural Net (’80)(10 E 3 Neurons)(10 E 3 Neurons)

NeuromporphicNeuromporphicHardware (’90)Hardware (’90)(10 E 6 )(10 E 6 )

Axon Hillock

Figure from Principles of Neural Science [2] p.22

Inhibitory Synapse

(10 E 6 neurons)(10 E 6 neurons)

Nanocrossbar Nanocrossbar (10 E 10 neurons)(10 E 10 neurons)

Biological Neuron ModelIonic Transport in Biological Neuron & its Silicon Implementation

HodgkinHodgkin--Huxley ModelHuxley Model

•Uniform motion of the conducting front – analog switching (memristor) •Creation of uniform conducting front by co-sputtering of a-Si & metal

Analog a-Si Memristors

Top Electrodesputtered a-Si onlyco-sputtered Si & Ag (~ 20nm)mixture ratio (rough # of atoms) gradual chang

DARPA SyNAPSE PROJECTDARPA SyNAPSE PROJECTWITH HRL LABORATORIESWITH HRL LABORATORIES

Pinaki MazumderU of Michigan – NDR Group

-2 -1 0 1 2 3-0.4

-0.2

0.0

0.2

0.4

0.6

0.8

1.0

V lt (V)

Successive Successive writingwriting

Successive erase pSuccessive erase processesrocesses

Voltage (V)

I (A

)

•Incremental conductance change•Conductance total charge through the device

Bottom Electrode

sputtered a-Si only

mixture ratio (rough # of atoms), gradual changei.e. Si : Ag = 20: 1 (bottom) 10: 1 (top)

•Highest process temperature < 260°C

All Credits Go to Prof. All Credits Go to Prof. Wei Lu’s OutstandingWei Lu’s OutstandingResearch GroupResearch Group

Memristor: A New Paradigm Circuit Design

Conductance controlled by the pulse width, and can b h n d

Programming pulses with different pulse widths

be changed incrementally.

Conductance change after each pulse

100 s

1.1. Digitally ControlledDigitally Controlled

2.2. Constant AmplitudeConstant Amplitude

3.3. Temporal CorrelationTemporal Correlation

Page 11: Outline of the Talk - web.eecs.umich.eduweb.eecs.umich.edu/~mazum/BIC-Mazumder.pdf · Biologically Inspired Algorithms for Micro and Nano System Design Prof. Pinaki Mazumder, Univ

3/20/2013

11

• Time Division Multiplexing– Events occur at proper

timeslots– Long Term Potentiation can

only occur in the 2nd timeslot– Long Term Depression can only

occur in the 1st timeslotI id d l i

Neuron Spiking Signals and STDP ControlNeuron Spiking Signals and STDP Control

|| if )().(

exp.exp..

postpreLTPijpostpre

b

postb

a

prea

LTPLTPij

ttwttuttu

ttd

ttdw

dt

d

HebbianHebbian--Type STDP Learning ModelType STDP Learning ModelNeurons that “FIRE” together.Neurons that “FIRE” together.also “WIRE” together. also “WIRE” together. –– D. HebbD. Hebb

– Inputs are considered only in the 0th timeslot

• Each pulse of amplitude V is not enough to make a significant change to memristance

• When there’s a net difference with amplitude 2V, memristance changes

Neuron spikes in first frame61

Pulse Width vs. Pre and Post Neuron Spike Times

SPDT with Memristor

LTPLTP

ttPRE PRE –– ttPOST > 0POST > 0ttPRE PRE –– ttPOST < 0POST < 0

Neuron Pulse Width Curves (across a synapse)For LTP case: The Post neuron is at –V while the Pre neuron at +V

For LTD case: The Pre neuron is at –V while the Post neuron at +V

LTP case is taken as positive while the LTD case negative

62

STDP obtained using memristor STDP obtained using memristor synapsesynapse

LTDLTD

Pulse Width vs tPulse Width vs tPREPRE -- ttPOSTPOST

STDP-Based Position Detector

+ +

Noise‐free Noisy

‐ ‐

‐‐‐

+ ++ ++

+++

n21

‐v

m11

m25m22

m26

m24m23

m21

m15m14m13m12

n11

n12

n13

n14

n15

n22

n23

n24

n25

University of MichiganVERILOG SIMULATIONVERILOG SIMULATION

Period:1746

Period:1746

Period:786

Period:786

Period:636

Position Detector‐ Noise‐free Noisy

‐ ‐

‐‐‐

+ ++ ++

+++

n21

‐v

m11

m25m22

m26

m24m23

m21

m15m14m13m12

n11

n12

n13

n14

n15

n22

n23

n24

n25

+ +

University of MichiganVERILOG SIMULATIONVERILOG SIMULATION

Period:2046

Period:642Period:684

Period:3030 Period:7242

STDP Neural Circuit for Position DetectorSTDP Neural Circuit for Position Detector

Bases Memristor Design

CMOS DesignDesign Design

Synaptic area < (0.5μm x 0.5μm)

17μm x 16μm

Synaptic Density 4 devices/μm2

x10000.0037

devices/μm2

Neuron area 20μm x 10μm 8μm x 12μm

Neuron Density 0.005 devices/μm2

x2

0.0104 devices/μm2

Volatility Nonvolatile Volatile

STDP Based Position Detector‐

Noise‐free Noisy‐ ‐

‐‐‐

+ ++ ++

+++

n21

‐v

m11

m25m22

m26

m24m23

m21

m15m14m13m12

n11

n12

n13

n14

n15

n22

n23

n24

n25

+ +

University of MichiganVERILOG SIMULATIONVERILOG SIMULATION

Period:1014

Period:7266

Period:660 Period:660

Period:1506

Page 12: Outline of the Talk - web.eecs.umich.eduweb.eecs.umich.edu/~mazum/BIC-Mazumder.pdf · Biologically Inspired Algorithms for Micro and Nano System Design Prof. Pinaki Mazumder, Univ

3/20/2013

12

Biologically Inspired Computing Biologically Inspired Computing

Neural Networks(Connectionism)

Evolutionary Computation

(Survival of the fittest)

…Genetic Genetic

Social Systems Memb-

rane

ArtificialI

WetwareComputing

DNA

Molecular

MODELSMODELS TECHNOLOGIESTECHNOLOGIES

MultilayerPerceptrons

HopfieldNetworks

CellularNeural Networks

NanoarchitecturesSelf-assembled Quantum Dot Arrayfor image processing,video motion detection,& spatial & temporal filtering (Gabor, VTF)

Self-healing,Self-repairableVLSI Chips

Genetic Programming

Genetic Algorithms

Grammatical Evolution

• VLSI Cell Placement• Chip Wire Routing• Chip Floorplanning• Chip Testing• Logic Synthesis

Swarm Ant

Immune Systems

My research employs BIC in Micro & Nano Systems Design

Reinforcement Learning Chipswith Action-CriticModel to performAdaptive DynamicProgramming

• Evolutionary Computing plays a critical role inLayout Synthesis and Testing of VLSI Chips.

Neural Inspired Self-Healing plays a criticalrole in improving manufacturing yield andSurvivability of chips.

Cellular Nonlinear Networks provide architectures forNanoelectronics.

E EE ETHE ENDTHE END

My research group is also working on THz Sensing of DNA, RNA and Other Biomolecules

Learning-based VLSI chipswill require major innovations in nanoelectronics from materials to architectures