Neuromorphic Computing - Helen Wills Neuroscience...

Neuromorphic Computing in the

European Human Brain Project

Karlheinz Meier @brainscales

Ruprecht-Karls-Universität Heidelberg

NICE 2016, Berkeley

1.  Neuroinforma-csPla3ormAggregateneurosciencedata,deliverbrainatlases

2.  MedicalInforma-csPla3ormAggregateclinicalrecords,classifybraindiseases

3.  BrainSimula-onPla3ormDevelopsoDwaretools,runclosedloopbrainsimula-ons

4.  HighPerformanceCompu-ngPla3orm

DevelopandoperateHPCsystemsop-mizedforbrainsimula-ons

5.  NeuromorphicCompu-ngPla3ormDevelopandoperatenovelbrainderivedcompu-nghardware

6.  Neurorobo-csPla3ormDevelopvirtualrobo-csystemsforclosedloopcogni-veexperiments

Publiclauncheventatendoframp-upphase–March,30th

The6ICTPla+ormsinHBP

Temporal Scales and Strong Scaling

Computa7onalComplexity

Memory

Requirement

1MB

10GB

1TB

100TB

100PB

CellularNeocor-calColumn

CellularMesocircuit

CellularRodentBrain

CellularHumanBrain

1Gigaflops 1Teraflops 1Petaflops 1Exaflops

SingleCellularModel

Subcellulardetailandplas-cityrequireadvancesinstrongscaling!

Glia-Cell/VasculatureO(1-10x)

Reac-on-DiffusionO(100-1,000x)

MolecularDynamicsO(>1,000,000,000x)

Plas-cityO(1-10x)

LearningO(10-100x)

DevelopmentO(100-1000x)

TheONLYwaytoevermakeuseof

ar-ficialneuralcircuitsderivedfrom

biologyistomakethemadap$ve

Connec-vity–Synapses–Neurons

byclosed-loopinterac-onwithdata

Click to edit Master title style

• Click to edit Master text styles

–  Secondlevel•  Third level

–  Fourth level

Previous work is essential

BrainScaleS

SpiNNaker

FACETS/BrainScaleS 2005-2015 SpiNNaker 2005-2015

8-10 years from chip design to system !

Requires roadmap and sustained funding

- Not developed

in HBP -

• 18ARM968Coresperchip• IntegerArithme-c

• 200MHzProcessorClock

• SharedsystemRAMondie

•  128MbyteDRAMstackedondie

•  EachChip6bi-direc-onallinks•  6millionspikes/s/link

•  RealTimeSimulator

SpiNNaker

Group(+HBP)

HBP SpiNNaker Machine Generations (Manchester Site)

103 104 105

Ra-onalesfortheBrainScaleSPhysicalModelSystem

Ø  Mixed-Signal(Localanalogcomputa-on,binaryspikecommunica-on)

Ø  Drivenbyarchitecture,notdevices(180nmCMOS)

Ø  HighNeuronInputCount(>10.000)

Ø  Configurability(cellparameters,connec-ons)->Universality

Ø  Scalability:ChipScale(105)->WaferScale(108)->Systems(>109)

Ø  Accelera-onx10.000,consistent-meconstants(1daycompressedto10seconds)

Ø  Short-termundlong-termPlas-city

Ø  Upgradabilitywithunchangedsystemarchitecture

Ø  HybridOpera-on,closedloopexperiments

Ø  Non-ExpertUserAccess

Objec-ve:Exploitconfigurabilityandaccelera-on

-rapidexplora-onoflargeparameterspaces

-covershortandlong-mescalecircuitdynamics

-performcompu-nginthepresenceofspa-alandtemporalnoise

HiCANNHigh

InputCount

Analogue

NeuralNetwork

Chip

Millner,S.,Grübl,A.,Meier,K.,Schemmel,J.andSchwartz,M.-O.,AVLSIImplementa-onoftheAdap-veExponen-alIntegrate-and-FireNeuronModel

AdvancesinNeuralInforma-onProcessingSystems(NIPS)(2010)

Physical Model, local analogue computing,

binary continuous time communication

Wafer-Scale Integration of 200.000 neurons and 50.000.000 synapses on

a single 20 cm wafer

Short term and long term plasticity, 10.000 faster

than real-time

Wafer-scaleintegra$onofanalogneuralnetworks,J.Schemmel,J,FieresandK.Meier

In:ProceedingsofIJCNN(2008),IEEEPress,431

Click to edit Master title style

• Click to edit Master text styles

–  Secondlevel•  Third level

–  Fourth level

HBP : Neuromorphic Computing Platform

THEPHYSICALMODELSYSTEM

Localanaloguecompu-ngwith4Millionneurons

and50Millionsynapses–binary,asynchronous

communica-on–runningatx10000real--me

Loca-on:Heidelberg(Germany)

Offering : Access to a unique set of 2 complementary, highly configurable neuromorphic machines for modelling neural microcircuits and applying brain-like principles in machine learning and cognitive computing

THEMANY-COREDIGITALPROCESSORSYSTEM

0.5–1MillionARMprocessors–address-based,smallpacket,

asynchronouscommunica-on–runningatreal--me

Loca-on:Manchester(UK)

1

5

67

23

4

8

8

500.000core

machine

Loca-on:

Manchester(UK)SeetalkbySteveFurber

20Wafermodule

machine

Loca-on:

Heidelberg(GE)

SeetalkbyJohannesSchemmel

Click to edit Master title style

• Click to edit Master text styles

–  Secondlevel•  Third level

–  Fourth level

Remote Access ready for users

Slide16

Heidelberg

PyNN

Job queue server

Model description

Experiment description

Data storage

Mapping

Calibration data HAL

Authentication Validation Notification

Manchester

PyNN

PACMAN

SpinnMan

See demo by Eric Mueller

HBPNeuromorphicCompu-ngGuidebook–Con-nuousUpdates

Comprehensiveopenaccessdocumenta-on:

Hardware,systems,firmware,low/highlevelsoDware

Benchmarks(neuroscience,machinelearning)

Tutorials,smallsystemsdescrip-on

hnp://electronicvisions.github.io/hbp-sp9-guidebook/

NeuromorphicLaptopAdd-on

PlugsintoUSB,fullsoDware

supportanddocumenta-on

498neurons

100.000plas-csynapses

100.000fasterthanreal--me

spikey@kip.uni-heidelberg.de

SeedemobyEricMueller

Increasingnumberofusecasesandapplica-ons

coveringawidespectrumofnetworktypes

Exploi-ngSubstrateUNIVERSALITY–selec-onofpublishedwork:

-  Canonicalcircuits(synfirechains,WTA,aOractorcircuits)-  Balancedrandomnetworks-  Liquidcompu$ng,temporalpaOerniden$fica$on-  MinicolumnLayer2/3circuits-  Closed-loophybridcontrolsystems-  Mul$variatedataclassifica$on-  Phasedetec$on,applyingSTDP-  Decorrela$onthroughinhibitoryfeedback-  Stochas$cinferencethroughneuralsampling-  BayesiannetworksasBoltzmannmachinesofLIFneurons-  Impleme$ngdeeplearningwithspikingneurons-  Implemen$ngHTMwithspikingneurons

SeetalkbyMihaiPetroviciandposterbyLuziweiLeng

2023 Roadmap details in FPA document

Click to edit Master title style

•  Click to edit Master text styles

–  Secondlevel•  Third level

–  Fourth level

From SpiNNaker to SpiNNaker2

Feature SpiNNaker SpiNNaker2

technology 130nm 28nm

cores 18 68

core frequency 200MHz >400MHz

external memory 128MByte (1 Gbyte/s) 2GByte (>10 Gbyte/s)

power 1W 1W

power management no yes

floating point support no yes

vector processing no yes

true random numbers no yes

biological realtime operation yes yes

no. of neurons / chip 16k 128k

no. of synapses / chip 16M 128M

energy/synaptic event 10-8J 10-9J

≈10ximprovementatconstantpower

PhysicalModel:TargetsinHBPfor2023

PrototypesintheLab

StructuredNeurons

Ac-vedendriteswithspa-al

structure:Neuronsascomplex

panerndetectors(e.g.

hierarchicaltemporalmemory

Plas-cityProcessor

400PowerPCprocessorsper

wafer:Re-wiringonthefly,data

drivenaccelerateddevelopment,

slowandfastcircuitsdynamics

SeedemobyEricMueller

Observables Controls Synapseevalua-on

Popula-onrates

Arbitraryinternal

parameters

Weights

Connec-vity

Rewiring

Neuronparameters

Homeostasis

S-mulusgenerators

Externalrewards

andcontrols

Essen-al:Any-mescale>100µs(bio)isaccessible

Wafer-PCBLamina-onforlargescale

highdensitymanufacturing

neuromorphic.eu