41
A Software Stack for Neuromorphic Computing James S. Plank Mark E. Dean Garrett S. Rose Catherine D. Schuman July 19, 2017 Neuromorphic Computing Symposium Knoxville, Tennessee

A Software Stack for Neuromorphic Computing · 7/19/2017  · Complexity Report Fitness Get Random network Do Crossover ... Managing a software team in academia takes ... Software

  • Upload
    others

  • View
    0

  • Download
    0

Embed Size (px)

Citation preview

Page 1: A Software Stack for Neuromorphic Computing · 7/19/2017  · Complexity Report Fitness Get Random network Do Crossover ... Managing a software team in academia takes ... Software

A Software Stack for Neuromorphic Computing

James S. PlankMark E. Dean

Garrett S. RoseCatherine D. Schuman

July 19, 2017Neuromorphic Computing Symposium

Knoxville, Tennessee

Page 2: A Software Stack for Neuromorphic Computing · 7/19/2017  · Complexity Report Fitness Get Random network Do Crossover ... Managing a software team in academia takes ... Software

What our group looked like in 5/2015

Katie and Doug developed NIDA:

- Simulator - Custom applications - Custom EO - Custom visualization (Meg)

Garrett developed mrDANNA:

- Memristor modeling - Simulator in SPICE (Gangotree) - Hand-tooled networks

Mark developed DANNA:

- FPGA Implementation (Chris) - Hand-tooled networks - Communications board (Jason)

Page 3: A Software Stack for Neuromorphic Computing · 7/19/2017  · Complexity Report Fitness Get Random network Do Crossover ... Managing a software team in academia takes ... Software

What it looks like now

SoftwareCore

Applications Learning

Architectures

Page 4: A Software Stack for Neuromorphic Computing · 7/19/2017  · Complexity Report Fitness Get Random network Do Crossover ... Managing a software team in academia takes ... Software

You've seen it in three of our talks here

DANNA

EO

SoftwareCore

Applications Learning

Architectures

Page 5: A Software Stack for Neuromorphic Computing · 7/19/2017  · Complexity Report Fitness Get Random network Do Crossover ... Managing a software team in academia takes ... Software

You've seen it in three of our talks here

SoftwareCore

NIDA

mrDANNA

EO

Applications Learning

Architectures

Page 6: A Software Stack for Neuromorphic Computing · 7/19/2017  · Complexity Report Fitness Get Random network Do Crossover ... Managing a software team in academia takes ... Software

You've seen it in three of our talks here

mrDANNA

XOREO + STDP

SoftwareCore

Applications Learning

Architectures

Page 7: A Software Stack for Neuromorphic Computing · 7/19/2017  · Complexity Report Fitness Get Random network Do Crossover ... Managing a software team in academia takes ... Software

What is in this talk

● What an architecture means in our software stack.● The structure of an application in this stack.● How to put “learning” into its appropriate place.● Some lessons learned with respect to software

and a project of the scope of this one.

Page 8: A Software Stack for Neuromorphic Computing · 7/19/2017  · Complexity Report Fitness Get Random network Do Crossover ... Managing a software team in academia takes ... Software

What an architecture means

The architecture encompasses the computing model, constraints and connectivity.

● 3D● Analog● Synapses defned by

Euclidean distance.

NIDA

● 2D● Digital● Synapses programmable

but constrained.

DANNA

● 2D● Mixed Analog/Digital● Synapses programmable

but constrained.

mrDANNA

Page 9: A Software Stack for Neuromorphic Computing · 7/19/2017  · Complexity Report Fitness Get Random network Do Crossover ... Managing a software team in academia takes ... Software

What an architecture means

Within the software stack, the architecture must defne a network and a device.

● Serialize / Deserialize● Defne inputs & outputs● Primitives for learning

(more on this later)

Network ≈ “Program” Device ≈ “Processor”

● Load / Pull Network● Apply input charge events● Read output charge events● Run● Capture State

Page 10: A Software Stack for Neuromorphic Computing · 7/19/2017  · Complexity Report Fitness Get Random network Do Crossover ... Managing a software team in academia takes ... Software

Within the core, an instance drives execution.

● Start job● Execute● Stop job

Instance

Network Device

● Why do we need this?

Page 11: A Software Stack for Neuromorphic Computing · 7/19/2017  · Complexity Report Fitness Get Random network Do Crossover ... Managing a software team in academia takes ... Software

● Start job● Execute● Stop job

Instance

Network Device

Gives you a handle on an execution ● EO / GPU's / Advanced applications

Instance

Network Device

Instance

Network Device

Instance

Network Device

Instance

Network Device

Instance

Network Device

Instance

Network Device

Instance

Network Device

Within the core, an instance drives execution.

Page 12: A Software Stack for Neuromorphic Computing · 7/19/2017  · Complexity Report Fitness Get Random network Do Crossover ... Managing a software team in academia takes ... Software

● Start job

● Execute

● Stop job

Allows the core to implement architecture-independent functionality.

Instance

Network Device

Stall

Prompt

Capture

Within the core, an instance drives execution.

Page 13: A Software Stack for Neuromorphic Computing · 7/19/2017  · Complexity Report Fitness Get Random network Do Crossover ... Managing a software team in academia takes ... Software

Architectures end up with four components

Network Device

Execution Unit Visualization

Program Processor

Simulation,Hardware

Processes events & capturesStatic (screenshots)

Live

Page 14: A Software Stack for Neuromorphic Computing · 7/19/2017  · Complexity Report Fitness Get Random network Do Crossover ... Managing a software team in academia takes ... Software

For example, with DANNA

Execution Unit

Instance

Network Device

Simulator FPGA VLSI

Load packetsInput packets

Run packets Capture packetsOutput packets

Captures

Visualization

Screenshots Javascript

Page 15: A Software Stack for Neuromorphic Computing · 7/19/2017  · Complexity Report Fitness Get Random network Do Crossover ... Managing a software team in academia takes ... Software

Execution Unit

Instance

Network Device

Simulator FPGA VLSI

Load packetsInput packets

Run packets Capture packetsOutput packets Captures

Visualization

Screenshots Javascript

For example, with DANNA

Page 16: A Software Stack for Neuromorphic Computing · 7/19/2017  · Complexity Report Fitness Get Random network Do Crossover ... Managing a software team in academia takes ... Software

The structure of an application.

Our canonical application structure has 5 components:

ApplicationLibrary

Read-Only

Per Run

ApplicationDriver

NeuromorphicLibrary

Read-Only

Per Run

NeuromorphicDriver

LearningLibrary

Read-Only

Per Run

Libraries:

Programs:

Page 17: A Software Stack for Neuromorphic Computing · 7/19/2017  · Complexity Report Fitness Get Random network Do Crossover ... Managing a software team in academia takes ... Software

The structure of an application.

The application library implements the gutsof the application.

ApplicationLibrary

Read-Only

Per Run

GetApplicationState()UpdateApplicationState()

Page 18: A Software Stack for Neuromorphic Computing · 7/19/2017  · Complexity Report Fitness Get Random network Do Crossover ... Managing a software team in academia takes ... Software

The structure of an application.

The application library implements the gutsof the application.

ApplicationLibrary

Read-Only

Per Run

GetApplicationState()UpdateApplicationState()

Page 19: A Software Stack for Neuromorphic Computing · 7/19/2017  · Complexity Report Fitness Get Random network Do Crossover ... Managing a software team in academia takes ... Software

The structure of an application.

ApplicationLibrary

Read-Only

Per Run

ApplicationDriver

Application Program exists without anything neuromorphic.

Text on stdin or socket.

Text on stdout.

Text on socket.

Text on socket.

Page 20: A Software Stack for Neuromorphic Computing · 7/19/2017  · Complexity Report Fitness Get Random network Do Crossover ... Managing a software team in academia takes ... Software

The structure of an application.

The neuromorphic library implements instance → application and back

AppState_To_Inputs()Outputs_To_AppInput()

NeuromorphicLibrary

Read-Only

Per Run

Page 21: A Software Stack for Neuromorphic Computing · 7/19/2017  · Complexity Report Fitness Get Random network Do Crossover ... Managing a software team in academia takes ... Software

The structure of an application.

Neuromorphic Program interacts with application

over sockets, and“runs” an instance.

Text on socket.Neuromorphic

LibraryRead-Only

Per Run

NeuromorphicDriver

Text on socket.Instance

Network Device

Page 22: A Software Stack for Neuromorphic Computing · 7/19/2017  · Complexity Report Fitness Get Random network Do Crossover ... Managing a software team in academia takes ... Software

The structure of an application.Application program and neuromorphic program

compose very nicely for testing and demonstration.

NeuromorphicDriver

Instance

Network Device

ApplicationDriver

NeuromorphicLibrary

Read-Only

Per Run

ApplicationLibrary

Read-Only

Per Run

Page 23: A Software Stack for Neuromorphic Computing · 7/19/2017  · Complexity Report Fitness Get Random network Do Crossover ... Managing a software team in academia takes ... Software

The structure of an application.Application program and neuromorphic program

compose very nicely for testing and demonstration.

NeuromorphicLibrary

Read-Only

Per Run

NeuromorphicDriver

Instance

Network Device

ApplicationLibrary

Read-Only

Per Run

ApplicationDriver

Page 24: A Software Stack for Neuromorphic Computing · 7/19/2017  · Complexity Report Fitness Get Random network Do Crossover ... Managing a software team in academia takes ... Software

The structure of an application.

Learning library expresses application needs to the

learning layer, and defnes ftness: instance → value.

LearningLibrary

Read-Only

Per Run

Page 25: A Software Stack for Neuromorphic Computing · 7/19/2017  · Complexity Report Fitness Get Random network Do Crossover ... Managing a software team in academia takes ... Software

The structure of an application.

All of the libraries are compiled with a driver fromthe Learning Module to develop networks.

NeuromorphicLibrary

Read-Only

Per Run

LearningLibrary

Read-Only

Per Run

LearningLibrary

Read-Only

Per Run

Learning driverfrom the

Learning Module

InstanceNetwork DeviceInstanceNetwork DeviceInstanceNetwork DeviceInstanceNetwork DeviceInstance

Network DeviceInstanceNetwork DeviceInstanceNetwork Device

InstanceNetwork Device

InstanceNetwork DeviceInstanceNetwork DeviceInstanceNetwork DeviceInstanceNetwork DeviceInstanceNetwork DeviceInstanceNetwork DeviceInstance

Network DeviceInstance

Network Device

Page 26: A Software Stack for Neuromorphic Computing · 7/19/2017  · Complexity Report Fitness Get Random network Do Crossover ... Managing a software team in academia takes ... Software

Current Applications

● Control– Pole, Flappy, RoboNAV, Helicopter, FF-SA

● Classifcation– UCI Database (Iris, Cancer, etc.), Audio

● Security– Anomaly Detection (e.g. Numenta)

● Microapplications: Benchmarking & Composition– Binary Ops, Pulse Comparison

Page 27: A Software Stack for Neuromorphic Computing · 7/19/2017  · Complexity Report Fitness Get Random network Do Crossover ... Managing a software team in academia takes ... Software

RoboNAV on DANNA

Page 28: A Software Stack for Neuromorphic Computing · 7/19/2017  · Complexity Report Fitness Get Random network Do Crossover ... Managing a software team in academia takes ... Software

Pole Balancer on NIDA

Page 29: A Software Stack for Neuromorphic Computing · 7/19/2017  · Complexity Report Fitness Get Random network Do Crossover ... Managing a software team in academia takes ... Software

Application Support: Neuro-IO

● Map application state values to neuromorphic input spikes:– Rate-Coding, Binning, Charge Values

– And their combination.

● Ditto output spikes – Counting

– Voting

– Binning

xdxθ

dθleft / right

Device

Pole BalancingApplication

Neuro-IO

Page 30: A Software Stack for Neuromorphic Computing · 7/19/2017  · Complexity Report Fitness Get Random network Do Crossover ... Managing a software team in academia takes ... Software

Learning – Where does it go?

● Current learning techniques:– EO: Evolutionary Optimization– Unsupervised Learning (STDP)– Supervised Learning (Ditto)

Mature enoughto be a module

Still in“research” mode

Mature enoughto be a module

Page 31: A Software Stack for Neuromorphic Computing · 7/19/2017  · Complexity Report Fitness Get Random network Do Crossover ... Managing a software team in academia takes ... Software

Learning – Where does it go?

● The Current Learning Module– Manages epochs & populations

– Directs crossover & mutations, but doesn't do them.

– Manages parallelism, both within a machine and within a cluster (or Titan).

Page 32: A Software Stack for Neuromorphic Computing · 7/19/2017  · Complexity Report Fitness Get Random network Do Crossover ... Managing a software team in academia takes ... Software

Learning – Where does it go?

● The Current Learning Module– Manages epochs & populations

– Directs crossover & mutations, but doesn't do them.

– Manages parallelism, both within a machine and within a cluster (or Titan).

LearningModuleApplication

Architecture(Network)Network

Complexity

Report Fitness

Get Random networkDo CrossoverDo Mutation

Network

Calculate Fitness

Page 33: A Software Stack for Neuromorphic Computing · 7/19/2017  · Complexity Report Fitness Get Random network Do Crossover ... Managing a software team in academia takes ... Software

Learning – Where does it go?

● The problem with this approach– Large burden on the architecture developer.

– Does not give the learning module the ability to do anything fancy.

LearningModuleApplication

Architecture(Network)Network

Complexity

Report Fitness

Get Random networkDo CrossoverDo Mutation

Network

Calculate Fitness

Page 34: A Software Stack for Neuromorphic Computing · 7/19/2017  · Complexity Report Fitness Get Random network Do Crossover ... Managing a software team in academia takes ... Software

Learning – Where does it go?

● Instead – put a parameterized graph engine into the learning module.

LearningModuleApplication

Architecture(Network)Network

Complexity

Report Fitness

Graph

Network

Calculate Fitness

Graph properties

Manage general graphs:Creation

CrossoverMutation

Page 35: A Software Stack for Neuromorphic Computing · 7/19/2017  · Complexity Report Fitness Get Random network Do Crossover ... Managing a software team in academia takes ... Software

Learning – Where does it go?

● Instead – put a parameterized graph engine into the learning module.

LearningModuleApplication

Architecture(Network)Network

Complexity

Report Fitness

Graph

Network

Calculate Fitness

Graph properties

Manage general graphs:Creation

CrossoverMutation

Reduces theburden here.

Page 36: A Software Stack for Neuromorphic Computing · 7/19/2017  · Complexity Report Fitness Get Random network Do Crossover ... Managing a software team in academia takes ... Software

Learning – Where does it go?

● Instead – put a parameterized graph engine into the learning module.

LearningModuleApplication

Architecture(Network)Network

Complexity

Report Fitness

Graph

Network

Calculate Fitness

Graph properties

Manage general graphs:Creation

CrossoverMutation

Reduces theburden here.

Allows us tofocus here:

Page 37: A Software Stack for Neuromorphic Computing · 7/19/2017  · Complexity Report Fitness Get Random network Do Crossover ... Managing a software team in academia takes ... Software

Learning – Status

● Still in a feature branch – waiting on mrDANNA.

● Much easier to explore architectural features.

● Poised to exploit speciation / minimal augmenting topologies (NEAT & beyond).

● Still need to explore a more structured approach to STDP.

Page 38: A Software Stack for Neuromorphic Computing · 7/19/2017  · Complexity Report Fitness Get Random network Do Crossover ... Managing a software team in academia takes ... Software

Some lessons learned (high level)

● Performing simultaneous research on a variety of areas, and getting them to impact each other takes a careful eye on software design.

● There are a lot of un-sexy things that go into a successful hardware/software research project.

● Figuring out how to program applications on neuromorphic computing devices is a larger challenge than developing the devices.

Page 39: A Software Stack for Neuromorphic Computing · 7/19/2017  · Complexity Report Fitness Get Random network Do Crossover ... Managing a software team in academia takes ... Software

Some lessons learned (low level)

● Managing a software team in academia takes an iron fst and a thick skin.

● Program like it's 1998...

● One key to success is decomposing your research space into units that ft your workforce.

Page 40: A Software Stack for Neuromorphic Computing · 7/19/2017  · Complexity Report Fitness Get Random network Do Crossover ... Managing a software team in academia takes ... Software

Neuro

Apps

Architectures

Learning

Pole Robo Flappy Anomaly ClassifyCopter Microapp

NIDA mrDANNA DANNA

Reservoir

Sim Viz Sim Viz Sim

Hardware

SoftwareOrg

Chart.

Page 41: A Software Stack for Neuromorphic Computing · 7/19/2017  · Complexity Report Fitness Get Random network Do Crossover ... Managing a software team in academia takes ... Software

A Software Stack for Neuromorphic Computing

James S. PlankMark E. Dean

Garrett S. RoseCatherine D. Schuman

July 19, 2017Neuromorphic Computing Symposium

Knoxville, Tennessee

Thank youAFRLDOENSF