AHaH Computing

Anti-Hebbian and Hebbian

(AHaH) Computing

MT5009

Analyzing Hi-Technology

Opportunities

1. Chow Ka Yau Daniel A0145207M

2. Muhammad Dzahir Bin Mohamed Zain Affandi A0129428Y

3. Gregory Chee Ken Khyun A0132405W

4. Jayapathma Herath Madhushanka Meranjan A0132398Y

5. Lim Yee Hao Marcus A0132390N

http://www.riken.jp/en/research/rikenresearch/highlights/7918/

• Current worldwide buzz - Big Data analytics

• Future: Not just analytics but also

• Solutions: Predictive and Prescriptive analytics

• We need

- Machine learning

- Intelligent computing

- Large scale simulations

Problem: Current Von-Neumann computing bottleneck

We believe, AHaH Computing can break the Von-Neumann

bottleneck and open up a new era of big data analytics

Problem Statement

CONTENTS1. The Von-Neumann Architecture and Limitations

2. Solutions to break Von-Neumann bottleneck

3. AHaH Phenomenon

4. AHaH Computing Architecture

5. Comparison of AHaH, VN and Neuromorphic Computing

6. AHaH Advantages

7. Memristor – Memory Trends

8. Big Data bottleneck, Model, Market and Challenges

9. Big Data Analytics Applications

10. Conclusion

http://hartenstein.de/cited/Damian_Millers-Award.pdf

Von-Neumann (VN) Architecture

and Limitationshttp://sybaseblog.com/2013/05/0

6/need-of-in-memory-technology-

sap-hana/

Memory access speed

Processor speed



3. AHaH Phenomenon



6. AHaH Advantages




10. Conclusion

http://www.gridgain.com/wp-content/uploads/2014/09/insideBIGDATA-Guide-to-In-

Memory-Computing.pdf

- Keeping data in a server’s RAM instead

of hard disk or flash devices

- Massive parallelization for faster

processing speeds

- Inexpensive way to speed up enterprise

software applications, including but not

limited to analytics

Challenges:

- Requires lots of RAM!

- Unsustainable brute force method

since data volumes continue to

explode in a big data world

- High power consumption

- Processor and memory are still

separated

http://www.toddmace.io/

1. Algorithmic Approach: In-Memory Computing

Solutions to break VN bottleneck

Increase

bandwidth

Cache

Pre

fetching

Multi

threading

Parallel

processing

Pipelining

In-

Memory-

Computing

(IMC)

1. Algorithmic Approach: Software

http://tonycosentino.ventanar

esearch.com/


2. Neuromorphic Approach

http://www.slideshare.net/Funk98/neurosynaptic-chips

IBM’s True North

supercomputer incorporates

the largest neuromorphic

chip in the world, but the

chip is not capable of

learning on its own


Limitations No active machine

learning on chip

No unsupervised

learning

Required

supercomputer

http://www.slideshare.net/Funk98/neurosynaptic-chips?qid=e85f972a-2571-49d0-ac6c-4b4395525901&v=default&b=&from_search=1

Aim: to achieve the level

whereby it is able to do

what brain does –

processing and memory

are performed by the

same component

https://upload.wikimedia.org/wikipedia/commons/d/df/PPTExponentialGrowthof_Computin

g.jpg

The Future of Computing

http://www.slideshare.net/Funk98/neurosynaptic-chips?qid=e85f972a-2571-

49d0-ac6c-4b4395525901&v=default&b=&from_search=1

3. Self - Organizational Approach

AHaH computing is more than just the integration of

memory and processing!

Anti-Hebbian and Hebbian

(AHaH) Computing




3. AHaH Phenomenon



6. AHaH Advantages




10. Conclusion

• Creation of conductive path through a common

medium

• It just happens naturally (self organization)

• No need for external control to produce path of

conduction

• Occurs in rivers, air, blood system etc.

Bifurcation video

AHaH PhenomenonExamples of Natural Adaptive System

http://knowm.org/blog/

It is this

energy

dissipating

pathways

competing and

conducting for

resources

It is a Learning Process

AHaH PhenomenonBiology – Brain Synapse

- Strong Spikes Strengthen the connection

- Open up new pathways

- Hebbian Learning

- Weak Spikes (misfiring) weaken the

connection

- Delete up pathways

- Anti - Hebbian Learning

Model

• Firing strengthen pathways by increasing

synaptic weight change. This Adaptation

is called Hebbian learning

• Misfiring weaken the pathway connection

by decreasing synaptic weight change.

This Adaptation is called Anti - Hebbian

learning

Which component can help to mimic this synaptic adaptation ?

?

?

http://www.smashinglists.com/top-10-amazing-facts-

about-the-human-brain/http://www.intechopen.com/books/reinforcement_learning/interaction_between_the

_spatio-temporal_learning_rule__non_hebbian__and_hebbian_in_single_cells__a_c

MEMRISTORS

AHaH PhenomenonMemristors - Mimic Synaptic Adaptation

Analogous to the adaptive water pipeline

1) High pressure difference -> more water flow -> diameter gets bigger

2) Cut water supply -> diameter stays same – Remembers how much water has flown

3) Less pressure difference -> less water flow -> diameter gets smaller

4) Water flows in the opposite direction if diameter gets two small

– Erasing the path and flow is bi-directional

Replace the water flow with Current flow

Replace the pressure with Resistance

Synaptic Adaptation

Memristors adapts its resistance as current flows through the memristor

Non - Volatile memory

Remembers how much current flowed through the memristor

InputsOutput

Memristors

http://cacm.acm.org/news/33675-memristor-minds-the-future-of-artificial-intelligence/fulltext



3. AHaH Phenomenon



6. AHaH Advantages




10. Conclusion

http://www.hpcwire.com/2015/09/09/knowm-snaps-in-final-piece-of-memristor-puzzle/

AHaH Computing ArchitectureOverall Architecture

Connected

MemristorsConnected KT-

Synapse in Parallel

– AHaH node

Map Many KT-Cores

into RAM

architecture

KT-Core

• Connected AHaH Nodes in

Parallel

• Column decoder and Row

decoder to select AHaH Nodes

• Controller to control Instruction

flow into AHaH Nodes

Basic RAM ArchitectureAHaH-NodeINPUTS

Output

http://i.cmpnet.com/pldesignline/20

05/07/zeidmanfigure1.gif

AHaH Unites Memory and processingExample- A . B = C

CPU

MEMORY

Row

Decorder

Column Decoder

InputOutput

B

A C

Row

Select

Column

SelectRead Write

CPU

1) Inputs are

connected with

multiple coresKT-RAM

Activation of

KT-Cores

2)Activation

Instruction

Act of accessing the

memory Becomes the act

of configuring the

Memory

- Weight Change

occurs inside memristors

KT-Core

During

Activation,

AHaH

Controller

connects with

Multiple AHaH

nodes

InputA

AHaH

Von-Neumann

3) Adaptation

Instruction

- AND Logic

4) Data Out

Output

C• Creates logics within the memory using sequential Instructions flow

• Adaption happens for free, because memristors adopt as we use them

• No back and forth data transfer between memory and CPU

ALU

C

BC



3. AHaH Phenomenon



6. AHaH Advantages




10. Conclusion

Comparison of AHaH, Von-Neumann and

Neuromorphic ComputingArchitecture:

Conventional

Computing

Neuromorphic

Computing

AHaH

Computing

Architecture Von Neumann Neural Network AHaH Architecture

Computing Unit CPU Synaptic Chip Synaptic Chip

Storing Unit Memory Synaptic Chip Synaptic Chip

Storing Element DRAM DRAM/SRAM Memristors

Suitability Logical and Analytical Machine Learning

(pattern recognition)

Logical & analytical and

Machine learning

(pattern recognition)

Processing Serial Processing

(multi cores)

Parallel Processing Parallel Processing

Backward

Compatibility

Only in Von-Neumann

Architecture

Unable to use in von-

Neumann architecture

directly

(Require Supercomputer)

Able to use in both Von-

Neumann & AHaH

Architecture

Power Consumption High Low Ultra Low

Speed Slow Fast Fast



3. AHaH Phenomenon



6. AHaH Advantages




10. Conclusion

• Barrier potential inherently exists between

CPU and RAM especially when the it is

physically separated.

• Sufficient energy must be applied to

overcome this barrier potential

• Power between the CPU and RAM heavily

depends on the distance between the CPU

and RAM

• Each read operation lowers the switch

barriers. Thus, the act of accessing the

memory becomes the act of configuring

the memory over time

In brains, d = 0 which

would mean high

amounts of power

saving

AHaH Advantages: Lower power

https://www.youtube.com/watch?v=CFSrC7kjbJo

Less processing

requests and reply

between CPU and

memory due to a

internal machine

learning

AHaH Advantages:Power and Time Reduction


More back-and-forth operations! Single package reply!

http://www.colocationamerica.com/blog/energy-wasting-data-centers

http://perspectives.mvdirona.com/2009/05/the-datacenter-

as-a-computer/

AHaH Advantages:Data Center Power Consumption

Servers and Storage

• Increasing energy consumption

per year due to increasing data

storage demands

• Higher energy consumption

equates to higher electricity cost

expenditure

http://www.itbusinessedge.com/info/PP-BuildDataCenter-pg9.aspx

AHaH Advantages: Machine Learning (Example)

http://image.slidesharecdn.com/bmvass2014breckonml-140709055858-phpapp01/95/machine-learning-fro-computer-vision-a-whirlwind-of-key-concepts-for-the-uninitiated-7-638.jpg?cb=1405575596

http://images.slideplayer.com/11/3288366/slides/slide_2.jpg

AHaH Advantages: Machine Learning (Example)

Example:

Rich person has a high confidence level of having

high education

Rich person has a high confidence level of being old

Sequence 3: Unsupervised learning

Sequence 1: Supervised Learning

Sequence 2: Assign Label for KT- Cores (rich and poor)

Model

Input

Spikes



http://knowm.org/thermodynamic-ram-

technology-stack-published/

AHaH Advantages: Flexibility of AHaH

• Integration AHaH circuits with existing

integrated circuits technology by

apply it at the end of the process line

at the very end

• Front-end-of-line refers to the current

integrated circuits.

• Back-end-of-line refers to AHaH

circuit would be fabricated on top of

it.

• Able to take advantage of an

existing process that is working very

well and bringing it to the next level

of performance

• Create new computers that adapts as

it is usedhttps://www.youtube.com/watch?v=w7q07eKPM9U

AHaH

circuits

Current

integrated

circuits



3. AHaH Phenomenon



6. AHaH Advantages




10. Conclusion

AHaH Enabler - Memristor

Memristor as building block

ReRAM AHaH NVRAM

Symbol

Unit:

“Most Memristors that I have seen do not behave like fast, binary, non-volatile, deterministic switches. This is a problem because this is how HP wants them to behave”

• Alex Nuegent – Lead Inventor and CEO of Knowm

HP’s Memristor Problem

The incumbents are limiting the applications of memristors

within the existing memory technology framework

Limit the use of memristor

http://knowm.org/the-problem-is-not-memristors-its-how-hp-is-trying-to-use-them/

https://indico.cern.ch/event/345619/session/1/contribution/10/attachments/681170/935777

/HW_trends_market_costs_BPS_Apr2015_v14.pdf

Memristors – Memory Trends: Cost

NVRAM become cheaper

NVRAM vs SRAM

• Closing gap

• IBM’s synapse chips uses SRAM

• Potential opportunity to expand AHaH computing market (memristor –NVRAM)

Memristors - Memory Trends: Storage Capacity

ReRAM ReRAM vs other emerging NVRAM

• Highest capacity over others

http://www.maltiel-consulting.com/ISSCC-2013-Memory-trends-FLash-NAND-DRAM.html

http://www.tomsitpro.com/articles/flash-data-center-advantages,2-744-3.html

Memristors - Memory Trends: Manufacturability

ReRAM vs other emerging NVRAM

• The only memory compatible for mass production

http://www.storagenewsletter.com/rubriques/market-reportsresearch/non-volatile-memories-yole/

NVRAM Trend AnalysisCurrent NVRAM Suppliers

ReRAM

Potential

Market

Non-Memristor based Memristor based

BIG DATA

Analytics

2015 onwards, prediction of the mass manufacturing of Memristor to be availableThis will bring forth the further improvement of AHaH computing architecture where AHaH synaptic chip can be produced based on memristors

http://www.reram-forum.com/2013/03/21/predicting-the-reram-

roadmap/

Future opportunity for Memristor production



3. AHaH Phenomenon



6. AHaH Advantages




10. Conclusion

http://www.datasciencecentral.com/forum/topics/the-3vs-that-define-big-data

Definition of Big Data

Big Data 3V Model

Definition of Big Data

http://www.csc.com/insights/flxwd/78931-

big_data_universe_beginning_to_explode

The global production of data

is expanding and reach

~40ZB by 2020

https://www.capgemini.com/blog/capping-it-

off/2014/07/are-you-effectively-using-big-data

> 85% of an

organization’s data

is unstructured

Time and energy

consuming to

process unstructured

data

https://web-assets.domo.com/blog/wp-

content/uploads/2014/04/DataNeverSleeps_2.0_v2.jpg

Data velocity is

measured against

time

Enable real time

streaming

processing

Volume, Variety & Velocity

http://wikibon.com/wp-content/uploads/kalins-pdf/singles/big-data-vendor-revenue-and-

market-forecast-2011-2026.pdf

Wikibon forecasted Big Data

market to have 17% Compound

annual growth rate over 15 years

(2011-2026)

McKinsey Global Institute, Game changers: Five

opportunities for US growth and renewal, July 2013

Big Data identified as one of the

game changer that can boost US

annual GDP by 2020

Big Market for Big data

Business Intelligence

comprises of tools and

methodology for data

analyzing

Data / Big Data Analytics

can be grouped under

Business Intelligence

Past in Nature:

Descriptive and Diagnostic

Analytics

Future in Nature:

Predictive and

Prescriptive Analyticshttp://www.fyisolutions.com/blog/advanced-analytics-seminar/

Big Data Analytics

The Bottleneck is in technology

Not only need new algorithms and techniques but

breakthrough computing architecture

The Big Hurdle

http://image.slidesharecdn.com/finalpresentation-150305004602-conversion-gate01/95/presentation-on-big-data-analytics-15-638.jpg?cb=1425516438



3. AHaH Phenomenon



6. AHaH Advantages




10. Conclusion

http://ayata.com/stage/wp-content/uploads/ayata-infographic-2012-09-04.jpg

Big Data Analytics

ApplicationsHealthcare Industry

Given agility to government

to combat flu epidemic

dealing with vaccine

production/delivery rate vs

outbreak numbers in various

states

Autonomous Vehicle

Google AV to recognize &

anticipate what might be

coming in real time at a

junction

Oil & Gas Industry

Chevron need to analyze 50

terabytes of seismic data

Drilling miss cost USD$100M

Retail Industry

Starbucks marketing strategy

aligned to real time data

and responses



3. AHaH Phenomenon



6. AHaH Advantages




10. Conclusion

CONCLUSION

• Big market and growth of Big Data applications

• Von-Neumann architecture bottleneck is hitting the limits

• Cutting edge of AHaH computing architecture

• Real time processing (Integrated memory & processing)

• Ultra less power consumption and less heat generate

• Self-Organized approach

http://ekvv.uni-bielefeld.de/bilddb/bild?id=87240

ANY QUESTIONS?

For further readings on AHaH computing, please

visit www.knowm.org

(Startup for AHaH – started July 2015)

http://www.knowm.org/