Benny Koren - Beyond CMOS · Beyond CMOS Computing Benny Koren Annapolis, November 2017 Next generation HPC Fabric and Interconnects

Beyond CMOS Computing

Benny Koren

Annapolis, November 2017

Next generation HPC Fabric and Interconnects

© 2017 Mellanox Technologies 2

Cloud Big Data

Enterprise

Business Intelligence

HPC

Storage

Security

Machine Learning

Internet of Things

Exponential Data Growth Everywhere


MoreData

BetterModels

FasterInterconnect

Mellanox Unleashes the Power of Artificial Intelligence

GPUs

CPUs

FPGAs

Storage

Enabling World-Leading Artificial Intelligence Solutions


NVMe, NVMe Over Fabrics

Big Data Storage

Hadoop / Spark

Software Defined Storage

SQL & NoSQL Database

Deep Learning

HPC

GPUDirect

RDMA

MPI

NCCLSHARP

Image , Video Recognition

Sentiment Analysis

Fraud & Flaw

Detection

Voice Recognition & Search

Same Interconnect Technology Enables a Variety of Applications


Highest-Performance 200Gb/s Interconnect Solutions

Transceivers

Active Optical and Copper Cables

(10 / 25 / 40 / 50 / 56 / 100 / 200Gb/s) VCSELs, Silicon Photonics and Copper

40 HDR (200Gb/s) InfiniBand Ports

80 HDR100 InfiniBand Ports

Throughput of 16Tb/s, <90ns Latency

200Gb/s Adapter, 0.6us latency

200 million messages per second

(10 / 25 / 40 / 50 / 56 / 100 / 200Gb/s)

16 400GbE, 32 200GbE, 128 25/50GbE Ports

(10 / 25 / 40 / 50 / 100 / 200 GbE)

Throughput of 6.4Tb/s

MPI, SHMEM/PGAS, UPC

For Commercial and Open Source Applications

Leverages Hardware Accelerations


Higher Data SpeedsFaster Data Processing

Better Data Security

Adapters SwitchesCables &

Transceivers

SmartNIC System on a Chip

Artificial Intelligence Needs the Most Intelligent Interconnect


Mellanox Accelerates Next Generation HPC and AI Systems

Summit CORAL System

Fastest Supercomputer in Japan

Fastest Supercomputer in CanadaInfiniBand Dragonfly+ Topology

Sierra CORAL System

Highest Performance with InfiniBand


Mellanox Accelerates TOP500 Supercomputers

InfiniBand connects 6 times more new systems versus OmniPath (June’17 - Nov’17)

InfiniBand connects 77% of new HPC systems (June’17 – Nov’17)

InfiniBand connects 2 of the top 5 supercomputers (#1 - China, #4 - Japan)

Mellanox connects 39% of overall TOP500 systems (192 systems, InfiniBand and Ethernet)

InfiniBand connects 33% of the total TOP500 systems (164 systems)

InfiniBand connects 60% of the HPC TOP500 systems

25G Ethernet first appearance on the Nov’17 TOP500 list (China Hyperscale company) - 19 systems, all are Mellanox connected

Mellanox connects all of 25G, 40G and 100G Ethernet system on the list

The Interconnect of Choice for HPC InfrastructuresEnabling Machine Learning, High-Performance, Web 2.0, Cloud, Storage, Big Data Applications


The Need for Intelligent and Faster Interconnect

CPU-Centric (Onload) Data-Centric (Offload)

Must Wait for the Data

Creates Performance BottlenecksAnalyze Data as it Moves!

Faster Data Speeds and In-Network Computing Enable Higher Performance and Scale


In-Network Computing

Self-Healing Technology

In-Network Computing

Unbreakable Data Centers

Delivers Highest Application Performance

GPUDirect™ RDMA

Critical for HPC and Machine Learning ApplicationsGPU Acceleration Technology

10X Performance AccelerationCritical for HPC and Machine Learning Applications

35XFaster Network Recovery5000X

10X Performance Acceleration

Performance Acceleration

In-Network Computing Delivers Highest Performance


InfiniBand Delivers Best Return on Investment

30%-250% Higher Return on Investment

Up to 50% Saving on Capital and Operation Expenses

Highest Applications Performance, Scalability and Productivity

1.3X Better 2X Better 1.4X Better 2.5X Better 1.3X Better

Molecular DynamicsChemistry Automotive GenomicsWeather


The 4x25G 100Gb/s SiP chips which migrate to 4x100G 400Gb/s

We can keep the same low-cost QSFP package for 200G; 400G requires new package but same optical path


Multi-Mode Optics

3m-100m

Where Silicon Photonics Fits in Data Center Fabrics Today

DACServer/ToR-to-ToR

SR4For structured cabling

Short Reaches

PSM4

DAC

CWDM4

8-Fiber

MPOAOC: 3-50m

AOCToR-Leaf/Spine

“DAC In the Rack”

3m

Quad 25G SFP

breakout

Dual 50G

Breakout

25G SFP

Quad 25G SFP

breakout

Dual 50G

Breakout

25G SFP

For Structured Cabling

Long Reaches

2-Fiber

LC

Single-Mode Optics

Up to 2Km

Optical

Patch

Panel

Served by SiP


Multi-Mode (MMF) to Single-Mode (SMF) Fiber Transition

40GbE <500m (MMF)73%

40GbE 500m-2km (SMF)…

40GbE > 10km (SMF)…

100GbE <500m (MMF)26%

100GbE 500m-2km (SMF)…

100GbE >10km (SMF)30%

Source: LightCounting Market Forecast, April 2016

Served by SiP

2014 2021

2014-2021 40GbE vs. 100GbE Cable length/type average usage


Data Center Optical Transceivers Forecast 2014-2021

2014 2015 2016 2017 2018 2019 2020 2021

Nx10 GbE Nx25 GbE Nx50 GbE

Served

by

SiP

Enabled

by

SiP

Source: LightCounting Market Forecast, April 2016.

Nx10G

Nx25G

Nx50G


Mellanox FK Modulator Scales From 100G PSM4 to 400G DR4

28GBd NRZ 50GBd NRZ 56GBd NRZ

28GBd PAM4 56GBd PAM450GBd PAM4

200G

DR4

400G

DR4

100G

PSM4200G

NRZ


The Rx With Ge Detectors Also Scales to 400G DR4

56GBd NRZ28GBd NRZ

50GBd PAM4 56GBd PAM428GBd PAM450GBd PAM4

200G

DR4

400G

DR4

100G

PSM4200G

NRZ


400G Tx-Rx Link: Ge PD Electrical Eyes using FK Modulator

28GBd NRZ 56GBd NRZ

28GBd PAM450GBd PAM4

56GBd PAM4

50GBd PAM4

200G

DR4

400G

DR4

100G

PSM4200G

NRZ


200G (4x25GB) SiP Private Demo at OFC 2017

Demo was an end-to-end 50G PAM4 (25GBaud) link

over 500m of SMF fiber, with Inphi TIA.

BER was less than 1E-12


Key Benefits of Mellanox’s Si Photonics Platform

Mellanox’s high confinement Si Photonics platform enables:

Broadband , polarization independent, low loss operation: better link budgets, better yields

WDM capability for high aggregate bandwidths

Monolithic integration of low power and high speed modulators and detectors

Low cost flip chip attach of III/V elements andwafer scale packaging and assembly

High manufacturing tolerances

1520 1530 1540 1550 1560 1570-0.5

-0.4

-0.3

-0.2

-0.1

0

Wavelength(nm)

Pro

paga

tion

Loss

(dB

/cm

)

TE

TM

1520 1530 1540 1550 1560 1570-0.5

-0.4

-0.3

-0.2

-0.1

0

Wavelength(nm)

Pro

paga

tion

Loss

(dB

/cm

)

TE

TM

Wavelength (nm)

Lo

ss d

B/c

m

20


Mellanox Offers Ready to use Optical Engine Solutions

Broad and Open Eco-System

• Multiple Laser Vendors

• Multiple IC Solutions, can source independently

• External and Internal Foundry

Tx ChipRx Chip

Glob topped laser

Fiber ferrule / block

Power

monitors

High speed

modulators

High speed Ge PD

Purpose

• Enable customer based innovation

• Generate volume (and cost reduction)

• Enhance and promote eco-system: Foundry, IC,

Laser , module products etc

• Enable broader deployment of SiP

What is the Optical Engine

• A ready to use SiP based TOSA and ROSA

solution

• Laser (FP / DFB etc) already attached on SiP

• Fiber attached and terminated as required

• Recommend working ICs: Driver, TIA, CDR etc

• Reference designs and process recipes to assist

deployment and use in module products


What is The Optical Engine Product?

FP or DFB Laser

attached, wire

bonded and

sealed

High speed integrated EA

modulators (with optional integrated

heaters for wavelength tuning)

Power monitors

measure true power

in waveguide

Integrated

temperature

sensors

Integrated temp

sensors

High speed

integrated Ge PDs

ROSA

TOSA

Fibers are

attached to the

Tx and Rx chip

Predetermined fiber

pigtail length

Optional connector

termination for LC,

MPO etc

Thank You

Documents

Benny Koren - Beyond CMOS · Beyond CMOS Computing Benny Koren Annapolis, November 2017 Next generation HPC Fabric and Interconnects