Unconventional Unconventional NetworkingNetworkingMakoto BentzMakoto Bentz

October 13, 2010October 13, 2010

CS 6410CS 6410

PapersPapers

U-Net: A User-Level Network Interface for Parallel U-Net: A User-Level Network Interface for Parallel and Distributed  Computing,and Distributed  Computing,  Von Eicken, Basu, Buch and Werner Vogels. 15th   Von Eicken, Basu, Buch and Werner Vogels. 15th SOSP, December 1995.SOSP, December 1995.

Active Messages: A Mechanism for Integrated CoActive Messages: A Mechanism for Integrated Communication and Controlmmunication and Control, Thorsten von Eicken, David E. Culler, Seth Copen , Thorsten von Eicken, David E. Culler, Seth Copen Goldstein, and Klaus Erik Schauser. Goldstein, and Klaus Erik Schauser. In Proceedings In Proceedings of the 19th Annual International Symposium on of the 19th Annual International Symposium on Computer ArchitectureComputer Architecture, 1992., 1992.

Evaluation of the Virtual Interface Architecture Evaluation of the Virtual Interface Architecture (VIA)(VIA). Xin Liu June 8, 1999 Department of . Xin Liu June 8, 1999 Department of Computer Science and Engineering University of Computer Science and Engineering University of California, San DiegoCalifornia, San Diego

OverviewOverview

Active Messages:Active Messages: Instructions between Instructions between multiprocessorsmultiprocessors

U-Net:U-Net: Networking beyond the kernel, Networking beyond the kernel, separation into the User Spaceseparation into the User Space

VIA:VIA: Fast Messaging on virtualized standard Fast Messaging on virtualized standard network interfaces (VIA)network interfaces (VIA)

Parallel versus Parallel versus DistributedDistributed

CPUCPUCPUCPU

CacheCacheCacheCache

MemorMemoryy

MemorMemoryy

CPUCPUCPUCPU

CacheCacheCacheCache

............

Parallel Parallel ComputingComputing

Multiple Multiple processors processors connected in a connected in a computercomputer

Tightly coupledTightly coupled

Shared memoryShared memory

Parallel versus Parallel versus DistributedDistributed

CPUCPUCPUCPU

CacheCacheCacheCache......

MemorMemoryy

MemorMemoryy

CPUCPUCPUCPU

CacheCacheCacheCache

MemorMemoryy

MemorMemoryy

Distributed Distributed ComputingComputing

Multiple Multiple autonomous autonomous computers computers connected by connected by networknetwork

Loosely coupledLoosely coupled

Distributed memoryDistributed memory

NetworkNetwork

Active MessagesActive Messages

Written at by von Eicken, Written at by von Eicken, Cohen and at University of Cohen and at University of California Berkeley, in David California Berkeley, in David E. Culler’s research groupE. Culler’s research group

Other projects by group Other projects by group include Split-C, Castle and include Split-C, Castle and NinjaNinja

NinjaNinja

Active Messages: Active Messages: AuthorsAuthors

Seth Copen Cohen is Seth Copen Cohen is now a professor at CMUnow a professor at CMU

Programmable matterProgrammable matter

Klaus Erik Schauser is Klaus Erik Schauser is now a professor at now a professor at UCSBUCSB

Parallel computingParallel computing

Thorston von EickenThorston von Eicken

Advised by David E. Culler (MIT, teaches at Advised by David E. Culler (MIT, teaches at U of C Berkeley)U of C Berkeley)

Former Cornell Assistant ProfessorFormer Cornell Assistant Professor

Active Messages becomes his Ph.D. thesisActive Messages becomes his Ph.D. thesis

Now teaches at UCSBNow teaches at UCSB

Worked at ExpertCity.com, worked on Worked at ExpertCity.com, worked on GoToMyPC and GoToMeetingGoToMyPC and GoToMeeting

Now works at RightScale, a cloud Now works at RightScale, a cloud computing companycomputing company

Active Messages: Active Messages: MotivationMotivation

Message passing machines Message passing machines have reasonable hardware have reasonable hardware costs but suffer fromcosts but suffer from

Poor overlap between Poor overlap between communication and communication and computationcomputation

High overhead for messagesHigh overhead for messages

And this is not due to And this is not due to hardwarehardware

Com

m.

Com

m.

Pro

c.

Pro

c.

WastedWasted

Tim

eTim

e

task

task

task

task

task

task

Active Active Messages:HardwareMessages:Hardware

nCube-2nCube-2

1.9 Gflops/s1.9 Gflops/s

Hypercube layoutHypercube layout

Thinking Machines Thinking Machines Connection Machines Connection Machines (CM)-5(CM)-5

Non-standard von Non-standard von Neumann architectureNeumann architecture

Fat-tree network MIMDFat-tree network MIMD

NSA FROSTBURG aNSA FROSTBURG aCM-5 machineCM-5 machine

Active Messages:DesignActive Messages:Design

Get around the comm/proc delayGet around the comm/proc delay

Active messages are messaging objects that Active messages are messaging objects that can perform its own processingcan perform its own processing

Each message carries a userspace handler or Each message carries a userspace handler or codecode

The message contains the arguments to pass The message contains the arguments to pass to the handlerto the handler

Single Program, Multiple Data

Active Messages:DesignActive Messages:Design

Program pre-allocates receiving structures, Program pre-allocates receiving structures, eliminating bufferingeliminating buffering

The network itself acts as a buffer for small The network itself acts as a buffer for small messagesmessages

Creates low overhead on both sidesCreates low overhead on both sides

Message handlers are not allowed to blockMessage handlers are not allowed to block

Active Active Messaging:TestingMessaging:Testing

nCube/2nCube/2

11us send + 15us receive (nearly an order of 11us send + 15us receive (nearly an order of magnitude reduction)magnitude reduction)

Almost near minimal instructionsAlmost near minimal instructions

CM-5CM-5

1.6us send + 1.7us receive1.6us send + 1.7us receive

Active Messaging:Split-CActive Messaging:Split-C

Spit-C PUT and GET Spit-C PUT and GET perform split-phase perform split-phase copies of memory copies of memory blocks between nodesblocks between nodes

For high performance, For high performance, should overlap should overlap communication and communication and computationcomputation

Active Messaging:Split-CActive Messaging:Split-C

Active Messaging:Message Active Messaging:Message Driven ArchitecturesDriven Architectures

Message Driven Architectures Support Message Driven Architectures Support languages with support for dynamic parallelismlanguages with support for dynamic parallelism

Active Message handlers execute immediately Active Message handlers execute immediately upon message arrival, cannot suspend or blockupon message arrival, cannot suspend or block

Active Active Messaging:Further Messaging:Further Work: TAMWork: TAM

Threaded Abstract Machine is a fine-grained Threaded Abstract Machine is a fine-grained parallel execution model based on Active parallel execution model based on Active MessageMessage

No testing resultsNo testing results

Active Messaging: Active Messaging: DiscussionDiscussion

This paper becomes von Eicken’s thesisThis paper becomes von Eicken’s thesis

I personally disliked the style of this paper, but I personally disliked the style of this paper, but liked the contentliked the content

Too much in one article, works much better Too much in one article, works much better as a thesisas a thesis

U-Net: AuthorsU-Net: Authors

Assistant Professor von EickenAssistant Professor von Eicken

Anindya Basu (advised by von Anindya Basu (advised by von Eicken)Eicken)

Vineet Buch (M.S. from Cornell)Vineet Buch (M.S. from Cornell)

co-founds like.com (now part of co-founds like.com (now part of Google)Google)

worked with BlueRun (VC)worked with BlueRun (VC)

Werner VogelsWerner Vogels

former Researcher at Cornellformer Researcher at Cornell

now Amazon CTOnow Amazon CTO

U-Net: MotivationU-Net: Motivation

Networks are now so fast the biggest delay Networks are now so fast the biggest delay is kernel processing of the message stacksis kernel processing of the message stacks

Developing in kernel space limits Developing in kernel space limits development of new message send/receive development of new message send/receive interfacesinterfaces

U-Net: MotivationU-Net: Motivation

Removing the kernel from Removing the kernel from the critical path would the critical path would

allow specializationallow specialization

U-Net: MotivationU-Net: Motivation

Previously, Kernel’sPreviously, Kernel’sResponsibility, andResponsibility, and

opaque to theopaque to theapplicationapplication

Want toWant tomake thismake thisapplicationapplication

specificspecific

Would like to be able to integrate Would like to be able to integrate application specific behavior to application specific behavior to

optimize performanceoptimize performance

U-Net: DesignU-Net: Design

U-Net must be able toU-Net must be able to

Multiplex the network between processesMultiplex the network between processes

Provide isolation and protection between Provide isolation and protection between processesprocesses

Manage resources without kernel pathsManage resources without kernel paths

Allow the development of an interfaceAllow the development of an interface

U-Net: Design ConcernsU-Net: Design Concerns

Focus on low overheads to optimize small Focus on low overheads to optimize small message transmission latency and bandwidthmessage transmission latency and bandwidth

Emphasis on protocol design and integrationEmphasis on protocol design and integration

To do both on standard off-the-shelf hardwareTo do both on standard off-the-shelf hardware

U-Net: Design: Low U-Net: Design: Low OverheadOverhead

U-Net components map toU-Net components map to

Real hardware in the NIReal hardware in the NI

-and/or--and/or-

Memory locations interpreted by the OSMemory locations interpreted by the OS

Zero-Copy tries to limit intermediate Zero-Copy tries to limit intermediate buffering between NI and User-Levelbuffering between NI and User-Level

U-Net: Design: ProtocolsU-Net: Design: Protocols

Tags specific to the network substrate used for Tags specific to the network substrate used for addressingaddressing

ATM (VCI)ATM (VCI)

Ethernet (Physical addressing)Ethernet (Physical addressing)

Endpoints, communication channels and Endpoints, communication channels and queues are only accessible by ownersqueues are only accessible by owners

U-Net: Design: HardwareU-Net: Design: Hardware

Implemented on Implemented on SPARCstationSPARCstation

Fore SBA-100 Fore SBA-100 interfacesinterfaces

Fore SBA-200 Fore SBA-200 interfacesinterfaces

Could use on-board Could use on-board processor on SBA-200, processor on SBA-200, custom firmware due to custom firmware due to poor bundled firmwarepoor bundled firmware

U-Net: Testing ResultsU-Net: Testing Results

U-Net: Testing ResultsU-Net: Testing Results

U-Net: DiscussionU-Net: Discussion

Feasible technologyFeasible technology

Good performance on off the shelf Good performance on off the shelf hardware with existing standardshardware with existing standards

““This encouraging result should, however, This encouraging result should, however, not obscure the fact that not obscure the fact that significant significant additional system resourcesadditional system resources, such as , such as parallel process schedulers and parallel file parallel process schedulers and parallel file systems, systems, still need to be developedstill need to be developed before the cluster of workstations can be before the cluster of workstations can be viewed as a unified resource.”viewed as a unified resource.”

FM/VIAFM/VIA

Virtual Interface Architecture standardizes Virtual Interface Architecture standardizes System Area Networks (Microsoft, Intel, System Area Networks (Microsoft, Intel, Compaq)Compaq)

VIA however, has no simple programming VIA however, has no simple programming interfaceinterface

Fast Messages provided layering for small Fast Messages provided layering for small messagesmessages

FM/VIA :: Testing ResultsFM/VIA :: Testing Results

FM/VIA :: TestingFM/VIA :: Testing

Effects of U-NetEffects of U-Net

FM/VIA was not useful because it did not meet FM/VIA was not useful because it did not meet the requirement of fast end pointsthe requirement of fast end points

FM/VIA goes on to influence InfiniBand, iWarpFM/VIA goes on to influence InfiniBand, iWarp

Zero-copyZero-copy