Mr. P. K. Gupta Sandeep Gupta 051416Roopak Agarwal 051309

Road Map:

Problem StatementIntroduction Storage Systems And Storage Devices

Brief Introduction To SAN and NAS DMA and RDMA

How Does RDMA WorksRDMA Application

Future Work

Problem Statement

• Network Bandwidth: 40%/yr• CPU Power: x2 every 18 months• Storage Capacity x2/yr

• Memory and I/O bandwidth?

A Server-to-Storage Bottleneck

Source: Brocade

Storage Area Network (SAN) Storage accessed over a Fibre Channel switching

fabric, using encapsulated SCSI. Block level storage system Fibre-Channel SAN IP SAN

• Implementing SAN over well-known TCP/IP

• iSCSI: Cost-effective, SCSI and TCP/IP

NAS• Network Attached Storage(NAS)• Storage accessed over TCP/IP, using industry

standard file sharing protocols like NFS, HTTP, Windows Networking Provide File System FunctionalityTake LAN bandwidth of Servers

04/20/23 10

Motivation for RDMA

• Motivation Take advantage of the RDMA protocol suite & RNIC hardware.

When RDMA technology advances, so does iSCSI.

Zero-copy, direct data placement can be achieved with generic protocols.

• High-level goal: Run “most efficiently” on generic RNICs.

Direct Memory Access (DMA)

• Direct Memory Access (DMA) is a capability provided that allows data to be sent directly from an attached device to the memory on the computer's motherboard.

Remote Direct Memory Access (RDMA)

RDMA is a concept whereby two or more computers communicate via Direct memory Access directly from the main memory of one system to the main memory of another .

DMA RDMA

What Makes RDMA Advantageous ?

• There is no CPU cache or context switching overhead needed to perform the transfer.

• Transfers can continue in parallel with other system operations.

Some Objections to RDMA

• More complex API than TCP/stream interface– A new API is not required (but worthwhile for some applications)– Sockets Direct Protocol (SDP)

• Hardware vendors must all agree for approach to succeed in the market– –Extremely valid, but worth the cost

• Security concerns about opening memory on the network– Hardware enforces application buffer boundaries. – Makes it no worse than existing security problem with a 3rdparty inserting data

into the TCP data stream.

RDMA Applications• Clustered Database , Application Servers : They

generate a large number of short messages , typically sent for synchronization and locking among nodes, so latency affects the degree to which they can scale.

• High performance Computing • Storage Infrastructure

Future Work

• RDMA over WAN

• iSCSI extension for RDMA (iSER)

iSCSI Extension for RDMA (iSER) :

• Protocol that maps the iSCSI protocol, available from RDMA

• Only data transfer in iSCSI protocol is changed

• Reduces TCP/IP overhead

04/20/23 20

iSER Big picture

• Layered on RDMAP protocol.

• iSCSI still requires direct TCP access for bootstrapping (login).– Transition to RDMA

• Same iSCSI well-known port

SCSI

iSCSI

TCP

Datamover Interface

iWARPprotocolsuite

RNICRNIC

iWARPVerbs

Generic RDMA over TCP/IP

1

RDMAP

DDP

MPA

iSER

2

A TCP-based implementation is shown as an example.

Any Queries?