Module 2 Storage Network Architecture

1. SCSI

2. FC Protocol Stack

3. SAN:FC SAN

4. IP Storage

5. Infiniband and Virtual Interfaces

By Shital H. More

FIBRE CHANNEL SAN 1. First consider the three FC topologies point-

to-point, fabric and arbitrated loop more closely.

2. Some hardware components that are required for the realisation of a Fibre Channel SAN.

3. InterSAN:The networking of small storage network islands to form a large SAN.

4. Interoperability in Fibre Channel SANs

By Shital H. More

FC topologies

Point-to-point topology • The point-to-point topology connects just two devices

and is not expandable to three or more devices.

• It connects a server to a storage device.

• It offer advantages compared to SCSI cabling:

1. Allows grater cable lengths with FC , up to 10km without repeater or extender than with SCSI which supports only up to 25m.

2. The optical transmission via fibre-optic is robust .

3. This type of dedicated connection allows full bandwidth utilization.

By Shital H. More

FC topologies

Point-to-point topology It has some drawbacks :

1. It offers only limited connectivity as only two devices can connect at a time.

2. It cannot be scaled to accommodate a large number of devices.

By Shital H. More

Fabric topology

• It is the most flexible and scalable of the three Fibre Channel topologies.

• One or more FC switches are connected together to servers and the storage devices.

• End devices (servers and storage devices) connected to the various FC switches can exchange data by means of switch-to-switch connections (Inter Switch Links, ISLs).

• Several ISLs can be installed between two switches in order to increase the bandwidth.

By Shital H. More

Fig.1 :The FC switches connected via inter-switch links(ISLs)

ISL

By Shital H. More

Zoning • Zoning within the FC network is used to limit the

visibility of end devices.

• It allows the servers to see and access storage devices that are located in the same zone.

• There are three types of zoning:

1. Port Zoning

2. WWN Zoning

3. Mixed Zoning

By Shital H. More

1. Port Zoning

• Zones are describe on the basis of port address, which needs to be altered every time as every port has a different port address.

• The port or end device in the FC network restrict itself to the information of the name server.

• If an end device asks the name server about other end devices in the Fibre Channel network, it is only informed of the end devices with which it shares at least one common zone.

• If, however, an end device knows the address (Port ID) of another device, it can still communicate with it.

By Shital H. More

2. WWN Zoning

• Zones are describe by WWNNs and WWPNs.

• Since the WWNs are unique and static

3. Mixed Zoning

• It combination of WWN zoning and port zoning.

• It facilitates the benefits of both zoning type.

By Shital H. More

InterSANs

• Fibre Channel SAN is a comparatively new technology.

• In many data centres in which Fibre Channel SANs are used, there will be several islands of small Fibre Channel SANs than one large Fibre Channel SAN (Figure.2 ).

By Shital H. More

Figure 2: Current IT environments (2009) consist almost exclusively of

small FC SANs that are designed with built-in redundancy.

• If a server has to access storage devices that are connected to a

different SAN, it has to take the indirect route via the LAN and a

further server.

By Shital H. More

InterSANs

• Over 80% of the installed Fibre Channel SANs consist only of up to four Fibre Channel switches.

• A server can only indirectly access data stored on a different SAN via the LAN and a second server.

• The reasons for the islands of small Fibre Channel SANs are that they are simpler to manage than one large Fibre Channel SAN and that it was often unnecessary to install a large one.

By Shital H. More

InterSANs

• FC SAN was used to replace SCSI, i.e it is an alternative to SCSI cabling.

• FC SAN offers flexible sharing of the capacity of a storage device between several servers.

• Benefits of using FC SAN are:

i. Application Backup

ii. Remote data mirroring

iii. Data sharing over FC SAN

iv. Storage virtualization

By Shital H. More

InterSANs

• To form a large SAN from multiple FC SAN is one field of application in which an FC director is used as an FC Switch(figure.3) .

By Shital H. More

Figure 3: More and more applications are exploiting the new

possibilities offered by Fibre Channel SAN.

• In order to make better use of the new possibilities of

storage networks it is necessary to connect the

individual storage islands. By Shital H. More

Interoperability of Fibre Channel SAN

• The FC SAN interoperability issues need to be addressed with respect to the underlying FC network layer, FC application protocols such as FCP (SCSI over FC) and the application running on the FC SAN.

• The interoperability of FC SAN can be understood by the interoperability of FCP.

• The upper layer protocol mapping of FCP in FC-4 is responsible to map SCSI protocol in an FC network.

By Shital H. More

Interoperability of Fibre Channel SAN

• FCP is a software, which is implemented in the form of device drivers.

• The low-level device drivers attract a lot of implementation errors.

• Hence, they need to be tested before deploying them.

By Shital H. More

Interoperability of Fibre Channel SAN

Two difficulties with testing of FCP driver:

• There are many servers sending request to a single storage device & vice-versa.

• Therefore, the end devices in the FC network should be multitasking.

• Testing of device drivers with such a multitasking environment is difficult mainly because of race conditions occurring in the system.

By Shital H. More

Interoperability of Fibre Channel SAN

• The 2nd difficulty is because of the presence of large number of components that come together in an FC SAN.

• There are different possibilities such as OS version, service packs, etc.

• Other hardware components such as HBAs, there are different manufacturers with several different models, different permutations and combinations of hardware & software, etc.

By Shital H. More

Interoperability of Fibre Channel SAN

• Therefore the manufacturers of FC components must perform interoperability test of FC components from various manufacturers to find out the compatibility of their components and third party manufacturer devices.

• Out of different Permutations & combinations, select those which are required by most of the customer.

By Shital H. More

Module 2 Storage Network Architecture

1. SCSI

2. FC Protocol Stack

3. SAN:FC SAN

4. IP Storage

5. Infiniband and Virtual Interfaces

By Shital H. More

IP STORAGE

• IP storage is an approach to build storage networks upon TCP, IP and Ethernet.

• First, introduces various protocols for the transmission of storage data traffic via TCP/IP.

• Then we explain to what extent TCP/IP and Ethernet are suitable transmission techniques for storage networks at all.

• Finally, we discuss a migration path from SCSI and Fibre Channel to IP storage

By Shital H. More

IP storage standards: iSCSI, iFCP, mFCP, FCIP and iSNS • Three protocols are available for transmitting storage

data traffic over TCP/IP:

1. iSCSI,

2. Internet FCP (iFCP) and

3. Fibre Channel over IP (FCIP)

• These are IP-based storage protocols known as IP storage.

By Shital H. More

The basic idea behind iSCSI is to transmit the SCSI protocol over TCP/IP (Figure.4).

Figure.4: Like Fibre Channel, iSCSI replaces the SCSI cable by a

network over which the SCSI protocol is run.

• In contrast to Fibre Channel, TCP/IP and Ethernet are used as the

transmission technology.

By Shital H. More

1. iSCSI

• iSCSI takes a similar approach to FC SAN, the difference is that in iSCSI a TCP/IP/Ethernet connection replaces the SCSI cable.

• Just like Fibre Channel FCP, iSCSI has to be installed in the operating system as a device driver.

• Like FCP, this realises the SCSI protocol and maps the SCSI daisy chain onto a TCP/IP network.

By Shital H. More

2. Internet FCP (iFCP)

• In contrast to iSCSI, which defines a new protocol mapping of SCSI on TCP/IP, iFCP describes the mapping of Fibre Channel FCP on TCP/IP.

• For the realisation of iFCP, LAN switches must either provide a Fibre Channel F-Port or an FL-Port.

• Alternatively, Fibre Channel FCP-to-iFCP gateways could also be used (Figure.5)

By Shital H. More

Figure.5: iFCP is a gateway protocol that connects

Fibre Channel devices via a TCP/IP network.

By Shital H. More

Internet FCP (iFCP)

• The difference between Metro FCP (mFCP) and iFCP is that mFCP is not based upon TCP/IP but on UDP/IP.

• In order to provide fabric services, iFCP/mFCP must evaluate the FC frames received from the end devices and further process these accordingly.

• It forwards data packets for a different end device to the appropriate gateway or switch via TCP/IP.

• It also map infrastructure services of the fabric such as zoning and name service on TCP/IP

By Shital H. More

3. Fibre Channel over IP (FCIP)

• FCIP was designed to remove the distance limitation of FC.

• Organizations need longer distance to be spanned for Backup and Disaster Recovery (DR) purpose.

• The production site and the DR and backup site have to be present in disaster locations to prevent data loss in the event of large-scale catastrophes.

• The conventional way of doing this is the taking Backup using tapes or sending through courier or using WAN techniques.

By Shital H. More

Fibre Channel over IP (FCIP)

• FCIP is an alternative to the conventional WAN techniques.

• It act as a tunnel between two FC islands connected together over a TCP/IP router. (Figure.6).

• FCIP creates point-to-point connection between two FC SANs forming a tunnel.

• It sends all FC frames into TCP/IP packets through this tunnel.

By Shital H. More

Figure.6 :FCIP is a tunnelling protocol that connects

two Fibre Channel SANs over TCP/IP.

By Shital H. More

Fibre Channel over IP (FCIP)

• Another advantage of FCIP is that it provides data encryption.

• IPSec is used to encrypt the data traffic between two IP routers.

• Apart from transmitting SCSI data traffic over IP, there is an additional service needed, which will provide device properties of the components in the IP network.

By Shital H. More

Internet Storage Name Service (iSNS)

• It is a standard which defines a service that allows the end device in the storage network to get properties of a device in IP network.

• iSNS is a Client-Server based application.

• Every client has to register its attribute with the server whenever any device joins or leaves the network.

• This information is updated and clients are informed about changes of the topology.

By Shital H. More

Module 2 Storage Network Architecture

1. SCSI

2. FC Protocol Stack

3. SAN:FC SAN

4. IP Storage

5. Infiniband and Virtual Interfaces

By Shital H. More

Infiniband

• InfiniBand replaces the PCI bus with a serial network (Figure.7).

• In InfiniBand the devices communicate by means of messages, with an InfiniBand switch forwarding the data packets to the receiver.

• The communication is full duplex and a transmission rate of 2.5 Gbit/s in each direction is supported.

• As a medium, InfiniBand defines various copper and fiber-optic cables.

• A maximum length of 17 metres is specified for copper cable and up to 10,000 metres for fiber-optic cable. By Shital H. More

Fig.7: InfiniBand replaces

the PCI bus by a serial

network. By Shital H. More

Infiniband

• The end points in an InfiniBand network are called channel adapters.

• InfiniBand differentiates between Host Channel Adapters (HCAs) and Target Channel Adapters (TCAs).

• HCAs bridge between the InfiniBand network and the system bus to which the CPUs and the main memory (RAM) are connected.

• TCAs make a connection between InfiniBand networks and peripheral devices that are connected via SCSI, Fibre Channel or Ethernet.

By Shital H. More

Infiniband

• We have to consider this: network cards and HBA cards can be located 100 metres apart. This means that mainboards with CPU and memory, network cards, HBA cards and storage devices are all installed individually as physically separate, decoupled devices.

• Figure.8 shows such an interconnection of CPU, memory, I/O cards and storage devices.

• The peripheral devices are connected via infiniBand network.

• CPU & RAM provide computing power for interconnection of the modules

By Shital H. More

By Shital H. More

Virtual Interface Architecture (VIA)

• The Virtual Interface Architecture (VIA) significantly influenced the design of InfiniBand.

• VIA facilitates fast and efficient data exchange between applications that run on different computers.

• For this, the underlying network should have low latency and a low error rate.

• This restricts VIA to be used over shorter distances, may be within a data centre or within a building.

• VIA was launched in 1997 by Compaq, Intel and Microsoft.

By Shital H. More

• Today communication between applications on different networks is complicated.

• Incoming data is accepted by the network card, processed in the kernel of the OS and finally delivered to the application.

• As part of this process, data is copied repeatedly from one buffer to the next.

• All this needs CPU power and places a load upon the system bus.

• As a result the communication throughput is reduced and latency is increased.

By Shital H. More

Virtual Interface Architecture (VIA)

• VIA reduces the complexity by providing a bridge between the application and the network card so that they exchange the data directly with one another, bypassing OS.

• The process is described below:

1. Establish Virtual Interface (VI):

• Setting up connection between two applications.

• This setup includes reserving a common memory area on both computers which will be used by application & local network card to exchange data.

By Shital H. More

Virtual Interface Architecture (VIA)

2. Application fills common memory area with the data to be sent to the first computer.

3. When buffer becomes full with data, the application announces a door bell indicating that there is a data to be sent using send queue of the VI.

4. The VI hardware then reads the data directly from common memory area and then transmit it to the VI hardware present on the second computer.

By Shital H. More

Fig.9 : VI Architecture

By Shital H. More

Summary

• SCSI : Small Computer System Interface

• It is a standard developed by ANSI to connect peripheral devices such as printers, scanners, disk drivers, etc.

• It is a set of parallel interface.

• FC SAN :

• It allows multiple servers to access network storage devices.

• It facilitates high-performance data transmission between multiple storage devices & servers.

By Shital H. More

Summary

• iSCSI : Internet Small Computer System Interface

• It allows SCSI command to be send between two end devices in LAN, WAN or Internet on top of TCP.

• iFCP : Internet Fibre Channel Protocol

• It allows organisation to extend FC storage networks over internet by using TCP/IP.

By Shital H. More

Summary

• InfiniBand (IB) :

• It is industry stds for interconnecting I/O devices .

• It provides highest bandwidth, lowest latency, low CPU overhead.

By Shital H. More