Advanced Software Engineering - SJTUwang-xb/wireless_new/course... · First install Sun VirtualBox as hypervisor Then, Guest OS will be Ubuntu 12.04.1 52. Installing Sun/Oracle VirtualBox

Advanced Software Engineering

Lecture 8: Data Centerby

Prof. Harold Liu

Agenda

IntroductionDesign and ConstructionManagement and MaintenanceHot Topics

Real‐time traffic analysisSDNWireless Data Center Networking

Core Elements of a Data Center

ApplicationsDatabases – Database Management System (DBMS) and the physical and logical storage of dataServers/Operating SystemsNetworksStorage Arrays

3

An Example

Consider an order processing system consisting of:Application for order entry.Database Management System (DBMS) to store customer and product information.Server/Operating System on which the Application and Database programs are run.Networks that provide

Connectivity between Clients and the Application/Database ServerConnectivity between the Server and the Storage system.

Storage Array

Local AreaNetwork

Storage Area Network

Storage Array

Client

Server

Application User

InterfaceDatabase

An Example ..Closer Look

A customer order is entered via the Application User Interface on a client.

Local AreaNetwork


ClientServer

Application User

Interface

Storage Array


A customer order is entered via the Application User Interface on a clientThe client accesses the Server over a Local Area Network.


ClientServer

Storage Array

Local AreaNetwork


A DBMS uses the operating system on the server to read and write this data to the physical location on a disk.


ClientServer

O/S and DBMS

Storage Array

Local AreaNetwork


A DBMS uses the operating system on the server to read and write this data to the physical location on disk.A Network provides the communication link between the server and the storage array, and transports the read/write commands and data between the server and the storage array.


ClientServer

Storage Array

Local AreaNetwork


A DBMS uses the operating system on the server to read and writethis data to the physical location on disk.A Network provides the communication link between the client andthe server, and transports the read/write commands and data between the server and the storage array.A storage array receives the read/write commands and data from the server and performs the necessary operations to store the data on the physical disks.


ClientServer

Storage Array

Local AreaNetwork

Database

Google Data Center

Google位于俄勒冈州的Dalles数据中心

at Lenoir, North Carolinahttp://www.google.com/about/datacenters/inside/streetview/

Galleryhttp://www.google.cn/about/datacenters/gallery/#/all

IBM Data Center

Equinix Data Center

Elements of a Data CenterThe SiteCommand CenterCable ManagementNetwork InfrastructureTerminal ServersEnvironmental ControlsPower

Standards ‐ 1

Standard ‐ 2

Structural Aspects

Structural Aspectsa raised floor ceilingBasement data center locations near water are not a good idea.Their must be a pathway for equipment to be moved in & out of the data center.Make sure the floor where the data center is to be located is rated for the estimated load.

Power and EnvironmentDual street power supply with UPSWhen necessary, a diesel generator as a second backup

Environmental Control FacilitiesTemperatureMoisturedust

HVAC (Heating, Ventilation, and Air Conditioning)

Hot‐Aisle and Cold‐Aisle Layout

Networking3‐layer architectureCore Layer Switch/RouterLayer‐2 SwitchInternal NIC Ethernet connection

DLink 48ports Gbps Layer-2 Switch

Cisco Core Layer Router

SecurityPhysical AccessLevels of AccessMonitoring

Past to Future

a single computer occupied the space of an entire Data Center. That same space can be occupied by thousands of servers today.

Let us Make it Online

HWSWRackDeployment and TestingMaintenance

ServersTower

Low cost, customizedSpace cost, not for parallel computing

Blade• save space, high computing ability• suitable for large‐size data center

Rack Server• small, easy to manage• need good cooling

Rack Unit (RU)Unit=44.45mm (height)Width=19‐inch or 23‐inch1U, 2U, 4U (half rack)

1U4U

Server Rack and Cluster

Data Center TopologyInternet

Servers

Layer-2 switchAccess

Data Center

Layer-2/3 switchAggregation

Layer-3 routerCore

Top of Rack Switch

Top of Rack (ToR)Switch

Data Center CostsJames Hamilton published basic 2008 breakdown

Servers: 45%CPU, memory, diskInfrastructure: 25%UPS, cooling, power distributionPower draw: 15%Electrical utility costsNetwork: 15%Switches, links, transit

Power CostApproximate distribution of peak power usage by hardware subsystem in one of Google’s datacenters (circa 2007).

Data Center ChallengesTraffic load balanceSupport for VM migrationAchieving bisection bandwidthPower savings / CoolingNetwork management (provisioning)Security (dealing with multiple tenants)

Non‐Virtualized Data CentersToo many servers for too little work

High costs and infrastructure needsMaintenanceNetworkingFloor spaceCoolingPowerDisaster Recovery

What is Virtualization?Run multiple OSes and user applications on the same hardware

e.g., run both Windows and Linux on the same laptopHow is it different from dual‐boot?

Both OSes run simultaneouslyOSes are completely isolated from each other

Reduce costs by consolidating services onto the fewest number of physical machines

Dynamic Data CenterVirtualization helps us break the “one service per server” modelConsolidate many services into a fewer number of machines when workload is low, reducing costsConversely, as demand for a particular service increases, we can shift more virtual machines to run that serviceWe can build a data center with fewer total resources, since resources are used as needed instead of being dedicated to single services

VM Workload Multiplexing

Multiplex VMs’ workload on same physical server

Separate VM sizing

VM multiplexing

s1s2

s3

We expect s3 < s1 + s2. Benefit of multiplexing !

So, it is just like Java VM, right?

Two Types of Hypervisors (or VMM)Hypervisor is a software layer that allows several VMs to runon a physical machineThe physical OS and hardware are called the HostVM OS and applications are called the Guest

VMware ESX, Microsoft Hyper‐V, Xen

Hardware

Hypervisor

VM1 VM2

Type 1 (bare‐metal)

Host

Guest

Hardware

OS

Process Hypervisor

VM1 VM2

Type 2 (hosted)

VMware Workstation, Microsoft Virtual PC, Sun VirtualBox, QEMU, KVM

Host

GuestProcess Process

Process Process

Bare‐metal or Hosted?Bare‐metal

Has complete control over hardwareDoesn’t have to “fight” an OS

HostedAvoid code duplication: need not code a process scheduler, memory management system – the OS already does thatCan run native processes alongside VMsFamiliar environment – how much CPU and memory does a VM take? Use top! How big is the virtual disk? ls –lEasy management – stop a VM? Sure, just kill it!

A combinationMostly hosted, but some parts are inside the OS kernel for performance reasons, e.g., KVM

5151

VM on Multi‐core CPUsEach core can be configured for multiple VMs

A Quad‐core CPU could be configured as a 32 node multi‐computerLimiting factor is often memory. Each guest OS has its own requirements (512 MB?)

Installing a Virtual machineBase OS is Windows 7

First install Sun VirtualBox as hypervisorThen, Guest OS will be Ubuntu 12.04.1

52

Installing Sun/Oracle VirtualBox

53

Installing VirtualBox

54


55


56

Installing Ubuntu VM

57


58


59


60

61


62

63


64

Data Center TopologyInternet

Servers

Layer-2 switchAccess

Data Center

Layer-2/3 switchAggregation

Layer-3 routerCore

Top of Rack Switch

Oversubscription

MotivationFlow‐based traffic monitoringVolume of processed data is reducedPopular flow statistics tools

Cisco NetFlowsFlow

What is sFlow?Accurate sampling is simple enough to be performed in hardware, at wire speed. Good accuracy under heavy loads Detailed complete packet header and switching/routing information for L2‐L7 traffic flows.Scalable capable of monitoring networks at 10Gbps, 100Gbps and beyond. Thousands of devices can be monitored by a single sFlow Collector.Low Cost sFlow Agent is very simple to implement and adds negligible cost to a switch or router.Timely an up to the minute view, for real‐time controls. QoS and DoS attack.

sFlow Architecture

SwitchingASIC

1 in N sampling

sFlow in Operation

packet header src/dst i/f sampling parms forwarding user ID URL i/f counterssFlow agent

forwarding tables

interface counters

sFlow Datagram

eg 128B ratepool

src 802.1p/Qdst 802.1p/Qnext hopsrc/dst maskAS pathcommunitieslocalPref

src/dstRadiusTACACS

sFlow Collector & Analyzer

Switch/Router

Statistical Model for Packet Sampling

NncNc⋅=

Total number of frames = NTotal number of samples = nNumber of samples in class = cNumber of frames in the class estimated by:

Relative Sampling Error

0%

25%

50%

75%

100%

1 10 100 1000 10000

Number of Samples in Class

% E

rror

c%error 1196 ⋅≤

Estimating Traffic per Protocol

Even so, Data Volume is Huge!

Short‐term period of flow dataMassive flow data from anomaly traffic data of Internet worm and DDoS

Cluster file system and cloud computing platformGoogle’s programming model, MapReduce, big tableOpen‐source system, Hadoop

Flow data in our campus network ( /16 prefix )# of Routers 1 Day 1 Month 1 Year

1 1.2 GB 13 GB 156 GB5 6 GB 65 GB 780 GB

10 12 GB 130 GB 1.5 TB200 240 GB 2.6 TB 30 TB

73

An Experiment: Travel Booking Website2 BNT switches

Traditional Flow‐based Traffic Monitoring

Run on a high performance central server

Flow Data

Routers

High Performance Server

Storage

75

Occupancy of Different Network Service Types

Heavy Hitters Workload

Offline Spike Analysis

Google System Health

Software Defined Networking (SDN)What is SDN?

BackgroundAn OS for networks

What is OpenFlow?How it helps SDN

The current status & the future of SDNConclusions

79

Limitations of Current Networks

80

Switches

Million of linesof source code

5400 RFCs Barrier to entry

500M gates10Gbytes RAM

Bloated Power Hungry

Many complex functions baked into the infrastructureOSPF, BGP, multicast, differentiated services,Traffic Engineering, NAT, firewalls, MPLS, redundant layers, …

An industry with a “mainframe‐mentality”

We have lost our way

Specialized Packet Forwarding Hardware

OperatingSystem

App App App

Routing, management, mobility management, access control, VPNs, …

Operating SystemOperating System

Reality

AppApp

App

Specialized Packet Forwarding HardwareSpecialized Packet

Forwarding HardwareSpecialized Packet

Forwarding Hardware

OperatingSystem

App App App

• Lack of competition means glacial innovation• Closed architecture means blurry, closed interfaces• Vertically integrated, complex, closed, proprietary• Not suitable for experimental ideas• Not good for network owners & users• Not good for researchers

Glacial process of innovation made worse by captive standards process

DeploymentIdea Standardize

Wait 10 years

• Driven by vendors• Consumers largely locked out• Lowest common denominator features• Glacial innovation

Old Ways to Configure a Network


App App App


App App App


App App App


App App App


OperatingSystem

OperatingSystem

OperatingSystem

OperatingSystem

OperatingSystem

App App App

84

No control plane abstraction for the whole network! It’s like old times – when there was no OS…

Wilkes with the EDSAC, 1949

85

Idea: An OS for Networks!!!


App App App


App App App


App App App


App App App


OperatingSystem

OperatingSystem

OperatingSystem

OperatingSystem

OperatingSystem

App App App

Closed

Idea: An OS for Networks


App App App


App App App


App App App


App App App


OperatingSystem

OperatingSystem

OperatingSystem

OperatingSystem

OperatingSystem

App App App

Network Operating System

Control Programs

87

Idea: An OS for Networks

Simple Packet Forwarding Hardware



Simple Packet Forwarding Hardware Simple Packet

Forwarding Hardware


Control Programs

88

Idea: An OS for Networks“NOX: Towards an Operating System for Networks”

Global Network View

Protocols Protocols

Control via forwarding interface


Control Programs

Software‐Defined Networking (SDN)

89

Windows(OS)Windows(OS)

Linux MacOS

x86(Computer)

Windows(OS)

AppApp

LinuxLinuxMacOSMacOS

Virtualization layer

App

Controller 1

AppApp

Controller2

Virtualization or “Slicing”

App

OpenFlow

Controller 1NOX(Network OS)

Controller2Network OS

Trend

Computer Industry Network Industry

Outline

What is SDN?Limitations of current networksThe idea of Network OS


The current status & the future of SDNConclusions 92

OpenFlow“OpenFlow: Enabling Innovation in Campus Networks”

Like hardware drivers – interface between switches and Network OS

93

OpenFlow

94

Data Path (Hardware)Data Path (Hardware)

Control Path (Software)Control Path (Software)

OpenFlow

95Data Path (Hardware)Data Path (Hardware)

Control PathControl Path OpenFlowOpenFlow

OpenFlowOpenFlow ControllerController

OpenFlow Protocol (SSL/TCP)

OpenFlow ProtocolSSL‐TCP

Network OS

Control Program A Control Program B

OpenFlow Basics

IBM 10 gigabit ethernet OpenFlow switch G8264, which has 48×10 GbE SFP+ ports and 4 × 40 GbE QSFP+ ports

OpenFlow Switching

97

Controller

PCHardwareLayer

SoftwareLayer

OpenFlow TableMACsrc

MACdst

IPSrc

IPDst

TCPsport

TCPdport Action

OpenFlow Client

**5.6.7.8*** port 1

port 4port 3port 2port 1

1.2.3.45.6.7.8

Control Program A Control Program B

Network OS

Flow Table

PacketForwarding Packet

Forwarding


Forwarding


Forwarding

FlowTable(s)Flow

Table(s)

“If header = p, send to port 4”

“If header = ?, send to me”

“If header = q, overwrite header with r, add header s, and send to ports 5,6”

Flow Table

Rule(exact & wildcard) Action Statistics



Rule(exact & wildcard) Default Action Statistics

Flow 1.

Flow 2.

Flow 3.

Flow N.

Flow EntryMatch fields

Match against packetsAction

Modify the action set or pipeline processingStats

Update the matching packets

Match Fields StatsAction

In Port SrcMAC

DstMAC

Eth Type Vlan Id IP Tos IP

Proto IP Src IP Dst TCP SrcPort

TCP DstPort

Layer 2 Layer 3 Layer 4

1. Forward packet to port(s)2. Encapsulate and forward to controller3. Drop packet 4. Send to normal processing pipeline

1. Forward packet to port(s)2. Encapsulate and forward to controller3. Drop packet 4. Send to normal processing pipeline

1. Packet2. Byte counters1. Packet2. Byte counters

ExamplesSwitching

*

SwitchPort

MACsrc

MACdst

Ethtype

VLANID

IPSrc

IPDst

IPProt

TCPsport

TCPdport Action

* 00:1f:.. * * * * * * * port6

Flow Switching

port3

SwitchPort

MACsrc

MACdst

Ethtype

VLANID

IPSrc

IPDst

IPProt

TCPsport

TCPdport Action

00:20.. 00:1f.. 0800 vlan1 1.2.3.4 5.6.7.8 4 17264 80 port6

Firewall

*

SwitchPort

MACsrc

MACdst

Ethtype

VLANID

IPSrc

IPDst

IPProt

TCPsport

TCPdport Action

* * * * * * * * 22 drop

ExamplesRouting

*

SwitchPort

MACsrc

MACdst

Ethtype

VLANID

IPSrc

IPDst

IPProt

TCPsport

TCPdport Action

* * * * * 5.6.7.8 * * * port6

VLAN Switching

*

SwitchPort

MACsrc

MACdst

Ethtype

VLANID

IPSrc

IPDst

IPProt

TCPsport

TCPdport Action

* * vlan1 * * * * *port6, port7,port9

00:1f..

OpenFlow UsageController

PC

OpenFlowSwitch

OpenFlowSwitch

OpenFlowSwitch

Alice’s codeAlice’s code

Decision?OpenFlowProtocol

Alice’s Rule

Alice’s Rule

Alice’s Rule

Alice’s Rule

Alice’s Rule

Alice’s Rule

103

OpenFlow UsageController

PC

Alice’s codeAlice’s code

104

» Alice’s code:˃ Simple learning switch ˃ Per Flow switching˃ Network access control/firewall

˃ Static “VLANs”˃ Her own new routing protocol: unicast, multicast, multipath

˃ Home network manager˃ Packet processor (in controller)˃ IPvAlice

OpenFlowStandard way to control flow‐tables in commercial switches and routers

Just need to update firmware

Essential to the implementation of SDN

105

Centralized/Distributed Control

Centralized Control

OpenFlowSwitch

OpenFlowSwitch

OpenFlowSwitch

Controller

Distributed Control

OpenFlowSwitch

OpenFlowSwitch

OpenFlowSwitch

Controller

Controller

“Onix: A Distributed Control Platform for Large‐scale Production Networks”, UNIX OSDI 2010.

Virtualizing OpenFlowNetwork operators “Delegate” control of subsets of network hardware and/or traffic to other network operators or usersMultiple controllers can talk to the same set of switchesImagine a hypervisor for network equipmentsAllow experiments to be run on the network in isolation of each other and production traffic

Switch Based Virtualization

Normal L2/L3 Processing

Flow Table

Production VLANs

Research VLAN 1

ControllerResearch VLAN 2

Flow Table

Controller

108

FlowVisorA network hypervisor developed by StanfordA software proxy between the forwarding and control planes of network devices

FlowVisor‐based Virtualization

OpenFlowSwitch

OpenFlowProtocolOpenFlowProtocol

OpenFlow FlowVisor& Policy Control

Craig’sController

Heidi’sControllerAaron’s

Controller

OpenFlowSwitch

OpenFlowSwitch

110

Topology discovery is per slice

OpenFlowProtocolOpenFlowProtocol

OutlineOutlineWhat is SDN?

Limitations of current networksThe idea of Network OS


The current status & the future of SDN

Conclusions111

OpenFlow Building Blocks

ControllerNOXNOX

SlicingSoftwareFlowVisorFlowVisor

FlowVisorConsole

112

ApplicationsLAVILAVIENVI (GUI)ENVI (GUI) ExpedientExpedientn‐Castingn‐Casting

NetFPGANetFPGASoftware Ref. SwitchSoftware Ref. Switch

BroadcomRef. SwitchBroadcomRef. Switch

OpenWRTOpenWRT PCEngineWiFi APPCEngineWiFi AP

Commercial Switches Stanford Provided

OpenFlowSwitches

ONIXONIX

Stanford Provided

Monitoring/debugging toolsoflopsoflopsoftraceoftrace openseeropenseer

Open vSwitchOpen vSwitch

HP, NEC, Pronto, Juniper.. and many more

HP, NEC, Pronto, Juniper.. and many more

BeaconBeacon TremaTrema MaestroMaestro

Current status of SDNHardware support

113

Ciena Coredirector

NEC IP8800

More coming soon...

Juniper MX‐series

HP Procurve 5400

Pronto 3240/3290

WiMax (NEC)

PC EnginesNetgear 7324

Current status of SDNIndustry support

Google built hardware and software based on the OpenFlow protocolVMware purchased Nicira for $1.26 billion in 2012IBM, HP, NEC, Cisco and Juniper also are offering SDNs that may incorporate OpenFlow, but also have other elements that are specific to that vendor and their gear.

Future Focuses of SDNNew policies for securityProgrammable WLANsThe placement of controllers (amount; location; centralized/distributed)Debugger for SDN

115

ConclusionsWhat is SDN?

A system‐layered abstractionProgrammable, flexible, and extensible

What is OpenFlow?Interface between switches and controllersEnabling SDN

Future SDNEnabling innovation

116

Wireless Data Center Networking60 GHz spectrum 7 GHz (57–64 GHz) wavebanddata rate over 1 Gb/s10 metersLine‐of‐Sight (LoS)

Documents

Advanced Software Engineering - SJTUwang-xb/wireless_new/course... · First install Sun VirtualBox as hypervisor Then, Guest OS will be Ubuntu 12.04.1 52. Installing Sun/Oracle VirtualBox