Performance investigation and comparison between …free.eol.cn/edu_net/edudown/cans2015/cans2015pdf/2201/1...virtual network • In idle traffic case, the deviation: 0.23%(single-thread

Performance investigation and comparison

between virtual networks and physical networks

based on Sea-Cloud Innovation Environment

Website: http://scie.ac.cn E-mail: [email protected]

CANS 2015, Chengdu, Sep 21, 2015

2015/9/28

Outline

• Background

• Architecture

• Software & Hardware

• Deployment

• Demonstration

• Performance investigation

Background

• Sea-Cloud Innovation Environment, a national wide

testbed supported by the “Strategic Priority

Research Program - New Information and

Communication Technology”(SPRP-NICT) of the

Chinese Academy of Sciences, is aiming to build

an open, general-purpose, federated and large-

scale shared experimental facility to foster the

emergence of new ICT.

Background

•Providing shared and sliceable experimental facilities for academia and

industry to bridge the gap between visionary research and large-scale

experimentation.

•Establishing and practicing the methodology of experimentally -driven

innovation for the clean-slate architecture of ICT.

•Evaluating and validating new protocols, devices and research

achievements of SPRP-NICT.

Objective

2015/9/28

Outline

• Background

• Architecture


• Deployment

• Demonstration

• Experimentation

Architecture

Experiment Topology Requests • SCIE portal

• Scie.ac.cn

• Resource control

framework

• Experiment

measurement

system

• SDN/VLAN-based

network slicing

2015/9/28

Outline

• Background

• Architecture


• Deployment

• Demonstration


Software--Overview

SCIE Portal

Control Center

Resource Site

Experiment

Measurement

System

Resource

Management &

Control System

Topology

Editor

Experiment

Playground

Resource

Management

Authorization

& Accounting

Site manager

Resource

Control

module

Measurement

Module Site manager…….

Experiment Service System

SoftwareSCIE Resource Control Architecture

• Distributed resource control framework with one control center and many site managers

• Defining resource control interfaces, measurement interfaces to integrate different resource

• Light-weight VM management tool

SoftwareSCIE measurement System

• External VM

measurement without

any plug-in in VMs

• AMQP based control

message &

measurement data

transfer

• sFlow based network

traffic measurement

• MongoDB as Storage

Engine

SoftwareExperiment Service System

• Experiment life cycle Management

• Java & Python based experiment control library

• Topology and experiment process visualization

Hardware

Smart-Flow Switch

• OpenFlow 1.2

• GRE tunnel

• QoS supported

• 24*GE

• 1*10GE

• Four slots

• Line Card & UTM Card

Hardware

SCIE Rack

•Integrated network,

computing and storage

• Built-in site management

module

• Virtualization

• Dynamic scheduling

2015/9/28

Outline

• Background

• Architecture


• Deployment

• Demonstration


Deployment

•Four contries&Seven cities &22 sites

•Data plane via GRE tunnel; Control plane via L3 network

•2234 cores, 1510TB storage, 512TB experimental data

2015/9/28

Outline

• Background

• Architecture


• Deployment

• Demonstration


2015/9/28

Outline

• Background

• Architecture


• Deployment

• Demonstration


• Performance investigation and

comparison between virtual

networks and physical networks

based on an advanced testbed

network

Deployment

• Three cities & 4 sites

• VM based on KVM

• Data transmission via GRE tunnel built

based on OVS

GRE Tunnel

VM1

VM8

VM2

...

VM1

VM8

VM2

...

Beijing, China

Beijing, China

Xinjiang, China

Michigan, US

Deployment

• Scenario– single-thread vs. multi-thread

• For each scenario

– the intra-domain case, from Beijing to Xinjiang in China

– the inter-domain case, from Beijing in China to Michigan in US

• Extensive performance evaluation tests

– UDP and TCP traffic in idle and non-idle period

• Key performance metrics– For UDP traffic

• round trip time (RTT), throughput, packet loss, and jitter

– For TCP traffic

• RTT and throughput

Experimental results and analysis

• UDP traffic

– throughput0

10

20

30

40

50

60

1 21 41 61 81 101 121 141 161 181 201 221 241 261 281 301 321 341

Thro

ugh

pu

t (M

bit

s/se

con

d)

Seconds

VM(single-thread,Idle)

PHY(single-thread,Idle)

VM(multi-thread,Idle)

0

10

20

30

40

50

60

1 21 41 61 81 101 121 141 161 181 201 221 241 261 281 301 321 341

Thro

ugh

pu

t (M

bit

/s)

Seconds

VM(single-thread,non-Idle)

PHY(single-thread,non-Idle)

VM(multi-thread,non-Idle)

2

3

4

5

6

7

8

9

10

1 21 41 61 81 101 121 141 161 181 201 221 241 261 281 301 321 341

Thro

ugh

pu

t (M

bit

/sec

on

d)

Seconds




2

3

4

5

6

7

8

9

10

1 21 41 61 81 101 121 141 161 181 201 221 241 261 281 301 321 341

Thro

ugh

pu

t (M

bit

/sec

on

d)

Seconds




(a) UDP Bandwidth BJ-XJ (Idle) (b) UDP Bandwidth BJ-XJ (Non-Idle)

(c) UDP Bandwidth BJ-MI (Idle) (d) UDP Bandwidth BJ-MI (Non-Idle)

• The virtual network is very similar to single-thread physical network very similar to single-thread physical network scheme in intra-domain

and in inter-domain

the deviation less than 0.35% in intra-domain

about 0.21% in inter-domain

• The deviation is stable (multi-thread virtual network vs. single-thread

physical network)


• UDP traffic

– packet loss rate(a) UDP Packet Loss BJ-XJ (Idle) (b) UDP Packet Loss BJ-XJ (Non-Idle)

(c) UDP Packet Loss BJ-MI (Idle) (d) UDP Packet Loss BJ-MI (Non-Idle)

• Single-thread virtual network and single-thread physical network are

better than multi-thread virtual network scheme in all cases

• The deviation is less than 1% (multi-thread virtual network vs. single-

thread physical network) 0.17% in intra-domain

about 0.23% in inter-domain

0

5

10

15

20

1 21 41 61 81 101 121 141 161 181 201 221 241 261 281 301 321 341

Pa

cket

Lo

st R

ate

(%

)

Seconds




0

5

10

15

20

1 21 41 61 81 101 121 141 161 181 201 221 241 261 281 301 321 341

Pa

cket

Lo

st R

ate

(%

)

Seconds

VM(singel-thread,non-Idle)

PHY(singel-thread,non-Idle)


0

5

10

15

20

1 21 41 61 81 101 121 141 161 181 201 221 241 261 281 301 321 341

Pac

kets

Lo

st R

ate

(%)

Seconds




0

5

10

15

20

1 21 41 61 81 101 121 141 161 181 201 221 241 261 281 301 321 341

Pack

ets

Lost

Rat

e (%

)

Seconds





• UDP traffic

– jitter (a) UDP Jitter BJ-XJ (Idle) (b) UDP Jitter BJ-XJ (Non-Idle)

(c) UDP Jitter BJ-MI (Idle) (d) UDP Jitter BJ-MI (Non-Idle)

• The jitter of inter-domain environment was higher than that of intra-

domain environment with about 13.5%

• Jitter in Multi-thread virtual network is higher than single-thread physical

network 2.6% higher in intra-domain environment

10.2 % higher in inter-domain environment

• The distance is the main factor affecting the jitter

0

0.2

0.4

0.6

0.8

1 21 41 61 81 101 121 141 161 181 201 221 241 261 281 301 321 341

Jitt

er (

ms)

Seconds




0

0.2

0.4

0.6

0.8

1 21 41 61 81 101 121 141 161 181 201 221 241 261 281 301 321 341

Jitt

er (

ms)

Seconds

VM(single-thread,non-Idle)PHY(single-thread, non-Idle)VM(multi-thread, non-Idle)

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1 21 41 61 81 101 121 141 161 181 201 221 241 261 281 301 321 341

Jitt

er (

ms)

Seconds




0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1 21 41 61 81 101 121 141 161 181 201 221 241 261 281 301 321 341

Jitt

er (

ms)

Seconds





• UDP traffic

– RTT(a) a UDP RTT BJ-XJ (Idle) (b) UDP RTT BJ-XJ (Non-Idle)

(c) UDP RTT BJ-MI (Idle) (d) UDP RTT BJ-MI (Non-Idle)

• Single-thread physical network is more stable and smaller than

single-thread and multi-thread virtual network • RTT is most stable and lowest in all cases in single-thread physical network

• The RTT of single-thread physical network is about 3.3% smaller than single-

thread and multi-thread virtual network schemes

• The RTT is not so stable in non-idle scenario, the deviation between

idle and non-idle cases is 0.1%

• The background traffic is the key influence factor of RTT and the

performance of network experiment in virtual network environment

50

55

60

65

70

1 21 41 61 81 101 121 141 161 181 201 221 241 261 281 301 321 341

RTT

(m

s)

Seconds




50

55

60

65

70

1 21 41 61 81 101 121 141 161 181 201 221 241 261 281 301 321 341

RTT

(m

s)

Seconds




200

210

220

230

240

250

260

270

280

1 21 41 61 81 101 121 141 161 181 201 221 241 261 281 301 321 341

RTT

(m

s)

Seconds




200

220

240

260

280

300

320

340

1 21 41 61 81 101 121 141 161 181 201 221 241 261 281 301 321 341

RTT

(m

s)

Seconds





• TCP traffic

– throughput(a) TCP Throughput BJ-XJ (Idle) (b) TCP Throughput BJ-XJ (Non-Idle)

(c) TCP Throughput BJ-MI (Idle) (d) TCP Throughput BJ-MI (Non-Idle)

• Throughput in Idle case is better than non-idle case

• Throughput in intra-domain environment is higher than that in inter-

domain environment

• After slow start, the throughput of single-thread physical network is

higher than that of single-thread and multi-thread virtual network

scheme

• The throughput of single-thread virtual network is the most stable and

is very similar to single-thread physical network in non-idle case

0

50

100

150

200

250

300

1 11 21 31 41 51

Thro

ugh

pu

t (M

bit

/sec

on

d)

Samples




0

50

100

150

200

250

300

1 11 21 31 41 51

Thro

ugh

pu

t (M

bit

/sec

on

d)

Samples




0

20

40

60

80

100

1 11 21 31 41 51

Thro

ugh

pu

t (M

bit

/sec

on

d)

Samples




0

20

40

60

80

100

1 11 21 31 41 51

Thro

ugh

pu

t (M

bit

/sec

on

d

Samples





• TCP traffic

– RTT(a) TCP RTT BJ-XJ (Idle) (b) TCP RTT BJ-XJ (Non-Idle)

(c) TCP RTT BJ-MI (Idle) (d) TCP RTT BJ-MI (Non-Idle)

• Stability of RTT single-thread physical network > single-thread virtual network > multi-thread

virtual network

• In idle traffic case, the deviation: 0.23%(single-thread physical network vs. single-thread virtual network)

0.55%(single-thread physical network vs. multi-thread virtual network)

• In non-idle traffic case, the deviation: 3.7% (single-thread physical network vs. single-thread virtual network)

4.5% (single-thread physical network vs. multi-thread virtual network)

50

52

54

56

58

60

62

64

66

68

70

1 21 41 61 81 101 121 141 161 181 201 221 241 261 281 301 321 341

RTT

(m

s)

Seconds




50

52

54

56

58

60

62

64

66

68

70

1 21 41 61 81 101 121 141 161 181 201 221 241 261 281 301 321 341

RTT

(m

s)

Seconds




220

230

240

250

260

270

280

290

300

1 21 41 61 81 101 121 141 161 181 201 221 241 261 281 301 321 341

RTT

(m

s)

Seconds




220

230

240

250

260

270

280

290

300

1 21 41 61 81 101 121 141 161 181 201 221 241 261 281 301 321 341

RTT

(m

s)

Seconds





• Conclusion• 1) the RTT and jitter of virtual networks have little deviation

from physical networks,

• 2) the throughput and packet loss rate of virtual networks are

similar to physical networks,

• 3) the performance of single-thread virtual network is more

similar to the existing physical network than the multi-thread

virtual networks,

• 4) the multi-thread virtual networks have certain deviation from

physical networks, but the deviation is stable and shows

certain characteristics.

• Thus, it is possible to get the performance

of real physical networks in virtual

networks

Documents

Performance investigation and comparison between …free.eol.cn/edu_net/edudown/cans2015/cans2015pdf/2201/1...virtual network • In idle traffic case, the deviation: 0.23%(single-thread