18
ONOS Controller Performance Test Report 0 ONOS Controller Performance Test Report Global SDN Certified Testing CenterSDNCTC 201674 www.sdnctc.com

ONOS Controller Performance Test Report - · PDF fileONOS Controller Performance Test Report 3 2. Turn on test tool OFsuite_Performance, use “--switches=[N]” option to set a certain

Embed Size (px)

Citation preview

Page 1: ONOS Controller Performance Test Report - · PDF fileONOS Controller Performance Test Report 3 2. Turn on test tool OFsuite_Performance, use “--switches=[N]” option to set a certain

ONOS Controller Performance Test Report 0

ONOS Controller Performance

Test Report

Global SDN Certified Testing Center,SDNCTC

2016.7.4

w w w . s d n c t c . c o m

Page 2: ONOS Controller Performance Test Report - · PDF fileONOS Controller Performance Test Report 3 2. Turn on test tool OFsuite_Performance, use “--switches=[N]” option to set a certain

ONOS Controller Performance Test Report 0

www.sdnctc.com

CONTENTS

1. INTRODUCTION ........................................................................................................................................................ 1

2. TEST ENVIRONMENT CONFIGURATION ..................................................................................................................... 1

2.1 CONTROLLERS UNDER TEST ................................................................................................................................................... 1

2.2 SERVER CONFIGURATION ....................................................................................................................................................... 2

2.3 TEST TOOL ......................................................................................................................................................................... 2

3. TEST ITEM AND TEST RESULT .................................................................................................................................... 2

3.1 CONTROL CHANNEL CAPACITY TEST ......................................................................................................................................... 2

3.2 TOPOLOGY DISCOVERY TIME TEST ........................................................................................................................................... 4

3.3 PACKET_OUT SENDING RATE TEST ....................................................................................................................................... 6

3.4 FLOW MOD SENDING RATE TEST ....................................................................................................................................... 10

3.5 END-END LINK ESTABLISHMENT TIME TEST ............................................................................................................................. 13

3.6 ROLE ELECTION TIME TEST .................................................................................................................................................. 15

4. GLOBAL SDN CERTIFIED TESTING CENTER ................................................................................................................... 16

Page 3: ONOS Controller Performance Test Report - · PDF fileONOS Controller Performance Test Report 3 2. Turn on test tool OFsuite_Performance, use “--switches=[N]” option to set a certain

ONOS Controller Performance Test Report 1

www.sdnctc.com

1. INTRODUCTION

As Software Defined Network (SDN) gradually becomes the new normal form in network world, the separation of

transition and control makes the data plane become the simple data engine, while control plane assumes all logic

and operation tasks. As the core component of control plane, the performance of SDN controller relates to the

performance of the whole SDN network. With the increase of SDN business deploy speed, the performance of SDN

controller will become the concerned focus of network users increasingly.

OFsuite_Performance is one of the test tools of OFsuite series independently developed by Global SDN Certified

Testing Center (SDNCTC), and devotes to the performance test of OpenFlow controller, it can simulate plenty of

OpenFlow 1.3 switches on the general purpose Linux server, and various network topologies and all OpenFlow

events. It has precise record and statistical functions, so it can effectively measure the processing capacity of

controllers to OpenFlow information. This test tool is brief, efficient and easy to use, and keeps updating in order to

provide more plentiful performance test cases and test scenarios for users.

The results of performance test of SDN controller can provide reliable data support for users to do SDN network

performance evaluation, model selection and business deploy. Global SDN Certified Testing Center leads the

development and compiling of test standards in ONF and other standard structures in a long run, the detailed test

Methodologies of all test cases in this report refer to the coming performance test specification of ONF, IETF and

other standard organizations.

This report takes ONOS controller as the controller under test, uses OFsuite_Performance to execute the test, and

summarizes the results to get the performance test report. All test cases are automatically completed by

OFsuite_Performance. In the large-scale network deploy, all controllers have the demand of clustered running, so

except the performance of single-point running of controller, the performance test results of ONOS controller in

cluster mode are also contained in this report.

2. TEST ENVIRONMENT CONFIGURATION

2.1 CONTROLLERS UNDER TEST

Controller under test are ONOS controllers, and edition is the 1.5.1 edition published on April. 20, 2016.

Page 4: ONOS Controller Performance Test Report - · PDF fileONOS Controller Performance Test Report 3 2. Turn on test tool OFsuite_Performance, use “--switches=[N]” option to set a certain

ONOS Controller Performance Test Report 2

www.sdnctc.com

2.2 SERVER CONFIGURATION

The ONOS controller under test solely runs on the server, and the server configurations are as the following:

Server model: Dell PowerEdge R720

CPU: Intel(R) Xeon(R) CPU E5-2609 v2 @ 2.50GHz quad-core

RAM: 16GB 1600MHz

Operating system: Ubuntu server 14.04.1 LTS 64bit

Network card: 1Gbps

Controller cluster:

The controller cluster environment of all tests referring to cluster mode in this test report is composed of three

above-mentioned servers with same configuration.

2.3 TEST TOOL

The test tool is OFsuite_Performance, when executing test, OFsuite_Performance runs on a server, which is directly

connected to an ONOS controller server (Standalone Mode), or respectively uses three network cards to directly

connect to three ONOS controller cluster servers (cluster mode). The server configurations of this test tool are as the

following:

CPU: Intel(R) Xeon(R) E3-1230 @ 3.20GHz quad-core

RAM: 8GB 1333MHz

Operating system: Ubuntu server 12.04 LTS 64bit

Network card: 1Gbps

3. TEST ITEM AND TEST RESULT

3.1 CONTROL CHANNEL CAPACITY TEST

Test Purpose

Test the memory usage of controller under test when maintaining a certain amount of OpenFlow 1.3 control channel

connections.

Test Methodology

In this test, the test tool simulates a certain amount of OpenFlow 1.3 switches and connects to controller under test,

and record the memory usage of the controller under test, the specific test procedures are as the following:

1. Turn on ONOS controller under test and install drivers, openflow, proxyarp and fwd modules;

Page 5: ONOS Controller Performance Test Report - · PDF fileONOS Controller Performance Test Report 3 2. Turn on test tool OFsuite_Performance, use “--switches=[N]” option to set a certain

ONOS Controller Performance Test Report 3

www.sdnctc.com

2. Turn on test tool OFsuite_Performance, use “--switches=[N]” option to set a certain amount of OpenFlow 1.3

switches and connect them to the controller under test;

3. Record the memory usage conditions of controller after the connection is stable;

4. Use add-sw command to increase the quantity of switches and record the memory usage conditions of

controller after the connection is stable;

5. Execute test with various switch quantities to get the final result.

Test tool requirements

1. Test tool can simulate plenty of OpenFlow 1.3 switches and connect with controllers;

2. Test tool can respond to all related OpenFlow 1.3 protocol messages (Hello, Echo, feature_request, etc);

3. Test tool can monitor control channel activity at any time;

4. Test tool can adjust the quantity of connected switches neatly;

5. Test tool has thorough Log function.

Test Results

Standalone Mode

Standalone Mode means that only one ONOS controller entity is turned on to execute test, the same below.

Results can be seen in the following figure:

In this test case, the ONOS is set up to start applications, i.e. drivers, openflow, proxyarp and fwd. Different numbers

of switches and switch topology structures are set up to test the memory usage of the controllers that are

maintaining the connection of all the switches, as shown in the graph above.

In the graph above, the memory usage of the ONOS controllers with different switch nodes. The more switches the

ONOS connects, the more memory it will take.

Cluster Mode

580622

644675 686

582623

654675

699

583625

646675 675

0

100

200

300

400

500

600

700

800

100 300 500 800 1000

Usa

ge

(MB

)

Number of Switches

Physical Memory Usage

linear ring tree

Page 6: ONOS Controller Performance Test Report - · PDF fileONOS Controller Performance Test Report 3 2. Turn on test tool OFsuite_Performance, use “--switches=[N]” option to set a certain

ONOS Controller Performance Test Report 4

www.sdnctc.com

In the cluster mode, three ONOS controllers can simultaneously connect 1000 switches. When the number of

switches is over 300, if the topological structure is set, the control channel may be disconnected, therefore, no

topological structure is set in the cluster mode. The memory usage of every controller with different numbers of

nodes in the cluster, as shown in the graph above. From the results of the test, it can be seen that there is little

difference between the controllers in terms of memory usage in the cluster mode. Compared to the single point

mode, the ONOS controllers in the cluster mode take more memory space.

3.2 TOPOLOGY DISCOVERY TIME TEST

Test Purpose

Test the discovery time of controllers for different quantities of switches and topological structures with different

types.

Test Methodology

In this test, test tool will simulate a certain amount of switches, and the switches connect to each other to form a

certain network topological structure, test the time of the used by the controller to completely discovery this

topological structure, and the specific test procedures are the following:

1. Turn on the controller under test;

2. Turn on test tool, only set the quantity of switch, and topological structure is not set when turning on;

3. When the connection of switch and controller is stable, use set-topo command to set topology (predefined

linear/ring/full-mesh/leaf-spine structure is supported, topological structure can also be set by users);

4. Test tool records the time when the controller sends the first LLDP message, and responds to the information

according to the contents of flow table;

5. Test tool records the time when the last LLDP message is sent to the controller;

6. Use show-result command to view test result;

7. Alter the quantity of switches and set same topological structure to proceed iteration test;

8. Alter topological structure while maintain the quantity of switches to proceed iteration test;

678704

751

826864

702 724771

810

876

670703

768

846892

0

100

200

300

400

500

600

700

800

900

1000

100 300 500 800 1000

Usa

ge

(MB

)

Number of Switches

Physical Memory Usage

controller1 controller2 controller3

Page 7: ONOS Controller Performance Test Report - · PDF fileONOS Controller Performance Test Report 3 2. Turn on test tool OFsuite_Performance, use “--switches=[N]” option to set a certain

ONOS Controller Performance Test Report 5

www.sdnctc.com

9. Proceed five times of tests to get the average test result.

Test Tool Requirements

1 Test tool can simulate plenty of OpenFlow 1.3 switches, and connect to controllers;

2 Test tool can respond to all related OpenFlow 1.3 protocol messages (Hello, Echo, feature_request etc);

3 Test tool can monitor control channel activity at any time;

4 Test tool can create various network topology structure;

5 Test tool can analyze Packet_out message in order to extract the LLDP message in it;

6 Test tool can analyze LLDP topology discovery message in order to give the relevant response;

7 Test tool has through Log function.

Test Result

Standalone Mode

The more the switches are, the longer it will take to find the topology of the tested controllers. On condition of the

same number of switches, the more the links of topology are, the longer it will take to find the correspondent

topology. Generally, the time spent on finding the topology is in a positive linear correlation with the complexity of

the links.

Cluster Mode

209.83462

780.85

1319.45

2166.59

255.44

490.6

791.76

1350.7

2617.59

233.55

626.99

972.3

1701.31

2792.42

0

500

1000

1500

2000

2500

3000

100 300 500 800 1000

Tim

e(m

s)

Number of Switches

Topology Discovery Time

linear ring tree

Page 8: ONOS Controller Performance Test Report - · PDF fileONOS Controller Performance Test Report 3 2. Turn on test tool OFsuite_Performance, use “--switches=[N]” option to set a certain

ONOS Controller Performance Test Report 6

www.sdnctc.com

In the cluster mode, with the same number of switches and the same topological structures, the time spent on

finding the topology is longer than that in the single point mode. In addition, when the number of switches is larger

than 300, the control channel is disconnected, when it cannot be statistically considered as effective results.

3.3 PACKET_OUT SENDING RATE TEST

Test Purpose

Test the delay and maximum rate that controller under test sends Packet_out message.

Test Methodology

In the test, test tool will simulate aswitch which is connected to the controller, test tool will send Packet_in message

with a certain constant rate, and ARP_requests are carried in the message. Record the time used by sending

Packet_in message. When receiving this Packet_in, the controller will send Packet_out message down, and test tool

will record the time of sending Packet_out down after receiving Packet_out message. Calculate the response speed

(latency) of controller under test for Packet_in message and sending rate (throughput) of Packet_out through all

Packet_in and Packet_out message records. OFsuite_Performance test tool can reach Packet_in sending rate (TCP

protocol itself is the limit, such as the size of receiving window and others) with constant speed of 500K level per

second under the hardware configuration that listed in Chapter 2.3 of this text. The specific test procedures are as

the following:

1. Turn on the controller under test;

2. Turn on test tool, set aswitch, do not set the topological structure, and wait the switch connects to the

controller;

3. Use set-arp-rate command to set the rate of switch to send Packet_in on the command line port of test tool;

4. Wait the test tool to complete the test;

5. Use how-result command to view test result;

6. Alter the rate to send Packet_in, and test it again;

7. Do the five iteration tests and gain the average test result.

1552.3

5236.43

2282.64

7456.05

3781.41

8663.34

0

1000

2000

3000

4000

5000

6000

7000

8000

9000

10000

100 300

Tim

e(m

s)

Number of Switches

Topology Discovery Time

linear ring tree

Page 9: ONOS Controller Performance Test Report - · PDF fileONOS Controller Performance Test Report 3 2. Turn on test tool OFsuite_Performance, use “--switches=[N]” option to set a certain

ONOS Controller Performance Test Report 7

www.sdnctc.com

Test Tool Requirements

1. Test tool can simulate OpenFlow 1.3 switch, and connect to controller;

2. Test tool can respond all related OpenFlow1.3 protocol message(Hello, Echo, feature_request etc);

3. Test tool can send Packet_in message containing ARP_request in order to trigger the controller under test to

send Packet_out message;

4. Test tool can define the contents of Data field of Packet_in message by users;

5. Test tool can send Packet_in message in constant speed;

6. Test tool can analyze Packet_out message in order to confirm that this message is triggered by relevant

Packet_in message;

7. Test tool has thorough Log function.

Test Result

Standalone Mode

It can be seen from the test results of the above figure that when the Packet_in rate set by test tool is not more than

8000 packets/s, ONOS controller can treat the received Packet_in message better, Packet_out sending rate and

Packet_in rate of test tool can be the same; when the Packet_in rate set by test tool is more than 8000 packets/s, it

can be seen from the test results that Packet_out sending rate cannot be the same as Packet_in rate, meanwhile the

delay increases accordingly (see figure below). By this token, the test has reached maximum Packet_out sending rate

of the controller under test.

2000

4000

6000

80008238

7084

0

1000

2000

3000

4000

5000

6000

7000

8000

9000

2000 4000 6000 8000 9000 10000

Pa

cke

t_o

ut R

ate

: P

acke

ts/s

Packet_in Rate : Packets/s

Packet_out Response Rate

Packet_out rate(standalone mode)

Page 10: ONOS Controller Performance Test Report - · PDF fileONOS Controller Performance Test Report 3 2. Turn on test tool OFsuite_Performance, use “--switches=[N]” option to set a certain

ONOS Controller Performance Test Report 8

www.sdnctc.com

What the figure above shows the delay that the controller under test sends Packet_out message, which is the delay

between test tool sending Packet_in and controller under test sending relevant Packet_out back. It can be seen from

the figure that with the increase of sending speed of Packet_in message, the delay of Packet_out message will be

bigger and bigger. The detailed test result data are shown in the following table.

A B C D E

1 packet_in rate minimum latency

(ms)

maximum

latency (ms)

average latency

(ms) packet_out rate

2 2000 0.23096 148.24097 9.63682 2000

3 4000 0.18506 152.32593 27.14163 4000

4 6000 0.17603 164.05591 22.9223 6000

5 8000 0.16992 126.63086 47.57525 8000

6 9000 72.46704 274.93018 234.59139 8238

7 10000 122.35913 865.40283 506.15687 7084

The packet_out rate in the above table means the when there is no packet loss, keep a record of the received

packet_out messages and calculate the average speed instead of the number of the actually received packet_out

messages. The results of the tests in the cluster mode are calculated in the same method.

Cluster Mode

In this test, three servers are respectively used to run the ONOS controller to form the controller cluster. In cluster

mode, each Packet_in message is sent, and then a plurality of the same Packet_out messages will be sent down, test

tool only records the first Packet_out message sent back by controller in order to calculate to get sending rate and

delay of Packet_out, which is shown as the following figure.

0

100

200

300

400

500

600

700

800

900

1000

2000 4000 6000 8000 9000 10000

La

ten

cy(m

s)

Packet_in rate : Packets/s

Packet_out Response Latency

minimun latency (ms) maximum latency (ms) average latency (ms)

Page 11: ONOS Controller Performance Test Report - · PDF fileONOS Controller Performance Test Report 3 2. Turn on test tool OFsuite_Performance, use “--switches=[N]” option to set a certain

ONOS Controller Performance Test Report 9

www.sdnctc.com

The figure above shows the test result to the rate that ONOS controller sending Packet_out in cluster mode. It can

be seen that when Packet_in rate set by test tool is more than 1000 packets/s, the Packet_out rate is lower than the

Packet_in rate, and the processing delay will rapidly increase. Comparing to the test result of Standalone Mode

above, it can be seen that the performance to send Packet_out in cluster is worse than that in Standalone Mode.

What the figure above shown is the latency that ONOS controller sends the Packet_out back in cluster mode. The

latency performance in cluster mode is inferior to the latency performance in Standalone Mode, which is the same

as Packet_out rate performance. Specific latency test result is listed in the following table.

200

400

600

800

989 969 947

0

200

400

600

800

1000

1200

200 400 600 800 1000 1200 1400

Pa

cke

t_o

ut R

ate

: P

acke

ts/s

Packet_in Rate : Packets/s

Packet_out Response Rate

Packet_out rate(cluster mode)

0

500

1000

1500

2000

2500

3000

3500

200 400 600 800 1000 1200 1400

La

ten

cy(m

s)

Packet_in rate : Packets/s

Packet_out Response Latency

minimun latency (ms) maximum latency (ms) average latency (ms)

Page 12: ONOS Controller Performance Test Report - · PDF fileONOS Controller Performance Test Report 3 2. Turn on test tool OFsuite_Performance, use “--switches=[N]” option to set a certain

ONOS Controller Performance Test Report 10

www.sdnctc.com

A B C D E

1 packet_in rate minimum latency

(ms)

maximum

latency (ms)

average latency

(ms) packet_out rate

2 200 1.00391 195.06787 6.95843 200

3 400 0.75195 218.17993 8.74307 400

4 600 0.77393 224.17798 11.45519 600

5 800 0.63599 238.61499 20.66721 800

6 1000 0.68311 220.61084 49.64049 1000

7 1200 56.20776 1480.61499 801.65281 1200

8 1400 62.98486 2923.64697 1410.71205 1400

3.4 FLOW MOD SENDING RATE TEST

Test Purpose

Test the latency and max rate when the controller under test sends the flow entry down.

Test Methodology

In this test, test tool will simulate aswitch that is connected to the controller, test tool will send Packet_in message

containing ARP_request up with a certain rate, test tool sends Packet_in message up containing relevant ARP_reply

after receving Packet_out sent down by the controller to be tested, and the controller that will be tested will

sending Flow_mod information down, then calculate the delay and rate that the controller under test sends the flow

table down according to the records of Packet_in message and Flow_mod information, the specific test procedures

are as the following:

1. Turn on the controller under test

2. Turn on test tool, set a switch, and do not set the topological structure, wait switch to connect to the controller;

3. Use set-arp-rate command to set the rate that switch sends Packet_in message up on test tool;

4. Wait test tool to complete the test;

5. Use show-result command to view test result;

6. Alert the rate to send Packet_in up and test it again;

7. Do five iteration tests to gain the average test result.

Test Tool Requirements

1. Test tool can simulate OpenFlow 1.3 switch, and connect to the controller;

2. Test tool can respond to all related OpenFlow 1.3 protocol information(Hello,Echo,feature_request etc);

3. Test tool can send Packet_in message containing ARP_request up in order to trigger the controller under test to

send Packet_out down;

4. Test tool can send Packet_in message containing ARP_reply up in order to trigger the controller under test to

send Flow_mod addition flow table down;

5. Test tool can analyze Flow_mod information, which makes sure this information is triggered by test tool;

6. Test tool can send Packet_in message up with constant speed defined by users;

7. Test tool has thorough Log function.

Page 13: ONOS Controller Performance Test Report - · PDF fileONOS Controller Performance Test Report 3 2. Turn on test tool OFsuite_Performance, use “--switches=[N]” option to set a certain

ONOS Controller Performance Test Report 11

www.sdnctc.com

Test Result

Standalone Mode

It can be seen from the test result in the figure above that when Packet_in sending rate of test tool is lower than

5000packets/s, ONOS controller can better process Packet_in message, the Flow_mod rate and Packet_in rate have

little difference; when Packet_in rate is more than 5000packets/s, the Flow_mod sending rate and Packet_in rate of

ONOS controller have larger differences, by this taken, this rate is the max flow table installing rate of ONOS

controller in this hardware configuration.

The figure above shows the latency that ONOS controller sends the flow entry down. It can be seen from the figure,

when the rate that test tool sends Packet_in is lower than 4000packets/s, the latency that ONOS controller sends

flow table down is lower relatively; when the rate that test tool sends Packet_in up is more than 4000packets/s, the

latency that ONOS controller sends flow entry down increases accordingly. The detailed test results are listed in the

following table.

1000

2000

3000

4000

5000

5730

6286

0

1000

2000

3000

4000

5000

6000

7000

1000 2000 3000 4000 5000 6000 7000

Flo

w_

mo

d R

ate

: P

acke

ts/s

Packet_in rate : Packets/s

Flow Setup Rate

flow_mod rate(standalone mode)

0

100

200

300

400

500

600

1000 2000 3000 4000 5000 6000 7000

La

ten

cy(m

s)

Packet_in Rate : Packets/s

Flow Setup Latency

minimun latency (ms) maximum latency (ms) average latency (ms)

Page 14: ONOS Controller Performance Test Report - · PDF fileONOS Controller Performance Test Report 3 2. Turn on test tool OFsuite_Performance, use “--switches=[N]” option to set a certain

ONOS Controller Performance Test Report 12

www.sdnctc.com

A B C D E

1 packet_in rate minimum latency

(ms)

maximum

latency (ms)

average latency

(ms) packet_out rate

2 2000 21.13013 130.25708 49.19182 1000

3 3000 29.49097 177.20386 55.13369 2000

4 4000 45.63306 208.44214 74.31397 3000

5 5000 52.16016 225.53101 102.40141 4000

6 6000 106.43799 351.04395 238.92432 5000

7 7000 169.18384 458.54614 342.36064 5730

8 8000 200.229 530.23096 380.48683 6286

The flow_mod rate results in the above table means when there is no loss of packets, count the received flow_mod

messages and calculate the average speed instead of the number of the actually received flow_mod messages. The

results of the tests in the cluster mode are calculated in the same method.

Cluster Mode

In this test, three servers respectively run ONOS controller to form controller cluster for the test. Test results are

shown in the following figure and table.

In can be seen from the figure above, comparing to Standalone Mode, the rate that ONOS controller sends flow

table down in cluster mode decreases obviously, and the max rate is 800 packets/s, when Packet_in rate is more

than 800 packets/s, the Flw_mod rate got by test has been already not the valid result, the latency of sending flow

table down also increases immensely (see the following figure).

200

400

600

800

541

394

0

100

200

300

400

500

600

700

800

900

200 400 600 800 1000 1200

Flo

w_

mo

d R

ate

: P

acke

ts/s

Packet_in rate : Packets/s

Flow Setup Rate

flow_mod rate(cluster mode)

Page 15: ONOS Controller Performance Test Report - · PDF fileONOS Controller Performance Test Report 3 2. Turn on test tool OFsuite_Performance, use “--switches=[N]” option to set a certain

ONOS Controller Performance Test Report 13

www.sdnctc.com

The figure above shows the latency that ONOS controller sends flow entry down in cluster mode, specific test result

is shown in the following table.

A B C D E

1 packet_in rate minimum latency

(ms)

maximum

latency (ms)

average latency

(ms) packet_out rate

2 200 118.35205 295.43701 178.56478 200

3 400 238.16113 383.13892 272.99643 400

4 600 317.48706 480.95386 351.55047 600

5 800 437.16992 590.61499 458.77796 800

6 1000 654.26221 810.68506 688.66228 541

7 1200 1019.29102 1185.32104 1112.06625 394

3.5 END-END LINK ESTABLISHMENT TIME TEST

Test Purpose

Test the time of setting end to end link of controller under different network topology.

Test Methodology

In this test, the test tool will simulate a certain amount of switches which are connected to controller by specific

topological structure. In addition, the test tool will simulate two hosts, which are connected to two switches

respectively (position of hosts can be user-defined). By simulating ARP request/reply process of hosts, setting time

from end to end path of controller which will be measured under the mutual connection among hosts. Specific test

steps are as follows:

1. Start the controller under test;

2. Start test tools, set the number of switches;

0

200

400

600

800

1000

1200

1400

200 400 600 800 1000 1200

La

ten

cy(m

s)

Packet_in rate

Flow Setup Latency

minimun latency (ms) maximum latency (ms) average latency (ms)

Page 16: ONOS Controller Performance Test Report - · PDF fileONOS Controller Performance Test Report 3 2. Turn on test tool OFsuite_Performance, use “--switches=[N]” option to set a certain

ONOS Controller Performance Test Report 14

www.sdnctc.com

3. Set the network topology structure well, after the connection between switch and controller become stable;

4. Wait for the controller to complete the topology discovery process;

5. Use the set-host command to set the host to connect to a particular switch;

6. Wait for test tools to complete the test;

7. Use the show-result command to view the test results;

8. Change the number of switches or topology structure and do test again;

9. Do five iteration tests to get average test results.

Test Tool Requirements

1. Test tools can simulate OpenFlow 1.3 switch, and establish a connection with the controller;

2. Test tools can respond to all relevant OpenFlow 1.3 protocol information (Hello, Echo, feature_request etc);

3. Test tools can send Packet_in message containing ARP_request to touch off the measured controller to send

Packet_out;

4. Test tools can send Packet_in message containing ARP_request to touch off the measured controller to send

flow mod addition flow table;

5. Test tools can resolve flow mod message to confirm whether the link is established successfully;

6. Test Tool can simulate a host connected to the switch;

7. The test tool can simulate communication between hosts.

Test Results

Standalone Mode

The test results of End-end link establishment time under Standalone Mode is shown in figure. In this test, the two

hosts are connected to two different switches respectively to test the link establishment time under the biggest

network hop count. For Linear topology, two hosts are connected to two switches at the head and the tail of the

topology of switch. The link establishment time of Linear topology is longer than that of Ring topology.

Cluster Mode

352.19

2898.97

12044.19

195.33958.74

3347.8

137.63 218.13 170.18

0

2000

4000

6000

8000

10000

12000

14000

100 300 500

Pa

th S

etu

p T

ime

(ms)

Number of Switches

Reactive Path Setup Time

linear ring tree

Page 17: ONOS Controller Performance Test Report - · PDF fileONOS Controller Performance Test Report 3 2. Turn on test tool OFsuite_Performance, use “--switches=[N]” option to set a certain

ONOS Controller Performance Test Report 15

www.sdnctc.com

The test results of End-end link establishment time under Cluster Mode is shown in figure. In this test, when the

number of switches is equal to or greater than 300, ONOS controller cannot complete acquisition of Leaf-spine

topological structure, the corresponding test from end to end link establishment cannot be completed, so N/A is

used to show

3.6 ROLE ELECTION TIME TEST

Test Purpose

Test time of certainty of master slave relation of each controller of cluster of controllers

Test Methodology

When the controller operates in a cluster mode, the master and slave roles allocation of each controller shall be

decided by internal mechanism. This process belongs to contents of east-west direction interface of SDN controller,

does not belongs to OpenFlow. But its effects must be considered, if the time consuming of this process is too long.

In this test, the test tool connects each simulative switch to each controller of controller cluster respectively,

calculates the time difference between each TCP message and the last Role_request information, in order to obtain

schedule of master slave relation of each controller of controller cluster. Specific test steps are as follows:

1. Start the controller cluster;

2. Start test tools, set the number of switch well;

3. Do packet capture at an end of the test tool;

4. Wait long enough period of time to determine the controller does not send Role_request message anymore;

5. Test tool stops the packet capture;

6. Calculate the schedule of master slave relation of each controller of controller cluster;

7. Change the number of the switch and do test again;

8. Do five iteration tests to get average test results.

Test Tool Requirement

1. Test tools can simulate OpenFlow 1.3 switch, and establish a connection with the controller cluster;

650.07

482.59

274.74

0

100

200

300

400

500

600

700

100

Pa

th S

etu

p T

ime

(ms)

Number of Switches

Reactive Path Setup Time

linear ring tree

Page 18: ONOS Controller Performance Test Report - · PDF fileONOS Controller Performance Test Report 3 2. Turn on test tool OFsuite_Performance, use “--switches=[N]” option to set a certain

ONOS Controller Performance Test Report 16

www.sdnctc.com

2. Test tool can respond to Role_request message in OpenFlow 1.3 protocol;

3. Test tool can simulate OpenFlow 1.3 switch with different quantity.

Test Result

It can be seen from the test results in the above figure that the more the number of switches is, the longer the time

needed by the controller cluster to complete assignment of master-slave roles is.

4. GLOBAL SDN CERTIFIED TESTING CENTER

As a third-party, neutral SDN testing and certification lab, Global SDN Certified Testing Center is committed to the

research and the popularization of SDN (Software Defined Network) technologies. Beyond the United States, it is

the first SDN certified testing center authorized by the ONF across the world. The center enthusiastically assists the

work group of ONF Interoperability Testing in making the rules for the test and establishing a testing and

certification system, whereupon enhancing SDN technical standards and commercial deployment. As the

third-party independent neutral evaluating organization, Global SDN Certified Testing Center supplies services to

various network / terminal devices, cloud computing data centers, operators, and enterprises. Based on actual

customer needs, formulate a detailed test program and certification program, and do rigorous independent testing,

and ultimately deliver detailed and authoritative test report.

0.99

4.13

7.38

12.08

15.03

0

2

4

6

8

10

12

14

16

100 300 500 800 1000

Tim

e(s

)

NUmber of Switches

Role Intimation Time

Time(s)