WHITE PAPER - xenanetworks.com · WHITE PAPER 3 Xena Networks – Global Price/Performance Leaders in Gigabit Ethernet Testing – ... NFV is the overall principle of implementing

WH

ITE

PAPE

R

1

Xena Networks – Global Price/Performance Leaders in Gigabit Ethernet Testing – www.xenanetworks.com

“SDN with OpenFlow

switches promises

flexibility and fast

configuration of

communication

networks. However

test and verification

of OpenFlow switch

performance is

essential.”

WHITE PAPER

OVERVIEW Communication networks have traditionally been built with dedicated devices like routers, switches and firewalls, each being designed for a specific task. The result is totally distributed network control, where decisions on data forwarding are done in the network devices based on various algorithms and protocols. Once set up and configured the network would work fine, but changing the behavior of the network can be costly and time consuming – if possible at all.

A new concept – Software-Defined Networking (SDN) – is generating huge interest because it promises to add higher flexibility and faster configuration of networks.

For SDN devices it is important that they meet the Open Networking Foundation (ONF) OpenFlow® specifications when OpenFlow® is included in the SDN solution. To ensure SDN/OpenFlow® products meet specifications ONF has defined conformance test specifications. Furthermore ONF has authorized test labs to perform the conformance testing.

In addition to conformance testing it is also important to know the performance of the SDN devices like switches to get information on how well an OpenFlow enabled switch performs OpenFlow message processing and data packet forwarding.

This White Paper describes relevant OpenFlow performance tests and explains how L2-3 test platforms ValkyrieBay and ValkyrieCompact equipped with appropriate test modules provide the features needed to quickly and cost-effectively test and verify the OpenFlow performance of switches in the SDN.

OpenFlow® OpenFlow® is an element in SDN solutions. Testing the performance of OpenFlow® switches is essential to ensure the quality of the SDN solutions.

WH

ITE

PAPE

R

2


OpenFlow Performance Testing

Contents

OVERVIEW ........................................................................................................................... 1

INTRODUCTION ................................................................................................................... 3

OpenFlow® Specifications ............................................................................................... 6

Other Protocols for the SDN Open Southbound API ................................................... 6

OpenFlow Performance Testing ...................................................................................... 6

OpenFlow Message Processing Performance ............................................................. 7

OpenFlow Data Packet Forwarding Performance ....................................................... 9

OpenFlow Vendors ........................................................................................................ 11

Xena Networks OpenFlow Performance Test Solutions ................................................ 11

Testing up to Layer 3 ................................................................................................. 11

CONCLUSION ..................................................................................................................... 13

WH

ITE

PAPE

R

3


INTRODUCTION Communication has always been essential and modern technology has provided increasing numbers of applications and solutions. Communication networks have been expanded and updated to support the new applications and solutions, which however have caused an increasing complexity of the networks. Traditionally networks have been built with dedicated, autonomous network devices like routers, switches, and firewalls, each being designed for a specific task. Decisions on data forwarding are done in the network devices based on various algorithms and protocols. The devices communicate with each other, but there is no centralized device that has the overall control. Once set up and configured the network would work fine, but changing the behavior of the network can be costly, cumbersome and time consuming – if possible at all.

A new concept – Software-Defined Networking (SDN) – is generating huge interest because it promises to add higher flexibility and faster configuration of networks. SDN is split into a control plane (which decides where the traffic is sent) and a data plane (or forwarding plane), which forwards traffic in the direction of the destination according to control plane decisions.

Figure 1: SDN architecture SDN network applications are located on top of the control plane. The network applications will have requirements for the behavior of the network, which they will communicate to the controller(s) in the control plane through the open northbound API.

The Open Networking Foundation (ONF) has defined the OpenFlow® switch specification. It defines the functioning of a Data forwarding device in the Data plane (see figure 1). It also defines the OpenFlow protocol for communication over the “Open southbound API” between an OpenFlow switch and a controller in the Control plane.

OpenFlow switches in the Data plane forward data based on flow tables, which are programmed by the controller in the Control plane using the OpenFlow protocol. The contents of the flow

https://www.opennetworking.org/

WH

ITE

PAPE

R

4


table have developed over the years. In the first version of the OpenFlow switch specification each table entry was defined as shown in figure 2.

Header Fields Counters Actions Figure 2: OpenFlow switch specifications 1.0.0 flow table entry • Header fields – to match against received packets

• Counters – to be updated for packet matching the header fields e.g.:

o Number of packets and bytes for each flow

o Time since the last packet matched the flow

The specifications include a list of mandatory counters that the OpenFlow switches must

support and a number of optional counters that may be supported by the OpenFlow

switches

• Actions – to apply to packets matching the header fields e.g.:

o Send the packet to a specific output port

o Modify a header field in the packet

o Drop the packet

Ingress port

Eth. SA

Eth.

DA

Eth. type

VLAN ID

VLAN

prio

IP SA

IP DA

IP prot

o

IP TOS

TCP/UDP source

TCP/UDP destinatio

n 1 2 3 4 5 6 7 8 9 10 11 12

Figure 3: Header fields defined in OpenFlow switch specifications 1.0.0 (the “12 tuple”) Up to OpenFlow switch specification 1.3.0 the flow table entry was extended as shown in figure 4. Also the Match Fields (Header Fields) were extended from version 1.0.0 to version 1.3.0 to also cover more protocols like MPLS, PBB, IPv6 and optionally metadata specified by a previous table.

Match Fields Priority Counters Instructions Timeouts Cookie Figure 4: OpenFlow switch specifications 1.3.0 flow table entry A flow is one or more data packets that match the contents of the “Match Fields” (“Header Fields”) part of a flow table entry. When a data packet is received, the OpenFlow switch will compare header fields in packet with the flow table entries. If there is a match, counters will be updated and the actions/instructions specified in the flow table entry will be executed. A packet that does not match any flow table entry can be sent to the OpenFlow controller, which may drop the packet or add a new flow table entry that can handle the new packet flow.

In OpenFlow switch specification 1.1.0 it was described that an OpenFlow switch must have at least one and optionally more flow tables. When a data packet is received it must be matched

WH

ITE

PAPE

R

5


with all flow tables in an OpenFlow pipeline as illustrated in figure 5. For each table that the data packet passes the following must be done:

• Find the matching flow entry with highest priority

• Apply instructions:

o Modify packet and update match fields

o Update action set

o Update metadata

• Send match data and action set to the next table

Figure 5: Packets are matched against multiple flow tables in the OpenFlow pipeline Table entry actions may direct data packets to a group. The group will define additional actions e.g. for flooding, fast reroute and link aggregation. This is defined in group tables, which contains group entries with a list of action buckets. An action bucket is a set of actions and parameters for the group.

The controller can add, modify or delete flow table entries using “FlowMod” messages. When the flow table entry is updated the switch will process data packets in the flow matching the updated flow table entry at line rate.

A concept related to SDN is Network Function Virtualization (NFV). NFV is the overall principle of implementing network functions (like routing, intrusion detection and intrusion prevention) as software running Commercial Off-The-Shelf (COTS) hardware. Functions implemented this way are called Virtual Network Functions (VNF). OpenFlow switches could be created as VNFs.

Alternatively the OpenFlow switches are dedicated products, which typically use Ternary Content Addressable Memory (TCAM) and proprietary software to support the handling of the flow table and the OpenFlow protocol. TCAM is a specialized type of high-speed memory that searches its entire contents for some data in a single operation and returns the address where the data is stored. A TCAM can store and look up data using “0”, “1” and “don’t care”, which makes it very efficient for handling flow tables.

WH

ITE

PAPE

R

6


OpenFlow® Specifications

The Open Networking Foundation (ONF) is a non-profit, user-driven organization dedicated to accelerating the adoption of SDN & NFV. ONF members include more than 100 companies e.g. equipment manufacturers and operators – please refer to the ONF member list. Since December 2009 ONF has published OpenFlow switch specifications (see table 1).

OpenFlow Specification Version

Released

1.0.0 December 2009

1.1.0 February 2011

1.2 December 2011

1.3.0 June 2012

1.3.1 September 2012

1.4.0 October 2013

1.5.0 December 2014

1.5.1 March 2015

Table 1: A selection of the OpenFlow switch specifications published by ONF

OpenFlow products will in most cases comply with specification version 1.0.0 and/or 1.3.0.

Other Protocols for the SDN Open Southbound API The OpenFlow protocol is the most commonly used protocol for the SDN Open Southbound API. However SDN does not require that the OpenFlow protocol is used, and there are other protocols that may be used for the SDN Open Southbound API, including:

• Forwarding and Control Element Separation (ForCES), which is standardized by IETF

• Locator/ID Separation Protocol (LISP), defined by the IETF

• NetConf/YANG, defined by the IETF

• Path Computation Element Protocol (PCEP)

OpenFlow Performance Testing

Several types of testing are relevant for OpenFlow switches:

• OpenFlow performance testing will provide information on an OpenFlow capable switch.

With this it can be verified that the switch behaves as expected. Furthermore the tests will

https://www.opennetworking.org/

https://www.opennetworking.org/our-members

WH

ITE

PAPE

R

7


provide information with which different switches can be compared. OpenFlow performance

testing covers:

o OpenFlow message processing

o Data packet forwarding

In both cases it will be necessary to send OpenFlow messages from a controller to the

OpenFlow switches through the open southbound API as a part of the test.

• OpenFlow conformance testing will verify that a product complies with the OpenFlow

specifications. ONF has defined conformance test specifications and have also authorized a

number of conformance testing labs (see Authorized Test Labs on the ONF web page).

OpenFlow conformance testing is outside the scope of this White Paper.

OpenFlow Message Processing Performance Several tests can be performed to verify OpenFlow message performance, including:

• Flow-Mod response time

• Flow table capacity

• Packet-In/Packet-Out capacity

• Timeout verification

• OpenFlow pipeline latency

Figure 6: Test setup for OpenFlow performance testing – OpenFlow message processing

https://www.opennetworking.org/certification/product#labs

WH

ITE

PAPE

R

8


Flow-Mod response time is the time it takes from a flow table entry is sent from a controller until the actions/instructions in the entry is implemented in the OpenFlow switch. This can be tested by adding (or changing) a flow table entry specifying that a flow of data packet should be forwarded from port A to port B. If you capture the OpenFlow messages to the switch under test on the open southbound API and data packets from an Ethernet tester generating traffic from port A to B you can use Wireshark to see the time stamp of the Flow-Mod message setting up the table entry and the time stamp of the first data packet that is sent on port B. With the two time stamps you can calculate the time it took to implement the flow table entry. A similar test can be conducted for flow table entries that are deleted.

Flow table capacity can be tested by first deleting all entries in a flow table and then inserting entries in that table until error messages indicate that the table is full. As a part of the test it should be verified with an Ethernet tester that at least some of the entries have actually been installed: If the entry specifies that a flow of data packet should be forwarded from port A to port B, an Ethernet tester can be connected to the two ports, set up to generate data packets matching the flow and verify that the packets actually go from port A to port B. Numerous streams of data packets should be generated – emulating different flows – to see that all flows are handled in accordance with the contents of the flow tables.

Packet-in/packet-out capacity: Packet-in is a facility in the OpenFlow protocol that transfers the control of a data packet to the OpenFlow controller. The data packet is embedded in an OpenFlow packet-in message, which also contains the reason for sending the packet to the controller:

• No matching flow (“table-miss”)

• An action explicitly defines that the packet must be sent to controller

• Packet has an invalid TTL

The controller can send the packets received in the packet-in message back to the data plane in a packet-out message.

The packet-in/packet-out facility can be used for applications like MAC learning and topology discovery. The capacity for sending packet-in/packet-out is typically limited and should be checked. Packet-in capacity can be checked by making an entry in the flow table defining that packets must be output to the controller. Then the Ethernet tester in figure 6 is used to send that type of packets starting with very few per second and increasing number of packets until the system is saturated. Packet-out capacity is checked in a similar way, with the controller sending packet-out messages. The communication between the switch under test and the controller should be captured and presented in Wireshark for detailed analysis.

Timeout verification: Newer versions of the OpenFlow specification include a “Timeout” section in the table flow entries. Two types of timeouts are supported:

• Hard timeouts – the maximum amount of time an entry is active in the flow table regardless

of the activity in the flow.

WH

ITE

PAPE

R

9


• Idle timeouts – the time without activity in the flow before the related table entry expires

To verify hard timeouts you can capture the FlowMod message creating a flow table entry directing a flow of data packets from port A to port B. If you send packets from port A to port B and also capture these packets, you can use Wireshark to compare the timestamp of the FlowMod message and the last received data package on port B to verify the hard timeout.

Idle timeouts for a flow table entry that defines data packets to be sent from port A to port B can be verified by first sending traffic from port A to B, then stopping and then send one data packet into port A after the idle timeout has expired. No data packets should appear on port B. Instead a packet-in message may appear sent from the switch under test to the controller to inform about the packet that don’t fit with the flow table entries (table-miss). This can be seen by capturing the OpenFlow messages sent between the switch and the controller.

OpenFlow pipeline latency measurements will show the implication of having multiple tables in a switch with many matches and action set updates compared with simple tables with only one or two matches. This is measured by creating flow table entries that emulates the two situations and measure the latency through the switch in the setup shown in figure 6.

OpenFlow Data Packet Forwarding Performance

Figure 7: Test setup for OpenFlow performance testing – Data packet forwarding When an SDN is configured the Data packet forwarding performance of the data plane can be tested. This includes verification of:

• Throughput – maximum data rate that can be transferred through the device/path

• Latency – time it takes to transfer data through the device/path

• Frame loss – data frames lost during the transfer of data through the device/path

WH

ITE

PAPE

R

10


• Back-to-back frames – identifies the longest burst of frames with minimum inter-frame gaps

that can be sent through the device/path without frame loss

• Jitter – variation in delay of transferred data packets (also known as Frame Delay Variation)

Testing is in typically done in accordance with the RFC 2544 Benchmarking Methodology for Network Interconnect Devices tests. RFC 2544 specifies throughput, latency, frame loss and back-to-back testing – most Ethernet testers also include jitter testing in their RFC 2544 test suite. The test setup in figure 7 should be used for end-to-end RFC 2544 testing of a path through a number of OpenFlow switches. If a single OpenFlow switch is tested in accordance with RFC 2544 the test setup in figure 6 will be relevant.

Enterprise customers typically sign Service Level Agreements (SLAs) with a network operator for their communication path through a network. This will also apply for paths through a SDN with OpenFlow switches in the data plane. Therefore it will be relevant to verify that the requirements in the SLA are fulfilled when flow tables are set up to support the customers’ requirements. ITU-T has defined the Y.1564 standard for turning up, installing and trouble-shooting Ethernet-based services. Y.1564 allows a complete validation of Ethernet SLAs in a single test.

Y.1564 focuses on the following service level parameters:

• Information rate (IR) – or Bandwidth

• Frame transfer delay (FTD) – or latency

• Frame delay variation (FDV) – or packet jitter

• Frame loss ratio (FLR)

• Availability (AVAIL)

The test setup in figure 7 is relevant for the Y.1564 test. It will measure the parameters for the two directions of a connection separately with two testers – one at each end of the connection. To identify FTD for each direction it is however necessary the synchronize time stamps in the two testers, typically using Precision Time Protocol (PTP), Network Time Protocol (NTP) or Global Positioning System (GPS).

A SLA will guarantee Service Acceptance Criteria (SAC), which are worst case values for FTD, FDV, FLR and AVAIL at a Committed Information Rate (CIR) – the maximum bandwidth guaranteed by the network operator to the customer. The SLA can also include a commitment on how large bursts the network will accept - the Committed Burst Size (CBS). The network operator may allow that both CIR and CBS are exceeded to some extend: Excess Information Rate (EIR) and Excess Burst Size (EBS). However when CIR or CBS are exceeded it is not expected that the SACs are met.

WH

ITE

PAPE

R

11


OpenFlow Vendors

A number of vendors are now offering OpenFlow solutions, including:

• Cisco

• Ericsson

• FiberHome Telecommunication Technologies

• Hangzhou DPtech Technologies

• Hewlett Packard Enterprise

• Huawei Technologies

• NEC Corporation

• ZTE Corporation

ONF maintains a list of ONF OpenFlow® Conformant/Certified Products.

Xena Networks OpenFlow Performance Test Solutions

Figure 8: The versatile and powerful Xena Networks Layer 2-3 testers ValkyrieBay and ValkyrieCompact

The versatile and powerful Xena Networks Valkyrie Layer 2-3 testers ValkyrieBay and ValkyrieCompact are the obvious choice for OpenFlow performance testing. When the Xena testers are equipped with the ValkyrieTimeSynch their clocks can be synchronized.

Testing up to Layer 3 Based on Xena’s advanced architecture, ValkyrieBay and ValkyrieCompact equipped with relevant test modules are proven solutions for Ethernet testing at layers 2 and 3. Advanced test scenarios can be performed with ValkyrieBay and ValkyrieCompact equipped with relevant test modules using the free test applications for the modules:

ValkyrieManager test software is used to configure and generate streams of Ethernet traffic between Xena test equipment and Devices Under Test (DUTs)/Systems Under Test (SUTs) and analyze the results. ValkyrieManager features include:

• Multistream traffic generation at line rate

• Generation of traffic streams with different rates and packet sizes

• Generation of IPv4 and IPv6 traffic

• Generation of traffic streams with UDP and TCP headers

• Generation of frames with VLAN tags, MPLS labels and PBB tags

https://www.opennetworking.org/openflow-conformance-certified-products

WH

ITE

PAPE

R

12


• A “Port-to-Port” mode where frames received on one port are sent out on another port. This

can be used for capturing OpenFlow messages between the controller and the switch under

test in figure 6. The captured messages can be analyzed in Wireshark

• A Stream Scheduler, which can be used to build a sequence of actions like sending traffic,

stopping after some time and resuming sending traffic (which could be just a single data

packet) after another period of time

One ValkyrieManager can control multiple ValkyrieCompact and ValkyrieBay test chassis, which can be located far away from each other, e.g. at the ends of connections-to-be-tested supporting one-way measurements.

Valkyrie2544 offers full support for the 4 test types specified in RFC 2544:

• Throughput

• Latency

• Frame loss

• Back-to-back frames

• Jitter (Frame Delay Variation) is also supported

Valkyrie2544 supports flow-based learning, which will emit a brief traffic preamble before starting the actual test. This can be used to ensure that the flow-based switch has learned all necessary addresses, preventing a latency spike.

Valkyrie1564 provides full support for both the configuration and performance test types described in Y.1564 for complete validation of Ethernet Service Level Agreements (SLAs) in a single test.

ValkyrieTimeSynch enables multiple ValkyrieCompact or ValkyrieBay test chassis to synchronize their local time to each other. This can be used for One-Way Latency (OWL) measurements between two test chassis, synchronized traffic start between multiple chassis and accurate timestamping of captured packets in exported PCAP files. ValkyrieTimeSynch is compatible with Valkyrie1564 test methodology and can be used for Y.1564 OWL measurements.

WH

ITE

PAPE

R

13


Figure 9: ValkyrieTimeSynch synchronizes reference time for the Xena Valkyrie Layer 2-3 testers.

Valkyrie2889 is an application for benchmarking the performance of Layer 2 LAN switches. The following RFC 2889 test types are supported:

• All Throughput and Forwarding rate tests (both Fully and Partially meshed)

• Congestion Control

• Address Caching Capacity

• Address Learning Rate

• Broadcast Frame Forwarding and Latency

• Forward Pressure and Maximum Forwarding Rate

Valkyrie3918 makes it easy to create, edit and execute all test types specified in RFC 3918. RFC 3918 describes tests for measuring and reporting the throughput, forwarding, latency and Internet Group Management Protocol (IGMP) group membership characteristics of devices that support IP multicast protocols.

ValkyrieCLI is another free application for ValkyrieBay and ValkyrieCompact. It is a powerful and easy-to-use command-line-interface (CLI) scripting API that makes test automation easier for test engineers:

• Ideal for test automation of e.g. production environments

• Controls ValkyrieBay and ValkyrieCompact chassis with installed test modules

• Powerful CLI approach from any TCP/IP capable tool environment

• Unified syntax for CLI- and GUI-generated test port configurations makes it easy to learn

• Script examples of Tcl, Perl, Java, Ruby, BASH and Python available

• Intelligent console tool bundled free with ValkyrieManager

CONCLUSION Traditional communication networks have been built with dedicated, autonomous devices like routers, switches, and firewalls, each being designed for a specific task. The result is totally distributed network control, where decisions on data forwarding are done in the network devices based on various algorithms and protocols. Once set up and configured the network would work fine, but changing the behavior of the network can be costly, cumbersome and time consuming – if possible at all.

Software-Defined Networking (SDN) is generating huge interest because it promises to add higher flexibility and faster configuration of networks.

WH

ITE

PAPE

R

14


It is important to know the performance of SDN devices like OpenFlow switches. This can provide information on how well the switch performs OpenFlow message processing and data packet forwarding. The Valkyrie L2-3 test platforms ValkyrieBay and ValkyrieCompact equipped with appropriate test modules provide the features needed to quickly and cost-effectively test and verify the OpenFlow performance of switches in the SDN.

Documents

WHITE PAPER - xenanetworks.com · WHITE PAPER 3 Xena Networks – Global Price/Performance Leaders in Gigabit Ethernet Testing – ... NFV is the overall principle of implementing