Summit 16: Automated Platform for Testing VNF Performance and Interoperability with Variable Flavors

June 20–23, 2016 | Berlin, Germany

Automated Platform for Testing VNF Performance and Interoperability with

Variable Flavors

Takashi Torii NEC CorporationTatsuya Tohyama, Okinawa Open Laboratory/OKIT

Introduction of Okinawa Open Laboratory(OOL)

Vision: Okinawa Smart Hub

• Okinawa Prefectural Government vision and policy for industrial development and promotion • Attract companies, talents and knowledge from other prefectures in Japan and overseas,

especially Asian countries, and promote new value creation in Okinawa• Form major international ICT hub in Asia

Okinawa Smart Hub and OOL

• Contribute to realizing Okinawa Smart Hub concept• Based in Okinawa with support from Okinawa Prefectural Government, attracting leading

companies, organizations, educational institutions, and research bodies from other prefectures in Japan and overseas, especially Asia, and promoting international R&D activities

• Contributes to promoting concentration of ICT industries and advancement of regional companies

SDN technology research and standardization base

OSS cloud development community

Leading Linux and OSS organization

OSS SDN development project

Okinawa Open Laboratory

Practical-use research base that integrates cloud technology and SDN technology

International R&D organization

Leading companies and organizationsfrom other prefectures in Japan

and overseas, especially Asia

Technology cooperation

Cooperation, attraction

Activity support

・Support for R&D activities・Promotion of cloud DC facility use

Mission

• Research and development activities for practical application and dissemination of advanced Information Communications Technologies (ICT), especially next-generation ICT platform technologies, such as

• Software-Defined Networking (SDN)• Cloud Computing using Open Source Software (OSS, especially Open Stack)

• Open Innovations

• Collaboration, Joint Work

• Bridging the gaps between users and providers

Activity Model

• OOL connects OSS development communities and market via use cases and reference models to expand user base of next-generation ICT platform technologies in cooperation with MEMBER and partners

• OOL Value Chain: Use Case à Verification à Ref. Model à Promotion• Feedback Loops: (1) OOL ó Market, (2) OOL ó OSS Dev. Communities

OSS Bodies (Community/Consortium)

Develop

Use Case Verify Promote

Issues/Needs

Apply/Use

OOL and its Members

Market (Providers and End Users)

Expand User Base

ReferenceModel

Testbed (OpenStack with SDN)

Member List

Board(5)

Member(29)

Fellow(14)

Total(48)

Project Formations

FullTimeEngineers

Organizer

CompaniesResearchInstituteAcademia

• Planningprojectsandgoals• Communicatewithcompanies• CommunicatewithBoards,Staffsand

Stakeholders(Governmentetc)• Managetheactivities• Promoteandcollaborate

Staffs Board

Dev Project Example: OF-Patch

• Patch panel function implemented using OpenFlowtechnology

OFPMUser

Change network topology by remote operation

OF-PatchGUI

OFC

OFS Patch by flow

Host1 Host2 Host3

OPNFV Test Lab

• VNF Test Automation

• Service Function Chaining

• Joint POC with oversea members

Internet

JumpServer

POD#1(Fuel)

POD#2(RDO)

VNF Test Automation Project

Background

• Market of VNF is now emerging• ETSI proposed NFV architecture• Open Source VNF

• VyOS (Vyatta）

• Open vSwitch…etc

• Vender VNF• Cisco CSR1000V• Juniper FireFly, vSRX• Brocade Vyatta Core

…etc

• But we don‘t have test automation platform of VNF

Background

• Problem of VNF testing• There is no standard• Performance and stability is depend on flavors• Many test patterns are required than H/W appliance

• Impossible to cover all test patterns by manual• Risk of human error or miss configuration

• Configurations are depend on vendors• Need support from venders or skilled engineers in all phase

• To solve these problems, we have developed VNF automation testing system

How to test VNF?

• Workflow• Deploy

• Deploy VNF on VIM with flavor• Initialize VNF

• Configuration• Configure and confirm• Check the network connectivity with other devices

• Run• Subscribe test patterns and run on traffic generator.• Check VNF status and behavior

• Reporting• Writing report based on result

Test workflow

TestDeployVNFInitialize Initialize

VNFConfig Delete

VNFLog

ArchiveAnalysisReportCheck

Test workflow

• Benefit of automation• Engineer can focus on troubleshooting• Engineers who don’t have much skills can operate the tests• High repeatability

Repeat


VNFConfig Delete

VNFLog


ArchitecturePoC User

Orchestrator

Web-Portal

Web-Portal

Report generationManagement of Test

REST API

Interoperability Traffic Test

REST API

Virtual Appliance Ctr(Deploy of VNF appliance)

NW Device Ctr(Insert/delete of configure)

Tester Ctr(Control of IXIA)


Orchestrator

Web-Portal

Web-Portal


REST API


REST API




Web Portal

• Set the test parameters• Select the test scenario from drop down lists

Orchestrator

Web-Portal

REST API

PoC User

Test Parameter


Orchestrator

Web-Portal


REST API


REST API




Web-Portal

Orchestrator

• Controller management• Controls each components based on the configuration of test scenarios.

• Creating test scenario• Create test scenarios by GUI

• Reporting• Output PDF format

Scenario generate tool


Orchestrator

Web-Portal


REST API


REST API




Web-Portal


Orchestrator

Web-Portal


REST API


REST API




Web-Portal

Virtual Appliance Controller

• Multi virtualized platform control• Deploy virtual appliance on selected virtualized platform.

Orchestrator

REST-API

REST-MessageOpenStack Link

Virtual Appliance Controller

REST-API REST-API


Orchestrator

Web-Portal


REST API


REST API




Web-Portal

NW Device Controller

• Multi vendor control• Provides the ability to control multiple vendor VNF

Orchestrator

NW Device Controller

REST-API

Telnet/SSH/Netconf

REST-MessageParameter Ex

・Vender Name・Vender Version・Test scenario

other…


Orchestrator

Web-Portal


REST API


REST API




Web-Portal

Tester Controller

• Generating Tcl script• Based on the parameters set by REST API from Orchestrator

• Ixia Driver• Load Tcl Script by IXIA API.

Orchestrator

Tester Controller

REST-MessageParameterEx

・InterfaceInformation・PacketSize(Short,Long,Imix)・Framerateother...

Test Traffic

IXIA API

REST API

Tcl Script

Test workflow

Repeat

VirtualApplianceController

TesterControllerNWDeviceController

Orchestrator


VNFConfig Delete

VNFLog


Our environment

VyOS

vSwitch

vnic0

vSwitch

Interop

erability

L2S

W

Traffic Test L2S

W

vnic1

vnic1

OpenStack (node2)

VMware Esxi (node1)

VMware Esxi (node3)

OpenStack (node4)

vnic2

vnic3

vnic1

vnic2

vnic3

MGMT-Seg

vnic0

vnic0

vSwitch

JuniperFirefly

CiscoCSR1000v

Vyatta5600

vnic0

vSwitchvnic1

VyOS

JuniperFirefly

CiscoCSR1000v

Vyatta5600

Our environment for interoperability

VyOS

vSwitch

vnic0

vSwitch

Interop

erability

L2S

W

Traffic Test L2S

W

vnic1

vnic1

OpenStack (node2)

VMware Esxi (node1)

VMware Esxi (node3)

OpenStack (node4)

vnic2

vnic3

vnic1

vnic2

vnic3

MGMT-Seg

vnic0

vnic0

vSwitch

JuniperFirefly

CiscoCSR1000v

Vyatta5600

vnic0

vSwitchvnic1

VyOS

JuniperFirefly

CiscoCSR1000v

Vyatta5600


VyOS

vSwitch

vnic0

vSwitch

Interop

erability

L2S

W

Traffic Test L2S

W

vnic1

vnic1

OpenStack (node2)

VMware Esxi (node1)

VMware Esxi (node3)

OpenStack (node4)

vnic2

vnic3

vnic1

vnic2

vnic3

MGMT-Seg

vnic0

vnic0

vSwitch

JuniperFirefly

CiscoCSR1000v

Vyatta5600

vnic0

vSwitchvnic1

VyOS

JuniperFirefly

CiscoCSR1000v

Vyatta5600

VNFappliancetestenvironment



VyOS

vSwitch

vnic0

vSwitch

Interop

erability

L2S

W

Traffic Test L2S

W

vnic1

vnic1

OpenStack (node2)

VMware Esxi (node1)

VMware Esxi (node3)

OpenStack (node4)

vnic2

vnic3

vnic1

vnic2

vnic3

MGMT-Seg

vnic0

vnic0

vSwitch

JuniperFirefly

CiscoCSR1000v

Vyatta5600

vnic0

vSwitchvnic1

VyOS

JuniperFirefly

CiscoCSR1000v

Vyatta5600

VNFapplianceforinteroperabilitytesting

VNFapplianceforinteroperabilitytesting


VyOS

vSwitch

vnic0

vSwitch

Interop

erability

L2S

W

Traffic Test L2S

W

vnic1

vnic1

OpenStack (node2)

VMware Esxi (node1)

VMware Esxi (node3)

OpenStack (node4)

vnic2

vnic3

vnic1

vnic2

vnic3

MGMT-Seg

vnic0

vnic0

vSwitch

JuniperFirefly

CiscoCSR1000v

Vyatta5600

vnic0

vSwitchvnic1

VyOS

JuniperFirefly

CiscoCSR1000v

Vyatta5600

Our environment for Traffic Test

VyOS

vSwitch

vnic0

vSwitch

Interop

erability

L2S

W

Traffic Test L2S

W

vnic1

vnic1

OpenStack (node2)

VMware Esxi (node1)

VMware Esxi (node3)

OpenStack (node4)

vnic2

vnic3

vnic1

vnic2

vnic3

MGMT-Seg

vnic0

vnic0

vSwitch

JuniperFirefly

CiscoCSR1000v

Vyatta5600

vnic0

vSwitchvnic1

VyOS

JuniperFirefly

CiscoCSR1000v

Vyatta5600


VyOS

vSwitch

vnic0

vSwitch

Interop

erability

L2S

W

Traffic Test L2S

W

vnic1

vnic1

OpenStack (node2)

VMware Esxi (node1)

VMware Esxi (node3)

OpenStack (node4)

vnic2

vnic3

vnic1

vnic2

vnic3

MGMT-Seg

vnic0

vnic0

vSwitch

JuniperFirefly

CiscoCSR1000v

Vyatta5600

vnic0

vSwitchvnic1

VyOS

JuniperFirefly

CiscoCSR1000v

Vyatta5600




VyOS

vSwitch

vnic0

vSwitch

Interop

erability

L2S

W

Traffic Test L2S

W

vnic1

vnic1

OpenStack (node2)

VMware Esxi (node1)

VMware Esxi (node3)

OpenStack (node4)

vnic2

vnic3

vnic1

vnic2

vnic3

MGMT-Seg

vnic0

vnic0

vSwitch

JuniperFirefly

CiscoCSR1000v

Vyatta5600

vnic0

vSwitchvnic1

VyOS

JuniperFirefly

CiscoCSR1000v

Vyatta5600

TrafficGenerator


VyOS

vSwitch

vnic0

vSwitch

Interop

erability

L2S

W

Traffic Test L2S

W

vnic1

vnic1

OpenStack (node2)

VMware Esxi (node1)

VMware Esxi (node3)

OpenStack (node4)

vnic2

vnic3

vnic1

vnic2

vnic3

MGMT-Seg

vnic0

vnic0

vSwitch

JuniperFirefly

CiscoCSR1000v

Vyatta5600

vnic0

vSwitchvnic1

VyOS

JuniperFirefly

CiscoCSR1000v

Vyatta5600

Demonstration (Movie)

Result of test automation

• Enables multiple patterns of test runs which are difficult by manual

Automatethecombinationtestof2736patterns by40hours

【CSR1000v,Firefly,Vyatta,VyOS】VNFappliance

【VMwareESXi,OpenStack】VIM

【CPU,Mem,HDD,VNIC】

【CSR1000v,Firefly,Vyatta,VyOS】Interoperabilitywithreferencerouter

【VMwareESXi,OpenStack】VIMofreferencerouter

【BGP:5 Pattern,OSPF:7 Pattern】

【TestPattern:3Pattern,FrameRate:6Pattern】

FlavorofVNFAppliance

Routingprotocol

Traffictestpattern

Current Activities

• We have just joined OPNFV, and deploy Community Lab.

• Refactoring to align NFV Architecture of ETSI• Using TOSCA based orchestrator (we are considering Tacker)• Collaborate with Functest

TesterTester

Collaborate with Functest

WebPortal

Functest

WebPortal

VNFTester

VNFM

VNFVNFFunctest

Any Questions?