Ensuring Successful OPNFV-based NFV Deployments | QualiTest Group

Ensuring Successful OPNFV-based NFV

Deployments

2

Hans AshlockQualiSystems

Benny Sand NFV Strategy Lead at QualiTest

Webinar Presenters

Agenda

NFV Testing Challenges & Solutions

Automated NFV Testing Environment as a Service Summary & Q&A

QualiTest & QualiSystems Partnership

3

High Level OPNFV

Agenda




4

High Level OPNFV

5

QualiTest & QualiSystems partnership

| Pure Play Testing focus

| Passion for testing| World’s 2nd largest | Telco expertise

| Automation software| Cloud management| Test & CI automation| Carrier DevOps

Agenda




6

High Level OPNFV

OPNFV Targets

| OPNFV aims to be a carrier-grade, integrated platform that introduces new products and services to the industry more quickly.

| Create an integrated and verified open source platform that can investigate and showcase foundational NFV functionality

| Provide proactive cooperation of end users to validate OPNFV’s strides to address community needs

| Form an open environment for NFV products founded on open standards and open source software

| Contribute and engage in open source projects that will be influenced in the OPNFV reference platform

7

OPNFV Advantages

| Higher reliability, more flexibility

| Faster, lower cost, and higher quality development

| Collaborative decisions about new features and roadmaps

| A common environment for uses and app developers

| Ability for users to focus resources on differentiating development

| Opportunity to drive open standards

8

Agenda




9

High Level OPNFV

Strategic Testing Challenges

| NFV involves very complex integration of various software elements

| Everything known becomes unknown

| Network testing expertise verses Virtualization and Software testing

| Agile orientation will be introduced in Network testing

| Frequent updates in the Network services will require more management and orchestration testing

| More test automation is required to support agility , impact testing and regression testing

| NFV test strategies are needed to accelerate and ensure the delivery of NFV solutions with guaranteed quality

| NFV is not only a technological shift it involves a culture change which impacts testing directly

10

Test Strategy Goals

| Build an NFV Ecosystem

| Drive Quality and governance

| Continuous integration / Dev Ops model

| Reduce cost of integration.

11

12

QualiTest NFV Validation Process

| QualiTest aims to create a unified end-to-end approach to service assurance and network performance

NFV Managemen

t

Orchestration

Infrastructure Migration

QualiTest NFV validation process

Assure full compliance to ETSI ISG Standards.

•Define NFV Testing process•Test Requirements•NFV Test Environment•Risk Assessment•Roles and Responsibilities•Team Skills and Training•Test Inputs / Basis

Assure full compliance to NFV requirements for any network equipment provider and streamline the ability of vendors and users to be able to implement compatible NFV solutions.

Test Environment: simulation tools, network analytics, QoS and QoE probes.Automated Certification Process for each VNF before it is introduced to the network

Test management and Orchestration prior to VNF migration, including each component (e.g. Orchestrator, Vnfm, VIM).

Test the VNF architecture and implementation to assure smooth transition.

NFV Test Policy

ETSI NFV ISG Compliance

(NFV-Co)

Selecting Mano Vendors

(NFV-Ven)

NFV Staging Environment

(NFV-SE)

MANO Testing (MANO-T)

VNF Migration (VNFM-T)

13

What is DevOps?

| Who: Developers, QA and Operations| What (Goal): Clarity, Consistency and Collaboration to

understand the changes that each release brings to the IT and customer environments

| Why: Increase business benefits by reducing the transaction cost associated with delivering incremental change.

| How: Methodology and tools that continue to evolve

14

Where does testing fit in DevOps?

| Addressing a misconception| Testing is not a bottleneck| Testing is a junction point between development

and operations| Testing is a change agent

| DevOps is about closing the distance between development and operations through:| Meaningful collaboration| Early engagement| Continuous processes

15

Shift Left Principle and implementations

| Engage QA early| Push tests to lower levels| Test continuously| Benefits:| Fewer defects created| Defects discovered earlier| Cost per defect drastically reduced

16

Positioning QA with Development and Operations

| Development

| Engage as early as possible in the life cycle

| Generate test automation at lower levels within the product

| Reduce the overall demand for manual testing and testing at the GUI level

| Operations

| Testing and Monitoring in all environments• Environments should be as

production-like as possible• QA needs more control and access

to environments| Continuous feedback and

communication| Testing of Operations processes

17

The Evolution of the Modern Tester| Traditional Tester Skills• Analytical and Logical Thinking• Intellectual Curiosity• Critical Thinking and Rational

Deduction• Ability to Identify and Apply

Fundamental Knowledge

| Modern Tester Skills•Grasping the Bigger Picture• Ability to Recognize and Address

Design Challenges• Ability to Communicate on both

Technical and Business Needs• Ability to identify appropriate use

cases for manual testing (exploratory testing for learning and test design for example)•Ability to generate automation

18

Agenda




19

High Level OPNFV

20

Automated BuildCode Commit Automated Unit Test

Automated Sanity TestAutomated & Scaled Full Regression TestAutomated Test Results Collection

Automated BI Report Generation

Automated Build, Deploy, TestAuto-scaled regression testsApplied to service & network changes

Goal: Agile and Continuous Service Deployments

Step 1: Automate all Infrastructure

Physical Labs

MaaS

Virtual Labs

IaaS & PaaS

• Build virtual labs for VNF-only scenarios• Consolidate or federate virtual & physical labs into private/hybrid clouds

Ensure Orchestration can Handle Network Topologies

=

Carriers adopting infrastructure orchestration to build DevOps have foundthat most cloud management platforms only support compute concepts

CloudShell orchestration is capable of handling E2E NS topologies

Step 2: Create Lab as a Service Cloud

• Create self-service access to full network environments built and provisioned on demand

• Make available to internal and external users (including vendors, contractors)

Key goals:• Increase efficiency utilization of test lab

resources

• Reduce ratio of setup to test time

Select environment

Verify availability

Reserve

SetupTesting

activities

Collaborate

Teardown

• “One-Click” is first step toward continuous• Self-service users can easily launch entire certification from UI• Integrate with ALM to ensure consistent reporting

• Certification can be reused for many stages:• New technology and early stage service proof of concept

testing• New service introduction• Vendor interoperability testing• Contractor code qualification

Step 3: Create “One-Click” Certification

Tight Integration between TestShell & CloudShell

Create an Environment Develop a Test

Create the Automation

Suite

Execute and Monitor the

Suite

Develop test in TestShell Studio in conjunctionwith a live CloudShell environment for greater accuracy,easier debugging

Suite = reservations & associated tests

What Happened when a Cable MSO Built an “As-a-Service” ApproachOverall: Complete change in the way devtest team conducted business

Measured:• 15% annual operations cost savings• 20% immediate reduction in certification cycles• 33% immediate increased in certification coverage and improving• 100% increase in resource utilization due to 24/7 service availability

Ongoing observations:• Earlier certification by internal departments and vendors moves quality to the left in

the cycle—still measuring outcomes• Eliminated need for extensive travel time and costs on part of vendors and

contractors• Increased compliance with their methodology and standards via automation• Increased pace of innovation

Agenda




27

High Level OPNFV

Summary

| Automation of both physical and virtual is key to leveraging the opportunity for agility that NFV represents

| On-Demand Test Environments and Lab-as-a-Service enable DevOps agility for NFV validation and certification

| Adopting new technologies like On-Demand Environments with Test Automation will enable your organization to maximize revenue and speed innovation

| Testing in the new world of NFV in general and OPNFV in particular, changes dramatically and becomes a core discipline that is central to the overall goal of pleasing customers in a cost effective way

| DevOps improve times to NFV solution implementation

| NFV Testing is not only matter of technology, it’s involves cultural changes and testing is the appropriate change agent

28

29www.QualiTestGroup.com/NFV | www.qualisystems.com

Visit us at

November 9-12, San Francisco, CA

http://www.qualitestgroup.com/NFV

http://www.qualisystems.com/

www.QualiTestGroup.com

Technology

Ensuring Successful OPNFV-based NFV Deployments | QualiTest Group