OPNFV - From The Trenches · © Cable Television Laboratories, Inc. 2015. 1 Randy Levensalor Lead...

1© Cable Television Laboratories, Inc. 2015. © Cable Television Laboratories, Inc. 2015.

Randy Levensalor Lead Architect / Business Technologies

OPNFV - From The Trenches

2© Cable Television Laboratories, Inc. 2015. © Cable Television Laboratories, Inc. 2015.

• Purpose of the Lab Install

• Challenges Encountered – Networking – Build server – Debugging – Repeatability

Agenda

2

3© Cable Television Laboratories, Inc. 2015. © Cable Television Laboratories, Inc. 2015. 3

About CableLabs

CableLabs •  non-profit research and

development consortium funded by MSO’s (Multiple System Operator)

•  dedicated to creating innovative ideas that significantly impact our cable operator members’ business

•  serves to define interoperable solutions among our members and their technology suppliers in order to •  drive scale •  reduce costs •  promote innovate solutions

.

4© Cable Television Laboratories, Inc. 2015. © Cable Television Laboratories, Inc. 2015.

Lab Details

4

Virtual Network Functions

Compute Node #2

•  Lab 1 –  Install

•  Stable release •  Foreman installer

–  Planed use •  VNF development •  Regression testing

•  Lab 2 –  Install

•  Top of tree •  APEX installer

–  Planed use •  Installer •  Infrastructure

HA HA

ODL Control Node #1 Control Node #2 Control Node #3

Build Server

Virtual Network Functions

Linux Compute Node #1

Jump Server

5© Cable Television Laboratories, Inc. 2015. © Cable Television Laboratories, Inc. 2015. 5

1. Preparing the Lab

Foreman Fuel

Installers Initial Network Setup

30 +Cables & 6 VLANs

Knowledge of OPNFV

6© Cable Television Laboratories, Inc. 2015. © Cable Television Laboratories, Inc. 2015. 6

1. Preparing the Lab - Solution

• Detailed port and VLAN mapping spreadsheet and diagram

• Foreman Specific

• Suggestion: Pay extra for blades

7© Cable Television Laboratories, Inc. 2015. © Cable Television Laboratories, Inc. 2015. 7

2. Violating build server

• Install must run on bare metal • Must run as root • Install is not immutable • VirtualBox failures

8© Cable Television Laboratories, Inc. 2015. © Cable Television Laboratories, Inc. 2015. 8

2. Build server solutions • Separate build and jump server for security

• Updated install directions and .iso

• Automated workarounds • Improve error reporting

• Suggestions: Report Bugs & Submit Patches

9© Cable Television Laboratories, Inc. 2015. © Cable Television Laboratories, Inc. 2015. 9

3. Install failed…somewhere • Failures after Foreman VM starts are difficult to trace

Deploy.sh ->

vagrant up ->

bootstrap.sh ->

Ansible ->

run.sh ->

Puppet

Foreman ->

pxe ->

10© Cable Television Laboratories, Inc. 2015. © Cable Television Laboratories, Inc. 2015. 10

3. Solution

• Check release notes and FAQ

• Look at reports on Foreman • Still can be difficult to trace

• Suggestion: Engage mailing list with problems

11© Cable Television Laboratories, Inc. 2015. © Cable Television Laboratories, Inc. 2015. 11

4. Reliability Concerns

• Install pulls scripts from multiple git & yum repos

• Failures are catastrophic • Can’t install without Internet access

Reliability

Stable

12© Cable Television Laboratories, Inc. 2015. © Cable Television Laboratories, Inc. 2015. 12

4. Suggestions for Reliability

• Check IRC, mailing list and Jenkins (you are not alone)

• Manually reproduce • Still not ideal

13© Cable Television Laboratories, Inc. 2015. © Cable Television Laboratories, Inc. 2015. 13

5. It’s running…or is it?

• Installs are unstable • Only one VM started • Can’t delete networks • Corrupt data bases

14© Cable Television Laboratories, Inc. 2015. © Cable Television Laboratories, Inc. 2015. 14

5. Suggestions for Unstable Install

• Check the release notes and FAQ • Arno is based on old versions of OpenStack and OpenDaylight and not all critical patches have been back ported yet

• Reinstall / automate post install setup and have a backup

15© Cable Television Laboratories, Inc. 2015. © Cable Television Laboratories, Inc. 2015. 15

Takeaways

• Lots of help – Wiki https://wiki.opnfv.org/ – Dev mailing list

https://wiki.opnfv.org/developer/getting_started#mailing_list

– Direct with the developers

• Engage users early & often • Run from a clean install • Standardize documentation

© Cable Television Laboratories, Inc. 2015.