Embed Size (px)
Citation preview
OpenStack - based NFV Cloud at Swisscom
Challenges and Experiences
Agenda
– Swisscom IPTV
– A story of Innovation and Success
– E2E Signal Chain
– E2E Service Orchestration Vision
– Lessons Learned
– Swisscom-HPE virtual headend project
– Legacy vs virtual headend
– Maturity levels of applications
– Enterprise Cloud vs. Media processing platform
– Virtual Network performance and Container Networking
– Multicast on OpenStack
– AVI Networks: Elastic Application Services
– A distributed architecture
– Application Services
– Elastic scale
– Demo
Marco Loetscher
Hewlett Packard Enterprise
Solution Architect
Ranga Rajagopalan
AVI Networks Inc.
CTO
Mihajlo Zivkovic
Swisscom (Schweiz) AG
Product Owner
3
Swisscom TV & Entertainment
2008 20102009 2011 2012 2013 2014 2015
1.5 Mio. Customers
2 Mio. STB’s in the field
20072006
3.4 Mio. Households
8 Mio. Population
2Mio. on Swisscom DSL
2016 2017
Swisscom is a Marketleadersince December 2015
4
Swisscom TV Some Facts and Figures
..more then 2 Mio STB’s
> 650 Channels (SD, HD, UHD)
..more then 1.5 Mio Customer
25 Pbyte Storage / Year
> 2.8 Tbit/s Unicast Streaming
peak from Datacenter
>90 Pbyte of Storage
> 750K unique user per month
on Web & Mobile
Customer Satisfaction > 8
5
6
The Journey of Swisscom TVIt’s a Story of Innovation and Success
201216. January
Launch Replay TV
2014Decembar
Launch Teleclub Play
(Subscription VOD)
2014April
Launch TV 2.0
2015Decembar
Marketleader
Switzerland
2016April
Voice Search with
Swiss Dialects
UHD
20061. November
Launch Bluewin TV
200925. February
Start HD-Sender
Launch TV1.0
201023. March
Swisscom TV Air
201130. June
Marketleader Digital TV
201328. Decembar
1 Million Customers
2015April
HBB TV
2017November
Mediaroom
Phase-out
Entertainment
0S3
Next Generation Entertainment & TV Services Swisscom TV E2E Signal Chain
7
Next Generation Entertainment & TV Services TV Platform Virtualization
1st step virtualization
2nd step virtualization
3rd step virtualization
8
MediaCloud
9
MediaCloud: The Big PictureGoal to have all Entertainment Service on the MediaCloud
Virtualization Layer / RedHat OpenStack / KVM
CPU
Hardware
Virtual
Network
Memory
Hardware
Storage
Hardware
Network
Hardware
Virtual
CPU
Virtual
Memory
Virtual
Storage
vHE
Service Orchestration
Radio AVI AppBE nPVR LiveTV LivingDocs xyz
VNF Mgr VNF MgrVNF MgrVNF MgrVNF Mgr VNF Mgr VNF Mgr VNF Mgr
Channel/Service Definition
Bussiness process workflow
Mo
nit
ori
ng
an
d S
erv
ice
Mo
nit
ori
ng
Media Cloud - Virtual HeadEndLessons learned
Organizational challenges:
– Vendor management (working mode with Vendors).
– Mindset of the project staff.
– Transforming engineers into the new "cloud" world.
– Operation of two environments (legacy and cloud).
– Knowledge and the experience (combined application with infrastructure and network)
– Organization and processes .
Technological challenges:
– Working on the bleeding edge of the technology.
– Technology unknowns.
– Infrastructure : very large high performance media cloud solution.
– Zero packet loss in cloud environment with the multicast.
10
11
Media Function Virtualization
Current headend operations challenges
VideoSignal
DedicatedManagement
Transcoders AManager
Transcoders BManager
Transcoders CManager
MultiplexerX
Multiplexers YManager
Multiplexers ZManager
Transcoder A
Transcoder A
Transcoder A
Transcoder B
Transcoder B
Transcoder C
Ch. 1
Ch. 2
Ch. 3
Ch. 4
Ch. 5
Ch. N
MultiplexerY
Multiplexer Z
MultiplexerX
Transcoder A
Transcoder A
Transcoder A
Transcoder B
Transcoder B
Transcoder C
Ch. 1’
Ch. 2’
Ch. 3’
Ch. 4’
Ch. 5’
Ch. N’
MultiplexerY
Multiplexer Z
Multiplexers XManager
Backup headend
Active headend Too many managers to manage
Disaster Recovery expensive
Costly Equipment Upgrades/Exchanges
Operational complexity
Missing Flexibility to add new channels
Current situation
Switch STB
CDN
12
Virtualized headend with HPE VHM
1 click channel deploymentOne interfaceto manage it all
HPE Virtual Headend Manager
MultiplexerX
Transcoder A
Transcoder A
Transcoder A
Transcoder A
Ch. 1
Ch. 2
Ch. 3
Multiplexer X
Virtualized Headend
Transcoder B
Transcoder B
Ch. 4
Ch. 5Multiplexer
Y
Transcoder B Multiplexer Y
Transcoder CCh. N
Multiplexer Z
Backup
Backup
Transcoder D Multiplexer WChannelson the fly
Switch STB
CDN
Fast time to launch new service One click channel deployment Non-proprietary infrastructure
Quick and automated lifecycles Simplicity: One interface for all CAPEX and OPEX reduction
VideoSignal
13
Virtual Headend
NFV
Orchestrator
EMS1
VNF1
EMS3
VNF3
EMS2
VNF2
VNF Manager(s)
NFV MANO
Virtualized
Infrastructure
Manager(s)
NF
VI
Computing
Hardware
Network
Hardware
Storage
Hardware
Hardware Resources
Virtualization Layer
OSS/BSS
Virtual
Computing
Virtual
Storage
Virtual
Network
Service, VNF and Infrastructure
Description
NS and VNF Catalogs
Standard Infrastructure
Virtualization Layer (HyperV, KVM, ESX)
Transcoder Muxer Probe
Service Orchestration
VNFM
Infrastructure
Orchestration
ETSI Reference Architecture
14
Maturity Levels of virtualized applications
Monolithic functions are decomposed into micro-services
Functions separated from underlying hardware
Functions deployed on hypervisor-driven, virtualized infrastructure resources
Operated as part of the cloud, lifecycle fully orchestratable
Decoupled
Virtualized
Cloudified
Decomposed
Market View R&D View
Bare Metal x86
Microservices
Automate / Orchestrate
Virtualize
15
Enterprise cloud vs. virtual Headend workloads
1. CPU, RAM or Storage bound performance
2. Aggregated view of resources(CPU, Memory, resources overcommitted)
3. Endpoints(Applications need the OS)
4. Many and small virtual machines
1. CPU & I/O bound performance(DPDK, SR-IOV, etc.)
2. Enhanced platform awareness(Internal Architecture relevant for guests)
3. Middlepoints(Data-plane network bypass the OS)
4. Fewer and larger VMs
IT Cloud Virtual Headend
16
Key network performance and quality features
PCI-passtrhough✓https://docs.openstack.org/admin-guide/compute-pci-passthrough.html
SR-IOV (Single Root I/O Virtualization)✓http://en.wikipedia.org/wiki/X86_virtualization#PCI-SIG_Single_Root_I.2FO_Virtualization_.28SR-IOV.29
✓https://wiki.openstack.org/wiki/SR-IOV-Passthrough-For-Networking
NUMA (Non Uniform Memory Access)✓http://en.wikipedia.org/wiki/Non-uniform_memory_access
DPDK (Data Plane Development Kit)✓http://en.wikipedia.org/wiki/Data_Plane_Development_Kit
17
Containers in VMs vs. Baremetal
• The transcoding applications from Ateme and
Harmonic are provided in Docker containers
• We decided to put the containers into a virtual machine
• Why?
• Docker Networking
• Hardware independence
• VM as “bridge” between application and physical
environment
• Management and Monitoring
• Virtualization Overhead: 5-10 %
• Additional Component (Guest OS) to be managed
• Cost of Hypervisor Physical Server Physical Server
Operating System Operating System
Docker Engine
App 1
Hypervisor
App 1 App 1
App 1 App 1 App 1
18
Docker Networking
–net=bridge:
• Container is connected with the Network Bridge
• Same IP addresses per Host, 1 NIC only
• Standard Mode
–net=none:
• No network connection of the container (i.e. for number
crunchers, batch jobs, etc.)
–net=container:<CONTAINER | ID>:
• Container is using the NW connection of a “neighbor”
• Allow reachability of multiple segregated processes that
under the same IP
–net=host:
• Container has direct access to physical NICs of the hos
19
Multicast in OpenStack
• No multicast available out-of-the-box
• Virtual layer-2 switches support IGMP snooping:
– Open vSwitch 2.5 supports IGMP snooping
– Open vSwitch 2.7 contains bugfixes to enable multicast with multiple provider VLANs
–http://openvswitch.org/features/
–https://github.com/openvswitch/ovs/blob/master/FAQ.md
– Linux Bridge 2.4 supports IGMP snopping
–http://www.linuxfoundation.org/collaborate/workgroups/networking/bridge#Snooping
20
Multicast in OpenStack – recommendations
• Neutron provider network:
• Don’t use Neutron-L3-Agent
• Neutron does not route any multicast traffic
• Use Neutron provider network
• Use routable public VLAN configured on top-of-rack switches
• Use router outside OpenStack
• Use gateway outside OpenStack
• Attach Nova instances directly to provider network
• Configure multicast routing / PIM on the physical router
21
22
AVI Networks
Monolithic Appliances to Modern Distributed Architecture
23
CONTROLDATA
Separate Control & Data
PlaneManage as one, not many devices
APPLICATIONS
Controller
Monolithic Appliance SoftwareManagement Plane: UI/CLI
Data Plane: LB
Service Engines
Avi Platform – Modern Distributed Architecture
24
CONTROLDATA
Universal SolutionBoth traditional and modern use cases
Service Engines Controller
Public Cloud
Bare Metal Virtualized Containers
On Premises
Separate Control & Data
PlaneManage as one, not many devices
VisibilityActionable insights key to automation
10x Performance• 4 Tbps• 12M SSL TPS
ElasticityOn-demand scalability up / down
REST API
AutomationHighly programmable, Plug-n-
Play
25
Application Services
Out-of-Box Automation
CONTAINERS SDN OPENSTACK AUTOMATION ON-PREM or OFF
Load BalancingL7 (HTTP) LB
L4 (TCP/UDP) LB
Global Load
Balancing
Content Switching
Caching/Compression
Auto-Scaling
WAF & SecurityWeb app firewall
(WAF)
SSL Termination
DDoS Protection
L3-4 ACLs
L7 Rules/Policies
Micro-Segmentation
AnalyticsApplication map
Service Health Score
Network performance
App Performance
Request Logging
Security Insights
MESOS
Bare Metal
PlatformCentral Management
100% REST API / SDK
Self-Service
Multi-Tenancy
Service Discovery
IPAM/DNS
Comprehensive Services – For All Major Environments
26
Record Performance – On Commodity ServersSingle Fabric – Central Management and Monitoring
• 32 cores• 80,000 SSL TPS
Performance scales with cores (Moore’s Law)
• 2 core• 5,000 SSL TPS
Avi Fabric Performance• 4 Tbps• 12M SSL TPS
• 2x 1-core SEs• 5,000 SSL TPS Fabric performance
scales with Engines
Centralized API, Mgmt, Monitoring
ECC- 48K, RSA 80K
• 1 core• 2,500 SSL TPS
VIPRION 4800, ~1M TPS
27
We are happy to answer questions
