Building Multi-site & Multi-OpenStack Cloud

with OpenStack Cascading

Chaoyi Huang Huawei Technologies

Jiongjiong Gu Huawei Technologies

Alexis Susset Vodafone Group

Last update Jan.12, 2016

Page 1

Agenda

Requirement and driving forces

Vodafone Case Study

Solution proposal of OpenStack cascading

Live Demo

Page 2

Consolidation: The essence of Cloud

Benefits by higher level of consolidation :

- Improved business Agility & Management Simplicity

- Larger resource pooling and optimized Resource utilization

- Enlarged range of resource automation

Cloud OS

(Openstack)

App1 App2 Appn

…

Isolated Resource & Data Silos Dedicated for certain App Consolidated resource & data pooling Shared by Multiple Apps

…

OS1 OS2 OSn

Big

Data

DC1 DC2 DCk…

OSn

Appn

DB

Server

Storage

Network

OSn

Appn

DB

Server

Storage

Network

OSn

Appn

DB

Server

Storage

Network

…

DC1 DCk

Cloud

Transforming

Page 3

Multi-site Openstack Consolidation with unified Openstack API

Due to maturity and difference on service cloud transformation ,

Openstack is normally built with standalone multiple DC sites in a

step by step way;

How to achieve unified orchestration & automation across multiple

Openstack & DC sites?

How maintain the continuity of OpenStack ecosystem?

OpenStack1

Host1 Hostk…

DC1

OpenStack2

Host1 Hostj…

DCn

DC Mngt. …DC mngt. Cloud App…

…

Openstack EcosystemOpenstack Ecosystem

Openstack APIOpenstack API

Cloud App

OpenStack1

Host1 Hostk…

DC1

OpenStack2

Host1 Hostj…

DCn

…

Openstack API

DC Mngt. …

Openstack Ecosystem

Cloud App

OpenStack1

Host1 Hostk…

DC1

OpenStack2

Host1 Hostj…

DCn

…

Proprietory API?

Page 4

Multi- Openstack PoD consolidation within the same DataCenter

Multiple modular “Openstack Cloud in Box”

consolidated into larger resource pooling;

Expanding the Openstack resource pooling by means

of adding more modular pre-integrated pooling to the

system;

OpenStack Pooling

(100 -> 1000 Hosts)

Openstack API

OpenStack1

Host1 Hostk…

…

Consolidation &

Orchestration

Openstack API

Cloud Portal

& Orchestration

3rd party / Existing

DC mng.t

Host1 Host100…

Host1 Host100…

OpenStack Pooling

Openstack API

Hostk

OpenStack-n

Host1 Hostj…

Pre-integrated Pooling

Hostn…

Page 5

Overlay network automation for VDC deployed across multiple DCs

Workloads of the same cloud tenants might span across multiple Openstack / DC sites, but

network connectivity still need to be automated rather than configured manually

Cross DC L2 connectivity Cross data center L3 networking Cross DC traffic optimization

DE

VM1 VM2 VM3 VM4

UK

Subnet1

Virtual Router

DE

VM1 VM2 VM3 VM4

UK

Subnet1

R

VR VR

Subnet2

DE

VM1 VM2 VM3 VM4

UK

Subnet1

VR VR

Subnet2

External Network External NetworkPluggableExternalNetwork

PluggableExternalNetwork

**DVR – distributed virtual router

Page 6

Challenges of deploying Single Openstack across multi DC sites

No fault isolation zones, data center is in risk of out of management if controller failed, no API/CLI

available

Difficult for cross-DC integration via RPC for multi-vendor cluster

Challenge for trouble shooting/configuration change/rolling upgrade/patching… cross data center via RPC.

No clear responsibility boundary

Risk of vendor-lock in for one OpenStack instance

Challenge for performance tuning cross data center via RPC…

Nova/Cinder/Neutron

Controller

Nova/Cinder/Neutron

Agent/Driver

Message Queue

DC1

Nova/Cinder/Neutron

Agent/Driver

WAN

DC2

Nova/Cinder/Neutron

Agent/Driver

Page 7

Agenda

Requirement and driving forces of multi-site cloud

Vodafone Case Study

Solution proposal of OpenStack cascading

Live Demo

Page 8

47 OpCOs and counting ..

Mostly Acquired: Each

site can use one or multiple Vendors

which leads to better use of local

resources and capabilities

Diverse: Any site can connect

to each other, therefor establishing

globally addressable tenants which

result in global services

Local relationships: Each site with its own pace and

requirements while maintaining

standard OpenStack APIs

Page 9

Vodafone use-case, World-wide OpenStack Orchestration

Multi-Vendor : Each site

can use one or multiple Vendors

which leads to better use of local

resources and capabilities

Global Networking: Any

site can connect to each other,

therefor establishing globally

addressable tenants which result in

global services

API Standard and

Multi-Version co-

existence: Each site with its

own pace and requirements while

maintaining standard OpenStack APIs

OpenStack APIs

Global Standard – Global Access

Page 10

Agenda

Requirement and driving forces of multi-site cloud

Vodafone Case Study

Solution proposal of OpenStack cascading

Live Demo

Page 11

Requirement summary for multi-site cloud

Multi-site, Multi-Vendor OpenStack distribution, Multi- OpenStack instance, Multi- OpenStack Version

Multi-vendor: anti-vendor lock in business policy.

Multi-instance: each vendor has his own OpenStack solution distribution

Multi-version: stepwise cloud construction, upgrade gradually

Restful open API /CLI for each site

OpenStack API in each site: Open, de facto standard API

make the cloud always workable and manageable standalone in each site

each site installation/upgrade/maintenance independently by different vendor or cloud admin.

Ecosystem friendly open API for the unified multi-site cloudEcosystem friendly open API : It takes almost 4 years for OpenStack to grow the eco-system, the

OpenStack API must be retained for distributed but unified multi-site cloud.

Cross DC orchestration: globally addressable tenants which result in global services. tenant virtual

resources will be distributed in multi-site but connected by L2/L3 networking

Page 12

Why not use Openstack itself as

multi-site/multi-Openstack orchestration layer?

OpenStack

( multi-site orchestrator )

…

OpenStack API

OpenStack APIOpenStack API

AZ1 AZn

DC 1 OpenStack DC n OpenStack

(Vendor1 /

Version 2.0)

(Vendor x /

Version 2.1)

Openstack API & Ecosystem kept untouched after multi OpenStack consolidation

multi-site orchestration become a part of OpenStack upstream which will benefits the whole

industry rather than specific vendor

Page 13

Good idea, but is it feasible?

OpenStack

…OpenStack OpenStack

OpenStack APIOpenStack API

Controller Node

Computer … Computer

OpenStack is generally used to

manage lots of compute nodes

OpenStack API

RPC RPC

Want to use OpenStack to orchestrator

OpenStacks?

?

Intuition - mapping the underlying OpenStack to a

compute node

OpenStack API

Page 14

Magic happens by just consider Nova/Cinder/Neutron

as his own backend !

Neutron Server

OVSAgent

Cinder API

Rabbit-MQ

Cinder

Volum

e

Cinder

Volum

e

Nova API

RabbitMQ

Nova

ComputeNova

Compute

Libvirt

DriverNova

Driver

LVM

Driver

Cinder

Driver

KVM Nova LVM Cinder

RabbitMQ

OVS

L3Agent

Linux

Router

Scheduler

Scheduler

Neutron

Agent

Neutron

**Architecture simplified for illustration onlyFor Ceilometer and Glance, please refer to wiki link in the slides…

Page 15

Concept and architecture – OpenStack cascading solution

DB

Message Bus

Nova-API

Nova-Scheduler

Nova-ConductorDB

Message Bus

Cinder-API

Cinder-Scheduler

DB

Message Bus

Neutron-API

Neutron-Plug-in

DB

Message Bus

Nova-API

Nova-Scheduler

Nova-ConductorDB

Message Bus

Cinder-API

Cinder-Scheduler

DB

Message Bus

Neutron-API

Neutron-Plug-in

Cascaded OpenStack 1 Cascaded OpenStack x

…

Controller NodeCompute 1 Compute n

…

Compute 1 Compute n

DB

Message Bus

Nova-API

Nova-Scheduler

Nova-ConductorDB

Message Bus

Cinder-API

Cinder-Scheduler

DB

Message Bus

Neutron-API

Neutron-Plug-in

Cascading OpenStack

Nova-API Cinder-API Neutron-API Nova-API Cinder-API Neutron-API

Controller Node Compute x

Nova-API Cinder-API Neutron-APIIntroduced for

OpenStack

cascading

solution

Nova-P

roxy

Cin

de

r-Pro

xy

L2-P

roxy

L3-P

roxy

LB

-Pro

xy

VP

N-P

roxy

No

va-P

roxy

Cin

de

r-Pro

xy

L2-P

roxy

L3-P

roxy

LB

-Pro

xy

VP

N-P

roxy

FW

-Pro

xy

FW

-Pro

xy

Compute 1

VM

VM

VM

VM

Controller Node

VM

VM

VM

VM

VM

VM

VM

VM

VM

VM

VM

VM

Page 16

OpenStack cascading works like a fractal

Fractal: Recursive self-similar, growth to scalehttp://ethshar.wikia.com/wiki/File:Tricircle_fractal.gif

Cascading: Also recursive self-similar, growth to scale. Just treat the child OpenStack as a super huge compute

Page 17

For example, cross OpenStack/DC L2 VxLAN

networking

Neuton-API

L2/L3-Proxy

L2/L3-Proxy

Neutron Neutron

VM1 VM2

VxLAN0

VM3 VM4

AZ1 AZ2

VM1 VM2

VxLAN0 DVR

VM3 VM4

VxLAN0

DVR

DVR

2.Periodic polling port

status( for example

VM2 port)

3. VM2 Port

status up

4. L2 population

5. fdb_add ( Port for VM2 IP / VM 2 mac / Host IP )

6. Create virtual remote Port for VM2

(with VM2 IP / VM2 mac / VM2 host IP)

VM2

7. Internal L2 population for

virtual remote port for VM2

Virtual remote port

Cascading OpenStack

Cascaded OpenStack

1. Internal L2 population for VM2

If VM2 was added to VxLAN0…

Self similar L2 population

happened in the cascading level

Page 18

Benefit summary of OpenStack cascading •Open-source multi-site cloud orchestration layer with OpenStack API

•Fault isolation AZ by AZ

•Reduce Upgrade/O&M challenge in AZ granularity

•Plug & Play Fast integration by OpenStack API

•No vendor-lock in, Fast Integration for multi-vendor/multi-site/multi-version cloud infrastructure

•Scale out architecture. Horizontal scalability, even cross multiple data centers

•Expansion with replication of AZ by AZ

OpenStack

Vendor1

OpenStack2.0

OpenStack API

OpenStack API

…

DC1

Availability

Zone 1

Vendor n

OpenStack2.1

Vendor2

OpenStack2.

0

Vendor1

OpenStack2.

1

OpenStack APIOpenStack APIDCn

OpenStack API

Availability

Zone 2

Availability

Zone 3

Availability

Zone n ***One AZ can still integrate heterogeneous

hypervisors and heterogeneous physical

resources

…… … … …

OpenStack cascading solution makes the distributed but unified multi-

site cloud always workable and manageable standalone in each site.

Page 19

Other benefits: Building ultra-large scale Cloud with 1 million VMs

OpenStack …

…

1 2 100

1 2 1000

OpenStack

…1 2 1000

OpenStack

…1 2 1000

OpenStack

Max. 100 data centers

Max. 100k server nodes

Max. 1 million VMs

We are working to make it happen, kindly invite you to join us…

Page 20

Agenda

Requirement and driving forces of multi-site cloud

Vodafone Case Study

Solution proposal of OpenStack cascading

Live Demo

Page 21

Use case1: Consolidating new OpenStack to the Cloud

Cascaded

OpenStack

Xi’an Hangzhou

Shenzhen

Shenzhen Hangzhou Xi’an

Dynamically

adding New

OpenStack

Page 22

Use case2: Single VDC deployed across multiple DCs

Cascading

OpenStack

Shenzhen DC (AZ1) Xi’an DC (AZ2)

Step1 : Creat Network (VxLAN ID1)

VM 1

VxLAN

Step2 : Creat VM(VM1 (VM2/VM3), VM Image1, AZ1, VxLAN ID1)

Step3 : Creat VM (VM4, VM Image2, AZ2, VxLAN ID1)

VM 2 VM 3

VxLAN ID1 VxLAN ID1 VxLAN ID1

Shenzhen Branch office

VM 4

Xi’an Branch Office

Page 23

Use case 3: Zero interruption DR across DCs

Cascading

OpenStack

Shenzhen DC Xi’an DC

vIMS

active

Step1.1 : Creat Volume (A, vIMS Image, AZ1)

Step1.2 : Creat Volume (B, vIMS Image, AZ2)

A B

Step2: Creat Shadow Volume (A, B)

Step3: Creat network (VxLAN1 & VxLAN2)

Step1 : Detach Volume (vMS active, VolA)

DR Preparation: DR execution:

Step2 : CreateVM (vIMS standby, VolB, AZ2,

VxLAN1, VxLAN2)

VxLAN ID1 vIMS

Standby

LB for vIMS

XData Replication

Openstack APIStep4 : Creat VM (vIMS active, VolA, AZ1, VxLAN1, VxLAN2)

Page 24

Use case4: VM Mobility across multiple DC

Cascaded

OpenStack

Shenzhen DC Xi’an DC

Step1.1 : Creat Volume (VolA, VDI Image, AZ1)

VDI 1VDI 4

Step1.2 : Creat Volume (VolB, VDI Image, AZ2)

A B

Step2: Creat Shadow Volume (A, B)

Step3 : Creat VM (VDI1, VolA, AZ1, VxLAN 1)

Step1 : Detach Volume (VDI1, VolA)

Before migration: Perform migration:

Step2 : CreateVM (VDI1, VolB, AZ2, VxLAN 1)

VDI3

VDI2

VxLAN ID1

Data Replication

VxLAN ID1VxLAN

Page 25

Welcome to join the design summit session:

Time: Tuesday November 4, 2014 14:50 - 15:30

Room: Manet

Topic: Approaches for scaling out

Page 26

Visit Huawei booth for live demo

More information:[1] wiki: https://wiki.openstack.org/wiki/OpenStack_cascading_solution

[2] source code: https://github.com/openstack/tricircle

[3] wiki: https://wiki.openstack.org/wiki/Tricircle

[4] Live demo video at YouTube: https://www.youtube.com/watch?v=OSU6PYRz5qY

THANK YOUwww.huawei.com

Copyright©2014 Huawei Technologies Co., Ltd. All Rights Reserved.

The information in this document may contain predictive statements including, without limitation, statements regarding the

future financial and operating results, future product portfolio, new technology, etc. There are a number of factors that could

cause actual results and developments to differ materially from those expressed or implied in the predictive statements.

Therefore, such information is provided for reference purpose only and constitutes neither an offer nor an acceptance. Huawei

may change the information at any time without notice.