Protecting Enterprise Workloads Within a vCloud

Service Provider Environment

Ian Perez Ponce, VMware

PHC5679

#PHC5679

2

Abstract

With the proliferation of vCloud service providers and the increased rate of

enterprise workload deployments on vCloud Director, the ability to protect

business critical application data in the cloud continues to surface as a key

priority for Business Continuity / Disaster Recovery (BC/DR) compliance. This

session is intended for both technical and business decision makers and aims to

demystify many of the challenges associated with implementing a self-service

and policy-driven data protection service that can scale with the most

demanding vCloud powered Infrastructure-as-a-Service (IaaS) environments.

3

Agenda

Disaster Recovery and Cloud – Misguided Assumptions

Service Layering Options for Disaster Recovery

vCloud Service Provider Environments

vCloud Director API Extensibility

Disaster Recovery Solutions Available for vCloud Director

vCloud Hybrid Service – A Real-World Use Case

References

4

Glossary of Frequently Used Terms (1 of 2)

Backup: A collection of data stored on (usually removable) non-volatile

storage media for purposes of recover in case the original copy of data is

lost or becomes inaccessible

Disaster Recovery (DR): The recovery of data, access to data and

associated process through a comprehensive process of setting up

redundant site (equipment and work space) with recovery of operational

data to continue business operations after a loss of use of all or part of a

data center

Digital Archiving: A storage repository for service used to secure,

retain, and protect digital information and data for periods of time less

than that of long-term retention

Digital Long Term Preservation: [Long Term Retention] Ensuring

continued access to, and usability of, digital information and records,

especially over long periods of time

Source: Storage Networking Industry Association

5

Glossary of Frequently Used Terms (2 of 2)

Recovery Point Objective: [Data Recovery] The maximum acceptable

time period prior to a failure or disaster during which changes to data

may be lost as a consequence of recovery.

Recovery Time Objective: [Data Recovery] The maximum acceptable

time period required to bring one or more applications and associated

data back from an outage to a correct operational state.

Restore Granularity Objective: The level at which a particular

restore/recovery operation takes place within the data environment.

• Application/file

• OS/image

Last Backup

Time

Event Data Restored

RPO RTO

Source: Storage Networking Industry Association

6

“Procrastination is the Foundation of All Disasters” -Pandora Poikilos

7

Misguided Assumptions about Data Protection and Cloud

Your cloud service provider is

performing regular backups on

your behalf.

Your cloud service provider is

storing your data in

geographically redundant

locations.

Your cloud service is

maintaining a hot site

somewhere with a duplicate

copy of your application data.

Your cloud service provider in

charge of your disaster

recovery plan.

TRUE

FALSE

TRUE

FALSE

TRUE

FALSE

TRUE

FALSE

8

Misguided Assumptions about Data Protection and Cloud

Hint…. Make ZERO assumptions!

YOU alone are responsible for your Business Continuity / Disaster

Recovery plan

When in doubt, ASK your cloud service provider the basics

regarding data protection:

• What types of service levels are guaranteed during a disaster?

• Are BC/DR plans and planning documents available for audit?

• Where (if any) are your recovery centers located?

• What happens to my data when single-site failures occur?

• What guarantees are in place to ensure my data will not be moved outside of

my country/region in the event of a disaster?

• If data protection services are offered, what is the recovery time objective

(RTO) and recovery point objective (RPO) guaranteed?

• How resilient are your data center facilities (i.e. Tier III or IV)?

9

Service Layering for Disaster Recovery and Disaster Avoidance

Various solution options may

be available for a layered DR

approach

A One-size fits all DR

methodology rarely applies

• Carefully evaluate options available

and match to your actual business

needs – not the provider’s

Make the effort to consolidate

and rationalize BC/DR

compliance standards between

cloud models:

• Private

• Hybrid

• Public

Redundant Cloud Service Providers

Redundant deployment across regions

Storage snapshots and/or inline replication

Data backup and archival

A

B

10

vCloud Service Provider Environments – Things to Know

vCloud Director includes

multiple constructs over

vSphere that require special

attention

Each construct configuration

bears potential impact on the

recoverability of infrastructure

services

Special consideration for DR

planning should include:

• Backing up of vCloud Director cells

• Object mapping via API for Org

VDC vApp VM relationship

• Coherence between vApp

metadata and member virtual

machines

VMware vSphere

VMware vCenter Server

VMware

vShield

Virtual Datacenter 1 (Gold) Virtual Datacenter n (Silver)

User Portals Security

VMware vCloud Director

Catalogs

Users IT

Organization 1 Organization m

11

Cloud API Framework: Build New Services to Expand Your Cloud

Extend the vCloud API with your

own *aaS offerings;

Leverage new Cloud Extensions

from the VMware ecosystem

Integrate provider’s value-added

services with vCloud Director

Overview

Backup, DR, Patch Management,

Database, Load Balancing,

Compliance - all delivered as a

service

Let vCloud Director take care of

logging, events, multi-tenancy,

security, and APIs for your service

Single point of control and

governance for Cloud APIs

Capabilities

An Extensible API Entry Point

Cloud Platform Services

provide common building

blocks

Install and Manage Cloud

Extensions

12

Existing Ecosystem of ISVs

APIs have been

critical to our

success…

but as our

products grew,

so did the APIs

$15 of ecosystem value created

for every $1 of vSphere licensing

Administrative SOAP Based

vSphere API built using VMODL

toolkit covers vSphere features

and ESXi

Increasingly successful vCloud API

is REST based and does not use

VMODL

vShield uses REST Based API

Large number of SDKs built in EE

targeting different aspects of

vSphere, vCloud, and vShield for

different native language bindings

We have to provide better ease

of use and integration!

APIs Play a Critical Role in Enriching Cloud Services

13

Overview of vCloud Director API Extensibility Services

The vCloud API Extensibility Framework is a set of features to

enable the construction and delivery of cloud services (XaaS APIs)

integrated with vCloud Director. It contains three elements:

1. An extensible API entry point- This enables a customer to interact with

with an Cloud Service as a part of the vCloud API.

2. APIs for Management and Operations of API extensions - This

enables an administrator to manage the vCloud API Extensions.

3. Provider-side APIs for the Cloud Platform Services - This enables the

author of a Cloud Service to use key (previously internal-only) features of

vCloud Director when building their Service, and enables a vCloud

Provider Admin to get a consistent administrative experience across all

vCloud Services, whether those shipped with Cloud Director or those

offered as an Extension.

14

An Extensible API Entry Point

• This is the piece that an API client actually talks to. It takes care

of routing API messages to and from Services.

• It consumes a formalized definition of the API extension, and

routes requests and responses to the implementation of the extension.

Compute,

Network,

Storage -aaS

Cloud API

Service 2

Cloud API

Service 1

API Client

vCloud

API Entry

Point

15

Management and Operations of API Extensions

The vCloud API Framework provides the ability to…

• Register a new API service with vCloud Director

• List/query available services

• Enable/disable services

• Control extensions processing ordering within the endpoint

…through provider-side APIs and UI.

Compute,

Network,

Storage aaS

Cloud API

Service 2

vCloud API Entry

point

Cloud API

Service 1

Cloud Operator

16

APIs for the Cloud Platform Services

• Provider-side APIs for the Cloud Platform Services take existing

aspects of the vCloud Platform (logging, eventing, multi-tenancy, object

security, task management) and make them available to the authors

of vCloud Services

Compute,

Network,

Storage

API

Extension

vCloud API Entry

Point

API

Extension

We want to enable authors of new Cloud Services to leverage the same

services that are used in vCloud Director today for Compute as a Service,

so that the building blocks that all Cloud Services share in common flow

through a common set of APIs and administrative interfaces.

Shared Services of the Cloud

Lo

gg

ing

Even

tin

g

Mu

ltit

en

an

cy

Secu

rity

Ch

arg

eb

ack

Pers

iste

nce

Jo

b C

on

tro

l

Searc

h/I

nd

ex/

Cach

e

Fed

era

tio

n

17

vCloud

API Entry

Point

Example: vApp-level Backup and Recovery

API Client

Let’s take an example of something that customers are

asking for today, and walk through how it might be

delivered as an extension:

User-driven Backup & Restore

18

How to Get it Done?

As the developer of the backup service API extension, I have to

build two things:

API Definition

The API Definition – a formal description of the

complete Request and Response model for the

API, the Events the API can generate, etc.

Implementation

The Implementation – the actual code that

receives the requests and responses, takes

snapshot and archives to backup, performs

restore, etc.

19

Part 1: The API Definition

As a vCloud Feature Developer, I want to formally define the

request and response semantics of the API for my feature, so that

both users and programmatic tools can understand the definition of

the API.

The definition of the REST API is created in an IDL (interface

definition language).

WSDL is an example of an IDL for SOAP APIs. Think of this part as

building the equivalent of a WSDL for a new technology generation

(REST).

From this formal definition, I can generate API clients as well as

server-side stubs.

Use Cases Request/Response

Model API Definition

20

What Would Our API for Backup Look Like? (example)

To create a backup:

• We’d like to follow REST conventions. To do so we will POST some xml

describing our operation to the URI of the vApp we want to affect: POST http://mycloud.com/api/myvdc-7/RyansVapp-65/backups/

Content-Type: application/vnd.vmware.vcloud.Backup+xml

<Backup name=“post-service-pack”>

<Vms>

<vm>http://mycloud.com/api/myvdc/RyansVapp-65/vm-4/</vm>

<vm>http://mycloud.com/api/myvdc/RyansVapp-65/vm-10/</vm>

<vm>http://mycloud.com/api/myvdc/RyansVapp-65/vm-11/</vm>

</Vms>

</Backup>

API Client

POST to vCD

Response

w/Task

Backup

Service

vCloud

Director

Backup

stored

in Archive

21

What Would Our API for Backup Look Like? (example)

To List the Backups for a vApp:

• Lets do a GET on the Backups for a vApp: GET http://mycloud.com/api/myvdc-7/RyansVapp-65/backups/

<Backups>

<Backup name=“post-service-pack”>

<link rel=“restore” href=“http://mycloud.com/api/myvdc-7/RyansVapp-65/backups/post-service-pack/”/>

<vm>http://mycloud.com/api/myvdc/RyansVapp-65/vm-4/20110909115532/</vm>

<vm>http://mycloud.com/api/myvdc/RyansVapp-65/vm-10/20110909115532/</vm>

<vm>http://mycloud.com/api/myvdc/RyansVapp-65/vm-11/20110909115532//</vm>

</Backup>

<Backup name=“latest-weekly”>

<link rel=“restore” href=“http://mycloud.com/api/myvdc-7/RyansVapp-65/backups/latest-weekly/”/>

<vm>http://mycloud.com/api/myvdc/RyansVapp-65/vm-4/20110907120000/</vm>

<vm>http://mycloud.com/api/myvdc/RyansVapp-65/vm-10/20110907120000/</vm>

<vm>http://mycloud.com/api/myvdc/RyansVapp-65/vm-11/20110907120000//</vm>

</Backup>

</Backup>

• Each backup includes a link to restore and each VM includes the timestamp.

22

What Would Our API for Backup Look Like? (example)

To restore from a backup:

• POST a backup link to a VDC where you want to restore:

POST http://mycloud.com/api/myvdc-8/

Content-Type: application/vnd.vmware.vcloud.Backup+xml

<Backup>

<link rel=“backup” href=http://mycloud.com/api/myvdc-7/RyansVapp-65/backups/post-

service-pack//>

<…some other params…>

</Backup>

23

Disaster Recovery Solutions Available for vCloud Director

Leading ISV partners offering

DR solutions with native

vCloud Director (5.x) interop

Majority of partners leveraging

vCloud Director API

Extensibility

Mixed solutions available to

address most common DR

requirements:

• Backup and Recovery

• Replication

• Hybrid solutions

2nd generation of DR solutions

already in development

24

vCloud Hybrid Service – A Real-world Use Case for DR

VMware vCloud

Hybrid Service

Your Data Center

Any Application… No Changes

Software-Defined

Data Center

VMware vSphere &

vCloud Suite

Existing & New Apps

Seamless Networking

Common management

One Support call

IaaS cloud owned and operated by VMware based on VMware software

25

vCloud Hybrid Service, Data Protection – Service Scope

Designed to deliver agentless,

policy-driven (CBT) backup and

recovery of virtual workloads in

the cloud

Enables DR compliance on top of

redundancy and resiliency

features present

Ensures Virtual Machine images

(VMDKs) are backed up at regular

intervals and available for on-

demand restore

Full service-serviceability, with

registration, backup, restore and

monitoring operations accessible

in the vCHS console

Daily/24-hour backup schedule

Synthetic-full and encrypted backup images

Unlimited self-service vApp-level protection

Customizable scheduling and retention policy

Unlimited self-service VM-level restores

26

vCloud Hybrid Service, Data Protection – Architecture Overview

27

vCloud Hybrid Service, Data Protection – VDC Setup

2. VDC-level policy settings for scheduling and retention

3. VM or vApp-level restore options

1. Dashboard view for Data Protection

28

vCloud Hybrid Service, Data Protection – VM Registration

28

1. Enable VM backup via Register option

2. vApp affinity notification & confirmation

Tag insertion results in all vApp VMs getting backed up during next cycle.

3. Metadata tagging ensures vApp-level Backup

29

vCloud Hybrid Service, Data Protection – VDC Status View

2. At-a-glance status, policy and consumption

1. Dashboard view for Data Protection

30

Lessons Learned from vCloud Hybrid Service Design

Self-Serviceability

End-users increasingly adamant about self-directed consumption/management

Transparency via logging and audits is paramount

“Hope is not a strategy for DR”

vApp metadata handling for automated restores less straightforward than it seems

1 Operations Scale

Precision capacity planning for shared

backup appliances and media

Provisioning and performance SLAs

Complex job scheduling algorithms

Metering/billing complexities given CBT and De-duplication

Secure encryption and destruction

2

ISV Solution Maturity

Several failing miserably at modernizing their APIs for the cloud and DevOps era

Those with 1st generation Web Services APIs focused primarily on end-user consumption – not SP admin operations

UI/UX experience still lousy for some

Excessive dependency on vSphere-level integration for management

3 Cost Differentiation

Competing with commodity cloud providers offering ¢.xx pricing per GB adds tremendous pressure

Pre-disposition from end-users to look at Backup & Recovery as commodity

Emphasizing DR compliance in the public/hybrid cloud as an extension of private BC/DR planning

4

31

Other VMware Activities Related to This Session

HOL:

HOL-SDC-1305

Business Continuity and Disaster Recovery In Action

Group Discussions:

PHC1003-GD

vCHS Use Cases and Workloads with Rachna Thusoo

Additional References

• vCloud Hybrid Service web site

• Stretch Cloud Blog (vCloud Hybrid Service, Data Protection)

• Yellow Bricks Blog (vCloud Director and SRM)

• Chris Colotti’s Blog (Disaster Recovery and vCloud Director)

• vCloud Director API Extension Services Documentation

THANK YOU

Protecting Enterprise Workloads Within a vCloud

Service Provider Environment

Ian Perez Ponce, VMware

PHC5679

#PHC5679