IBM Smart Cloud Provisioning Overview

© 2013 IBM Corporation

IBM SmartCloud Provisioning overview

Rossella De Gaetano : [email protected]

May 2013

© 2013 IBM Corporation2

Companies need an affordable entry point for cloud

Customer goals:

•Provide agile service delivery that caters to changing business needs.•We know virtualization helps reduce capital expense but how do I control my operational cost?•Need to increase the performance, utilization, reliability and scalability of IT resources while reducing IT complexity•Ability to expand capabilities

VISIBILITY CONTROL AUTOMATIONSee services in real time &

better predict business outcomes

Better manage assets, service & compliance.

Achieve greater efficiency and service quality

Organizations need a low-cost, low-risk entry point to cloud computing technology with compelling features that achieves their business goals.


Core capabilities of SmartCloud Provisioning (and its components)

Distributed architecture for solution resilience. Rapid scalable deployment designed to deliver near-instant deployment of 100s of virtual machines

in seconds instead of mins or hours. Continuous operations during upgrades and maintenance resulting in no outages or downtime. Reliable, non-stop cloud capable of automatically tolerating and recovering from software and

hardware failures. Save IT labor resources at scale by enabling self-service request and highly automated operations Hypervisor & hardware agnostic enabling choice and avoiding vendor lock-in. Open source, commodity skills, small footprint.

High Scale Low Touch ICCT – Image Construction Virtual Image Library IWDSmartCloud Provisioning



Advanced Image lifecycle management & image composition tooling. Tooling to simplify migration of workloads between hypervisors. Hypervisor agnostic supporting image composition on different platforms. Image publishing and image repository. Run-time image activation allowing advanced customization from standardized templates. SmartCloud Enterprise enabled, supporting integration with IBM’s public cloud.




Discovery of images across the virtual infrastructure. Automatic indexing / cataloging of images. Image comparison tooling to identify changes, and manage change. Image version control to help standardise images. Supports a wide range of image and OS types. Portability checks and remediation to help create cross hypervisor images




PaaS pattern deployment. GUI based pattern creation. Deployment of IaaS & PaaS to advanced hypervisor managers (eg. VM control) Pre-canned images available (chargeable)



Extending the Cloud capabilities beyond SmartCloud Provisioning

SmartCloud Monitoring Tivoli Storage Manager fVE Tivoli Usage & Accounting MgrTivoli Endpoint Manager

Health AnalyticsHost & VM MonitoringEvent Response & Mgt

Capacity PlanningWhat-if Scenarios?

Patch ManagementCompliance Reporting

Policy Enforcement

Centralised BackupPolicy DrivenData Restore

Image Snapshots

Usage ReportingCognos Reporting

Accounting & RatingInvoice Creation

Security and Compliance

Availability and Performance

Usage and Accounting

Security and Compliance



IBM SmartCloud Provisioning architecture and functionalities

May 2013


SmartCloud Provisioning architecture

VMVM

Hypervisors & IAAS Management SHIMSHypervisors & IAAS Management SHIMS

vCenterSHIM

REST APIsREST APIsSelf Service and Admin UISelf Service and Admin UI

HSLTKVM, Xen, ESXi

IAAS API ShimIAAS API Shim

vCenter

Placement/DeploymentPlacement/Deployment

Virtual SystemsVirtual

Systems

SCPVirtual Image Library

ICON

VMControl

Virtualization Management

Based Provisioning

VMControlSHIM

zLinux

zLinuxSHIM


Patterns

Deploy more than one VM in one shot The VMs can be deployed with no interconnection (e.g. Deploy a pattern of intermediate or basic VMs) The VMs can be deployed assuming after boot they will be connected (e.g. WAS cluster) Add-ons and packages scripts can be used to customize images

Note:pre-canned image for pattern deployment can be downloaded from IBM (charged!)

They can be used not only for IBM software


Patterns

VM1

VM 2

VM 3

VM 2VM 2VM 2

VM 3

VM1VM1Virtual systems

Pattern


Image Management

June 2012


In the beginning, there was the perfect image…1

Then users starting making changes and “snapshots”……and what they put in the images is unknown…

2

Then they get copied to multiple locations……and some change again…

3

Then you need to apply a critical security patch…how? … where?4

What’s the problem with Virtualization and Standardization?


Capturing images is easy There are many tool that simplify the process All you need is enough space the store them

However managing images is hard

Standardization How to contain the proliferation of image variants?

Search How does one find a useful image?

Version control Who did what to which image, and when?

Drift Detection Identify images that diverge from initial configurations

Vulnerability Images must be updated with security patches

Image building Reduce manual labor to compose images


IBM SmartCloud Provisioning Puts You Back in Control

Control over Image Versions, Content and Locations– Image library allows check in, check out, and tracking of versions in the environment

•Changes can finally be tracked

– Powerful image analytics finds the content you need and can show the specific changes from one image to another

•Encourages reuse and gives needed visibility to analyzing whole systems at a glance

– Images are tracked across multiple Clouds and/or multiple sites•Critical for disaster recovery arrangements and decentralized use

Control over Image Construction– Build images using Company-certified OS, middleware, and application packages

•Avoids images being taken out of production due to non-compliance


Two optionally installable components:•Image Construction and Composition Tool

•Import images from heterogeneous providers•Extend images

Avoid images being taken out of production due to non-compliance

•Image library•Import & export images across an etherogeneous set of repositories•Assign version to images•Search for images•Compare images

Get full control on image lifecycle


High Scale Low Touch

May 2013


Storage node Storage

node

Compute node

First Box

Compute node

• Out-of-the-box and running in less than 4 hours• Get started with only 4 servers• Out-of-the-box and running in less than 4 hours• Get started with only 4 servers

• Quickly stand up a cloud• Start small and scale based on need• No additional pre-reqs such as databases, app

servers, messaging middleware• Freedom of choice for hypervisors

• Avoid expensive vendor lock in• VMWare ESX, KVM, Xen, Hyper-V

• Highly cost effective solution• Requires no additional hypervisor management

tools• Requires no HA hardware or software

• Rich set of interfaces into the cloud• Web Interface, scripting and web services• All function can be driven by a user or by scripts

for complete automation

HSLT

Quickly get started with HSLT


Requested VMs will be up and running under a minute using standard HW

• It’s Fast• Can start 100 VMs in under 3 mins• Can start a single VM and load OS in under 10 seconds• Can go from bare metal to ready for work in under 5 minutes

• It Scales up to and beyond 50,000 VMs in an hour (50 nodes)• Add capacity by simply plugging in a blade or server• Writes only the data you change• Peer-peer architecture to avoid traditional bottlenecks

• It's Fault-Tolerant• “Live Update” capability to patch or upgrade the Cloud• No single point of failure• Automatic failure recovery

HSLT

Provides Unparalleled Scalability, Speed and Fault Tolerance

Hardware

Hypervisor

Bot

Hardware

Hypervisor

Mgt VM

VM Bot

Hardware

Hypervisor

Bot

Hardware

Hypervisor

VM Bot

Hardware

Hypervisor

VM Bot

Hardware

Hypervisor

VM Bot

Hardware

Hypervisor

Bot

Hardware

Hypervisor

Hardware

Hypervisor

VM Bot

Bot


• HSLT coordinates three indipendent ingredients:• Virtual servers• Network addresses• Storage volumes When a request is made, a given virtual server is attached to one or more network addresses and one or more storage volumes

• This has several advantages:• Quickly recover from failures – cheaper to replace than to fix:

- Swap out old VM and replace with fresh new VM on

different hardwareDramatically simplifies patch, image and change management:

- New versions can be easily swapped in, and old versions can be easily put back if problems arise

- Servers can be restarted and then reattached to their previous storage volumes and addresses to quickly restore their state

HSLT key concepts..


No single points of failure, no bottlenecks in data serving/processing, no intervention to repair broken parts!

Failures that will impact your users, slowdowns that your users will notice, and extra work for the admin team!

Multiple, load balanced instances of all services Parallel processing against storage Self-adapting peer-to-peer communication & coordination Recovery oriented computing Network deployed software & image updating

Distributed request processing, data storage and messaging Designed to run on lower cost hardware and storage devices All services monitor and restart each other, and dynamically elect the leader Base software is loaded via network boot (PXE) Services are images, so update themselves by restarting with new image version

Single instance of critical services Serialized processing No automatic restart and rerouting of requests Patches and upgrades go everywhere

Requires very high cost hardware The bigger the cloud the worse the damage Request for images bound to a single location Serialized, slow access and susceptible to peak overloads Patches/upgrades require taking down the Cloud to apply and they need to be carefully applied When a service breaks, that part of the system is out Admins have to troubleshoot and fix

HSLT vs traditional hierarchical architecture

VM

VM

VM

VM

VM

VM

VM

VM

VM

VM

VM

VM

VM

VM

VM

VM

VM

VM

VM

VM

VM

VM

VM

VMVM

VM

VM

VM

VMVM

End Users

Server machines Storage devices(iSCSI)

VMVMVMVMVM

End Users

VMVMVMVM

VMVMVMVMVM

VMVMVMVMVM

VMVMVMVMVM

VMVMVMVMVM


Standard IaaS Interface

Hypervisor

Dom0VM1VM2VM3

iSCSI∆‘s∆‘s∆‘s

Image streaming

Hypervisor

VM0

VM0

VM0

VM0

iSCSI

∆‘s ∆‘s ∆‘s

Hypervisor

VM0

VM0

VM0

VM0

iSCSI

∆‘s ∆‘s ∆‘s

Hypervisor

VM0

VM0

VM0

VM0

iSCSI∆‘s ∆‘s ∆‘s

Hypervisor

VM0

VM0

VM0

VM0


Hypervisor

VM0

VM0

VM0

VM0


Hypervisor

VM0

VM0

VM0

VM0


Hypervisor

VM0

VM0

VM0

VM0


Management Agent

Dynamic P2P network overlay

Storage Nodes

Compute Nodes

1. Leader fails• triggers reelection and another takes over

2. Agent fails• watchdog restarts

3. Storage node fails• Multi-path continues uninterrupted• Agent connects to another replica• Leader tries to restart

4. Compute node failure• Partial faults cause agent to stop accepting requests

to evacuate (live migration available on KVM)• Once empty will restart (PXE fresh image)• Full failure detected by leader which stops sending

requests• Leader cycle’s server power (PXE fresh image)

5. Service VM fails• All services are clustered to tolerate individual faults• Leader VM detects and replaces failed VM

6. Network fault1.Redundant interfaces automatically take over

1. Leader fails• triggers reelection and another takes over

2. Agent fails• watchdog restarts

3. Storage node fails• Multi-path continues uninterrupted• Agent connects to another replica• Leader tries to restart

4. Compute node failure• Partial faults cause agent to stop accepting requests

to evacuate (live migration available on KVM)• Once empty will restart (PXE fresh image)• Full failure detected by leader which stops sending

requests• Leader cycle’s server power (PXE fresh image)

5. Service VM fails• All services are clustered to tolerate individual faults• Leader VM detects and replaces failed VM

6. Network fault1.Redundant interfaces automatically take over

Failure mode and automatic recovery in HSLT


Additional resources

IBM SmartCloud Provisioning information center:https://www.ibm.com/developerworks/mydeveloperworks/wikis/home?lang=en#/wiki/IBM

%20SmartCloud%20Provisioning/page/Home

IBM SmartCloud Provisioning WIKI:https://www.ibm.com/developerworks/mydeveloperworks/wikis/home?lang=en#/wiki/IBM

%20SmartCloud%20Provisioning/page/Home

•Additional resources

•Customer interaction

•Demos

•Upcoming features

SmartCloud Provisioning and Orchestration community:https://www.ibm.com/developerworks/mydeveloperworks/groups/service/html/

communityview?communityUuid=e5a54efe-3c9f-491b-af2a-e5400516b5aa

IBM SmartCloud Provisioning forum:http://www.ibm.com/developerworks/forums/forum.jspa?forumID=2670


Backup...


vCenter or HSLT cloud groups?

vCenter HSLT deploy/delete persistent images Capture images Modify hw resources of deployed images Apply fixes deploy/delete patterns VMs takeover IPv6 support Can leverage Vmotion

deploy/delete persistent images deploy/delete non persistent images Capture images Modify hw resources of persistent deployed images Apply fixes Attach elastic IP Attach disks Deploy/delete patterns Live migration available (KVM only) Low touch High scalability Supports KVM, ESXi, Xen, Hyper-V


Static IPs vs elastic IPs

Static IPs Elastic IPs The IP group is configured to use static IPs The IP address is assigned at instance deployment time Can be an IPv4 or an IPv6 address (HSLT cloud group supports IPv4 only) The vNIC of the instances is configured for that IP

The IP group is configured to use DHCP The IP address is assigned after the instance has been deployed Must be an IPv4 address The vNIC is not touched at all: NAT is used

Technology

IBM Smart Cloud Provisioning Overview