View
204
Download
0
Category
Tags:
Preview:
Citation preview
Virtual Volumes Technical Deep Dive
STO1965
Rawlinson Rivera, VMware, Inc Suzy Visvanathan, VMware, Inc
CONFIDENTIAL 2
Disclaimer • This presentation may contain product features that are currently under development. • This overview of new technology represents no commitment from VMware to deliver these
features in any generally available product. • Features are subject to change, and must not be included in contracts, purchase orders, or
sales agreements of any kind.
• Technical feasibility and market demand will affect final delivery. • Pricing and packaging for any new technologies or features discussed or presented have not
been determined.
CONFIDENTIAL 3
Agenda
1 Virtual Volumes: Introduction
2 High level Architecture
3 Data path
4 Managing Storage Capacity
5 Ensuring SLOs through policies
6 VM Objects – Virtual Volumes
7 Data Services
8 Making the transition to Virtual Volumes
9 Learn more
CONFIDENTIAL 4
The Software-Defined Data Center
Transform storage by aligning it with app demands
Management tools give way to automation
Expand virtual compute to all applications
Virtualize the network for speed and efficiency
CONFIDENTIAL 5
The Software-Defined Data Center
Transform storage by aligning it with app demands
CONFIDENTIAL 6
VMware Software-Defined Storage and Virtual Volumes
Policy-driven Control Plane
Virtual Data Plane
Virtual Datastores
Performance Mobility Data Protection
Virtual Data Services
Cloud Object Storage SAN / NAS
x86 Servers
6
• Interface between apps and storage services • Interprets VM-policies and composes service
• Initial placement and on-going optimization
• Abstraction and pooling = Storage container • HW capabilities expressed to control plane
• New storage tiers for data persistency
• Applied to individual VMs • Dynamically invoked by control plane
• Array-based, 3rd party VA
Virtual Volumes
CONFIDENTIAL 7
Today’s Challenges in External Storage Architectures
Today
1. Create fixed-size, uniform LUNs
2. Lack of granular control
3. Complex Provisioning cycles
4. LUN centric storage configurations
û Extensive manual bookkeeping to match VMs to LUNs
û LUN-granularity hinders per-VM SLAs
û Overprovisioning (better safe than sorry!)
û Wasted resources, wasted time, high costs
û Frequent Data Migrations
External Storage
vSphere
Array-a Array-b
LUN
LUN LUN
LUN LUN
CONFIDENTIAL 8
Goals of Virtual Volumes
• Ability to express application (VM/VMDK) granular data services
• Provide easy on-demand Capacity provisioning
• Compliance Monitoring • Ability to get most out of the
storage system
• Easy Capacity management • Meet VM SLOs • Access Control and Security
VI admin
Storage admin
CONFIDENTIAL 9
Replication Snapshots Caching Encryption De-duplication
vSphere
External Storage Architectures with Virtual Volumes
External Storage without VVols External Storage with VVols
Policy based Management
Offloaded Data Services
Eliminates LUN Management
Provides Per-VM Granularity
datastore a
vSphere
Array-a Array-b
LUN
LUN LUN
LUN LUN
AND
CONFIDENTIAL 10
High Level Architecture
Published Capabilities
Snapshot
Replication
Deduplication
Encryption
Overview
• No FileSystem
• ESX manages Array through VASA (vSphere APIs for Storage Awareness) APIs.
• Arrays are logically partitioned into containers, called Storage Containers
• VM disks, called Virtual Volumes, stored natively on the Storage Containers.
• IO from ESX to array is addressed through an access point called, Protocol Endpoint (PE)
• Data Services are offloaded to the array
• Managed through storage policy-based management framework
vSphere Storage Policy-Based Mgmt.
Virtual Volumes
Storage Policy
Capacity
Availability
Performance
Data Protection
Security
PE VASA Provider
PE
CONFIDENTIAL 11
VASA Provider (VP)
• Software component developed by Storage Array Vendors
• ESX and vCenter Server connect to VASA Provider
• Provides Storage awareness services
• Single VASA Provider can manager multiple arrays
• Supports VASA APIs exported by ESX
• VASA Provider can be implemented within the array’s management server or firmware
• Responsible for creating Virtual Volumes
Virtual Volumes
VASA Provider
Characteristics
Data Path
CONFIDENTIAL 13
Protocol Endpoints (PE) Why Protocol Endpoints? • Separate the access points from the storage
itself
• Can have fewer access points
What are Protocol Endpoints? • Access points that enables communication
between ESXi hosts and storage array systems.
• Part of the physical storage fabric
• Created by Storage administrators
Virtual Volumes
VASA Provider PE
CONFIDENTIAL 14
Protocol Endpoints (PE) Scope of Protocol Endpoints • Compatible with all SAN and NAS Protocols:
- iSCSI - NFS v3 - FC - FCoE
• A Protocol Endpoint can support any one of the protocols at a given time
• Existing multi-path policies and NFS topology requirements can be applied to the PE
VASA Provider iSCSI/NFS PE
Virtual Volumes
CONFIDENTIAL 15
Protocol Endpoints (PE) Protocol Endpoint discovery process • SCSI PEs are discovered during an ESX
rescan
• NFS PEs are maintained as IP addresses or file paths
• ESX will identify PE and maintain all discovered PEs in a database.
VASA Provider iSCSI/NFS PE
Virtual Volumes
Managing Storage Capacity
CONFIDENTIAL 17
Storage Container (SC) What are Storage Containers?
• Logical storage constructs for grouping of virtual volumes.
• Setup by Storage administrators
• Capacity is based on physical storage capacity
• Logically partition or isolate VMs with diverse storage needs and requirement
• Minimum one storage container per array
• Maximum depends on the array
• A single Storage Container can be simultaneously accessed via multiple Protocol Endpoints
Virtual Volumes
CONFIDENTIAL 18
Storage Containers (SC)
Virtual Volumes
Snapshot Replication
vCenter
VASA Provider
Storage Container Discovery Process
• Storage admin sets up Storage Container with desired capacity
• Desired Capabilities are applied to the Storage Containers
• VASA Provider discovers Storage Container and reports to vCenter
• Any new VMs that are created will subsequently be provisioned in the Storage Container
CONFIDENTIAL 19
Differences between Storage Containers and LUNs • Size based on array capacity • Max number of SCs depend only on the array
ability • Size of SC can be extended • Can distinguish heterogeneous capabilities
for different VMs (Virtual Volumes) provisioned in that SC
• Managed by VASA APIs
• Fixed size • Fixed size mandates more number of LUNs • Needs a FileSystem • Can only apply homogeneous capability on
all VMs (VMDKs) provisioned in that LUN. • Managed by In-band FileSystem commands
Storage Containers
LUN
CONFIDENTIAL 20
Storage Container (SC)
Do I still need to create Datastores?
Storage Container vSphere Datastore
CONFIDENTIAL 21
Storage Container (SC)
CONFIDENTIAL 22
Storage Container (SC)
Storage capabilities
Storage policies
vSphere Web Client
Storage Management UI
Datastore
Storage Container What do the Admins see?
Ensuring SLOs through Policies
CONFIDENTIAL 24
Storage Policy Based Management (SPBM) – Array Capabilities
Virtual Volumes APIs
Storage Policy-Based Mgmt.
CV
CV
CV
Storage admin
Publish Capabilities
• Array based features and data services
• Defines what an Array can offer
• Advertised to ESX through VASA APIs
Sample Default Profile for (6090a058-cd89-ffe3-87763007db37] - capabilities
Disk Types
Disk Encryption
Dedupe
Replication
Snapshot
CONFIDENTIAL 25
Storage Policy Based Management (SPBM) – VM Policies
CONFIDENTIAL 26
Storage Policy Based Management (SPBM)
VM Objects - Virtual Volumes
CONFIDENTIAL 28
Virtual Volumes – An Introduction • Virtual Volumes
– Virtual machine objects stored natively on the array storage containers.
– No Filesystem on-disk formatting required
• There are five different types of recognized Virtual Volumes: – Config-VVol – Metadata – Data-VVol – VMDKs – Mem-VVol – Snapshots – Swap-VVol – Swap files – Other-VVol – Vendor solution specific
vSphere Web Client View
vvol
CONFIG
DATA
SWAP
MEM
CONFIDENTIAL 29
Virtual Volumes
Storage capabilities
Storage policies
vSphere Web Client
Storage Management UI
Virtual Machines Datastores
Virtual Volumes
Storage Container What do the admins need to get familiar with?
VVol VVol VVol
CONFIDENTIAL 30
Virtual Volumes
VM objects view from a storage container on an array VM objects view from a datastore
vSphere Web Client Storage Management UI vSphere Admin View Storage Admin View
CONFIDENTIAL 31
Virtual Volumes – Bind requests
• IO path is established through a VASA Bind request
• VASA Provider does two things upon receiving a Bind request
• Returns a PE ID to which the VVol is bound • A unique secondary ID to be used for IO
between the bound VVol and PE
• For SCSI, the secondary ID is the secondary LUN ID
• For NAS, the secondary ID is the file path
• M:M Relationship between VVol and PE
Virtual Volumes
VASA Provider
VVol
PE
VVol
Bind
CONFIDENTIAL 32
Virtual Volumes – Other Bind Scenarios
UNBIND
Virtual Volumes
VASA Provider
VVol
PE
VVol
UNBIND
• Delete binding of a VVol to the PE
• VASA Provider may choose to delete a binding for several reasons
REBIND
Virtual Volumes
VASA Provider
VVol
PE
VVol
PE
REBIND
• Move VVols across different PE
• VASA Provider may choose to issue a rebind for several reasons Eg., Load balancing
Data Services
CONFIDENTIAL 34
Snapshots • Offloaded to Array • Copy on write image of a Virtual Volume
• Two type of snapshots supported: – Managed Snapshot – Managed by ESX.
• A maximum of 32 snapshot are supported for fast clones
– Unmanaged Snapshot – Managed by the storage array.
Managed Snapshot - vSphere
Unmanaged Snapshot - Array
Piecing It All Together
CONFIDENTIAL 36
Virtual Volumes – The New De-facto Storage Paradigm
Capabilities
vSphere Storage Policy-Based Mgmt.
Virtual Volumes
VASA Provider PE PE
Making the Transition to Virtual Volumes
CONFIDENTIAL 38
Support by Broad Ecosystem Makes Transition Smooth
More than 20 VVOL
Partners
Partners Announcing
GA
Virtual Volumes in
Beta
And Many More…
CONFIDENTIAL 39
Migration Scenario: VAAI vs Virtual Volumes
VAAI & VASA API Compatible Array
vmkernel data mover use VAAI APIs
XCOPY /WRITE_SAME primitive for cloning
operation
2
vSphere
VMFS VVol
1
vSphere Admins
clone VM from VMFS to VVol container
Learn More…..
CONFIDENTIAL 41
Nimble Storage Array Group
vCenter
ESXi ESXi
VVols and SPBM in Nimble – Architecture Overview
CONFIDENTIAL 41
CONFIDENTIAL 42
CONFIDENTIAL 43
CONFIDENTIAL 44
Learn More…….
STO1963 – Virtual Volumes Business Overview
STO 3163 – Virtual Volumes and Scalable Data Protection in a Software Defined Enterprise
STO3162 – Satisfy requirements of your application at the granularity of VVols (Nimble)
STO3246 – Scalable Virtual Volumes Storage Management with IBM XIV storage
STO3161 – What can Virtual Volumes do for you? (EMC)
STO2142 – Hypervisor & Storage QOS; Two Great Tastes that Taste Great Together (SolidFire)
STO3247 – VVol Technical Preview with DELL storage
STO2554 – How Virtual Volumes will provide Shared Storage with X-ray vision (HP)
STO2752 – Deploying VVol with Hitachi Data Systems
VMware HP Netapp Dell EMC IBM HDS Nimble Tintri SolidFire Atlantis Computing SANBlaze
HOL-SPL-1429 VVol Tech Preview
Breakout Sessions Demos at Booth
Participate in Virtual Volumes Beta
Misc
Focused Sessions Engage with VMware
Engage with Vendors
Attend NDA sessions
Q&A
Thank You
Fill out a survey Every completed survey is entered into a
drawing for a $25 VMware company store gift certificate
Virtual Volumes Technical Deep Dive
STO1965
Rawlinson Rivera, VMware, Inc Suzy Visvanathan, VMware, Inc
BACKUP
CONFIDENTIAL 50
Snapshots • Snapshots are a point in time copy on write
image of a Virtual Volume with a different ID from the original
• Virtual Volumes snapshots are useful in the contexts of creating: – a quiesced copy for backup or archival purposes,
creating a test and rollback environment for applications, instantly provisioning application images, and so on
• Two type of snapshots supported: – Managed Snapshot – Managed by ESX
• A maximum of 32 snapshot are supported for fast clones
– Unmanaged Snapshot – Manage by the storage array • Maximum snapshot dictated by the storage array
Managed Snapshot - vSphere
Unmanaged Snapshot - Array
CONFIDENTIAL 51
Fast Clone • Fast‐clones are an out‐of‐band space
efficient cloning operations, performed exclusively on VMDKs – performed on the same storage container using
the storage profile of the original virtual volume, or the profile specified
• ESXi hosts guarantee not to issue any I/O to the source or destination virtual volumes during fast clone operations
• Fast‐clones are similar to the snapshot operation with a few differences: – There are no revert operations – Space-efficient
vasa vendor provider
prepared for snap
progress update 1
progress update N
ready
create
acknowledge
vvol
DATA
Fast clone Snapshot Workflow
Read only to based VVol allowed
CONFIDENTIAL 52
Storage Policy Based Management – Array Capabilities Name Description Sample_Default_Profile for (6090a058-cd.. Default capability profile for storage
container
General Capability sets
Default Profiles Connectivity with Hosts
Protocol Endpoints
Sample Default Profile for (6090a058-cd89-ffe3-87763007db37] - capabilities
Disk Types
Disk Encryption
Dedupe
Replication
vmwareR vSphere Web Client
CONFIDENTIAL 53
clone VM from VVol container to different VVol container
vSphere Admins
vSphere
1
2
offload to array
3
Fully VAAI & VASA APIs Compatible Array
SC-A SC-B
always attempt default operation with VASA API primitives
If default operations fails VAAI API primitives are used
clone VM from VVol container to different VVol container
vSphere Admins
vmkernel data mover uses
VAAI primitives for cloning operation
vSphere
1
2
offload to array
3
Fully VAAI & VASA APIs Compatible Array
SC-A SC-B
default 1 fail back 2
vendor native clone utilized with VASA primitives
cloning operation
VAAI vs VVol
CONFIDENTIAL 54
Recommended