VMworld Europe 2014: Storage DRS - Deep Dive and Best Practices

Disclaimer

• This presentation may contain product features that are currently under development.

• This overview of new technology represents no commitment from VMware to deliver these features in any generally available product.

• Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.

• Technical feasibility and market demand will affect final delivery.

• Pricing and packaging for any new technologies or features discussed or presented have not been determined.

CONFIDENTIAL 2

Storage DRS Deployment:

Thin Provisioning

Deduplication

Auto-Tiering

Outline

Storage IO Control Capabilities

New Control: IO Reservations

Storage DRS

Storage IO Control

New ESX IO Scheduler


Storage DRS Integration

Array-Based ReplicationStorage DRS Integration

vSphere Replication

vSphere 6.0 Beta

• Features we are presenting here are available in beta

• Perfect time to get your voice heard

https://communities.vmware.com/community/vmtn/vsphere-beta

4

The Problem

What you see

Online store:

Product Catalog

Back-up

(low priority)

Shared

Datastore

Online Store:

Order Processing

What you want to see

Shared

Datastore

Online store:

Product Catalog

Back-up

(low priority)Online Store:

Order Processing

5

Storage Performance Controls

Shares: Relative importance of VMs

– IOPS will be allocated in this proportion

Limit: Maximum IOPS allowed per VM

Reservations: Minimum IOPS per VM

IOPS: 1200

600

600

IOPS: 1200

400

400

400

Online store:

Product Catalog

Back-up

(low priority)Online Store:

Order Processing

R: 100L: 500S: 20S: 100S: 100

Outline



ESX 5.5 I/O Scheduler (mClock)

• Allocation of A and B will be in 1000:3000 ratio subject to reservation and limit constraint.

L: ∞

S: 3000

R: 100

L: 500

S: 1000

R: 200 Capacity (IOPS) VM A (IOPS) VM B (IOPS)

1000 250 750

500 200 300

200 133.33 66.66

A B

8


Compared to previous scheduler:

– Supports Reservation, Shares, and Limits

– Break large I/Os in 32KB

Limit: 100 IOPS

IO Size: 64KB

100 IOPS

Limit: 100 IOPS

IO Size: 64KB

50 IOPS

ESX 4.1 ESX 5.5


Outline



Storage IO Control



Shared Storage

S: 100

SIOC

Local I/O Scheduler

Local I/O Scheduler+

Storage I/O Control

S: 100 S: 100 S: 100 S: 100 S: 100

1200 IOPS600 IOPS 600 IOPS 1200 IOPS

600300300 400 400 400300 300600

Storage IO Control

Control Congestion in shared datastore

Detect Congestion

– SIOC monitors average IO latency for a datastore

– Latency above a threshold indicates congestion

SIOC throttles IOs once congestion is detected

– Control IOs issued per host

– Based on VMs shares, reservations, and limits on each host

– Throttling adjusted dynamically based on workload

• Idleness

• Bursty behavior

Congestion Threshold

Datastore overload

Congestion threshold value (ms):

– Higher is better for overall throughput

– Lower is better for latency

Changing default threshold:

Percentage or absolute value

Default: 90% of peak IOPs capacity

Thro

ughput

(IO

PS

)

Datastore Load

No benefit

beyond certain

load

Late

ncy

Datastore Load

SIOC and Reservations

SIOC needs to be aware of VMs reservations

– The queue depth allocation of each host depends on the VM reservation

Backward compatible (ESX 5.5 or below)

Are reservations always satisfied?

– IOPS too low due to low latency threshold

– Background operations or errors in the array

In case reservations are not satisfied:

– SIOC will notify Storage DRS for further action

R: 500

SIOC

R: 500 R: 500

Outline



Storage DRS

Storage IO Control

Storage DRS:

IO Reservations

Functionality Overview

Storage DRS and IO Reservations

• Monitor SIOC reservation enforcement

– Migrate workload if reservations are not met

• Balance reserved IOPS usage in cluster

– Match datastore IOPS capacity to reservations

• Reservations as VM placement constraints

– Hard or soft constraints

• Per-datastore reservable IOPS

– Manual override allowed

Datastore Cluster

17

Ease of Storage Management

Initial Placement

Out of Space Avoidance

IO Load Balancing

Virtual Disk Affinity (Anti-Affinity)

Datastore Maintenance Mode

Add Datastore

Storage DRS Functionality: Brief Overview

Datastore

Cluster

Storage vMotion

•••


Thin Provisioning

Deduplication

Auto-Tiering

Outline

Storage DRS

Datastore Capacity Management

Used Space : 700 GB

Thin VMDK: 100 GBThick VMDK: 500 GB

Datastore Free: 1000 GBDatastore Free: 500 GBDatastore Free: 400 GB

Thin VMDK: 100 GB

Datastore Free: 300 GB

Storage DRS Configuration Option: PercentIdleMBInSpaceDemand, default = 25%

Provisioned Space : 1500 GB

Space Demand : 900 GB

20

Datastore: 2 TB

Thin Provisioned Datastores

Storage Array

Datastore: 1 TB

Bac

Datastore: 1 TB

Capacity : 4 TB

Used Capacity : 4 TB

Provisioned Capacity : 12 TB

Allocated Capacity : 6 TB

Virtual Disks

Backing Storage Pool

21

How does Storage DRS handle space over-commit?

• Hypervisor Level: Storage DRS manages space over-commit

– Fine grain control for overcommit magnitude

– Out of space avoidance using storage vmotion

• Datastore Level: Storage arrays manage space over-commit

– Storage DRS manages logical space usage in datastores

– VASA integration (v1): Storage DRS handles space outage signal from backing pool

– VASA integration (v2): Storage DRS controls space usage in backing pools

• Best practice: can use either method, but not both

22

Deployment with Deduplication

• Provides space efficiency

• Dedupe pool can span across multiple datastores

Dedupe

Storage DRS uses free space in datastore

VASA Integration (v2) : Storage DRS manages

logical space while keeping virtual disks in the

same dedupe pool

⤬Problem: Datastore appears to store

more data than capacity!

Total Virtual Disk

Size: 4TB

LUN Capacity: 1TB

23

Deployment with Auto-Tiered Arrays

• Multiple storage tiers

• VM data across tiers

• Tier use changes with workloadCapacity Tier

Performance Tier

Logical LUN of Auto-tier Array

Storage IO Control

IO priority

IO over-load remediation

Automatic initial placement

Space load balancing

Rule enforcement

Maintenance mode

Latency

IO Load

Auto-tiered array

SIOC Threshold

SDRS Threshold

24

Fine Grain Controls

All aspects of Storage DRS can be

controlled to suit your environments


Thin Provisioning

Deduplication

Auto-Tiering

Outline

Storage DRS


Array-Based Replication

vSphere SRM: Array-based Replication

• Storage DRS identifies replicated datastores

• All recommendations are in sync with replication policies:

– Automated moves within the same consistency group

– Manual moves for all VMs residing on replicated datastores

• Accounting of replication overhead due to Storage vMotion

27

Outline

Storage DRS


Array-Based ReplicationStorage DRS Integration

vSphere Replication

vSphere Replication (VR)

• Storage DRS discovers VR-replicas in datastores

• Storage DRS understands space usage of replica disks

• Storage coordinates moves with VR

– Space balancing

– Maintenance mode

29

Storage IO Control Best Practices

Avoid mixing vSphere LUNs and non-vSphere LUNs on the same physical storage

– SIOC will detect this and raise an alarm

Configure host IO queue size with highest allowed value

– Maximum flexibility for SIOC throttling

Keep congestion threshold conservatively high

– Will improve overall utilization

– Set lower if latency is more important than throughput

Datastore Cluster Best Practices

Similar datastore performance

May not be identical

Similar capabilities

Data management

Backup

✔Cluster1: Wide Perf

Variation

Cluster2: Similar

Datastores

31

Datastore and Host Connectivity

• Maximum possible host and datastore connectivity

• Improves DRS and Storage DRS performance

Partially Connected Datastore Cluster Fully Connected Datastore Cluster

More datastores in cluster better space and I/O balance

Larger datastore size better space balance

DRS Cluster DRS Cluster

32

vSphere Storage Policy based Management

Silver Disk Pool Gold Disk Pool

Data

store1

Data

store2

Data

store3

Data

store4

Cluster-1 (Tier2 VMs) Cluster-2 (Tier1 VMs)

Datastores with any storage

profileSilver Disk Pool Gold Disk Pool

Data

store1

Data

store2

Data

store3

Data

store4

Cluster-1 (Tier1 + Tier2 VMs)

Datastores with identical

storage profile

33

Summary

• Easier Storage Management

• Effective storage capacity and performance management

– Support for IO reservations

– Integration with storage environments

– Integration with vSphere solutions

• Many exciting features available in vSphere 6.0 Beta!

34

https://communities.vmware.com/community/vmtn/vsphere-beta

Technology

VMworld Europe 2014: Storage DRS - Deep Dive and Best Practices