Cisco HyperFlex Deep Dive and Performanceclnv.s3.amazonaws.com/2017/eur/pdf/BRKCOM-2004.pdfmay conflict with existing config • Consult your CSE and TAC for guidance and assistance

Cisco HyperFlex Deep Diveand Performance

Brian Everitt, Technical Marketing Engineer

BRKCOM-2004

Cisco HyperFlex 2.0 All Flash Systems Offer High Performance, Low Latency, and Consistent Results

• Introduction

• What is HyperConverged?

• Cisco HyperFlex Overview

• What’s New in HX 2.0

• Architecture

• Performance and Benchmarks

• Conclusion

Agenda

Brian Everitt

• Technical Marketing Engineer with the CSPG BU

• Focus on HX performance, benchmarking, quality and SAP solutions

• 5 years as a Cisco Advanced Services Solutions Architect

• Focus on SAP Hana appliances and TDI, UCS and storage

• First time at Cisco Live EU

About Me…

© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public

What Is Hyperconvergence?

HyperconvergenceOn-Demand

Agile

Efficient

Simple

Scalable

Enterprise Ready

Cost Effective

+

+

+

+

+

+

+

CPU

MEM

CPU

MEM

NO LEGACY STORAGE CONTRUCTS

APP APP APP APP APP•••••••HYPERCONVERGENCE

SERVERS

1-3yr Planned

Large Footprint

Inefficient

Complex

SERVER VIRTUALIZATION

APP APP APP APPOn-Demand

Agile

Efficient

Simple

Vendor E Vendor N Vendor P

••••••••••••

•••••••••••••••

+

+

+

+

–

–

–

–

Tra

ditio

nal N

AS

CPU

MEM

Hybri

d S

tora

ge

All F

lash S

tora

ge

CPU

MEM

CPU

MEM

CPU

MEM

APP

Traditional Approach

STORAGE

6BRKCOM-2004


HyperFlex Key Customer Benefits

Efficient

AdaptableScale-As-You-Grow

AdaptiveScaling

Non-Stop Infrastructure

Built on the UCS Platform

FlexibleDeployment

Always-on Data Optimization

CompleteHyperconvergence

IntuitiveManagement

CloudSpeed

All 3 Layers of Network, Compute

and Storage Intelligently Integrated

Into a Single Solution for <60 Minute

Deployment

Eliminate Infrastructure Silos and Use

Existing Management Tools to Manage

and Automate Operations

Instantly Provision, Clone or Snapshot Applications

Unified Management Using Existing

Skillsets in the Organization

Variety of Configurations to Handle

Diversified Set of Workloads

Inline Dedupe and Compression Ensuring Hyper-Efficient Resource Utilization

Add Resources Non-disruptively

and Scale Performance Linearly

in Small Increments

Scale Compute or Capacity to Match

Application Needs

Self-Healing Fabric Based Hyperconvergence with Cloud Monitoring

Agile

12BRKCOM-2004

Cisco HyperFlex SystemsHX-Series


What is HyperFlex?

1 Complete Hyperconvergence

Unified Compute and

Network Infrastructure Wide Array of Rack and Blade Form Factors

Integrated, High Performance

Network Fabric

Programmable Automation of

all Hardware

Cisco UCS: The Ideal Platform for Hyperconvergence

BRKCOM-2004 14


1 Complete Hyperconvergence

Unified Compute and

Network InfrastructureCisco HyperFlex

Cisco HX Data Platform

Data Services and Storage Optimization

2 Next Gen Data Platform

Designed for

Distributed Storage

What is HyperFlex?

BRKCOM-2004 15


Cisco HyperFlex Systems

1

3

Complete Hyperconvergence

Unified Compute and

Network Infrastructure

Part of a Complete Data Center StrategyElastic and Secure at Enterprise Scale

Cisco HyperFlex

Cisco HX Data Platform

Data Services and Storage Optimization

2 Next Gen Data Platform

Designed for

Distributed Storage

Cisco One

Enterprise Cloud SuiteCisco ACI

Cisco Security

BRKCOM-2004 16


Mainstream Computing

Scale Out

Cisco UCS: Platform for All Architectures

HyperconvergedInfrastructure

Converged Infrastructure

Unified ManagementSingle Control Plane

Single API

Edge Cloud

ROBO

UCS S-Series

Storage

Fourth Generation

UCSHyperFlex Systems

UCS Mini

E-Series

C-Series

Rack Servers

UCS Integrated

Infrastructure Solutions

Single Operational Model

UCS Manager UCS Director Enterprise Cloud Suite

BRKCOM-2004 17


Cisco HyperFlex Hybrid Configurations

HX240c Cluster

Capacity-heavy 3–8 Node Cluster

(VSI: IT/Biz Apps, Test/Dev)

Per-Node1 x 1.6 TB Cache SSD

up to 23 x 1.2 TB HDDs

SD Card/120GB Back SSD

(Boot/Housekeeping)

HX220c Cluster

Smallest Footprint 3–8 Node

Cluster (VDI, ROBO)

Per-Node1 x 480 GB Cache SSD

6 x 1.2 TB HDDs

SD Card/120GB SSD

(Boot/Housekeeping)

Hybrid Clusters

Compute-heavy Hybrid

(Compute Bound Apps/VDI)

Up to 8 Compute Nodes

Blade or RackB200 M4, C220 M4 or C240 M4 (Supported)

Local Disk, SD Card or SAN Boot

3-8 Node HX220 or HX240 Cluster

4

1

5

2

6

3

7 8 UCSC220 M4

Intel

Inside

XEON

Console!

C220 M4

!

Intel

Inside

XEON

UCSC240 M4

Con

sole

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

1 6 12 18 24

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

C240 M4

B200 M4

BRKCOM-2004 18

What’s New In HX 2.0


Introducing All Flash HyperFlex Systems

High Performance

High IOPS, High Throughput,

Consistent, Low Latency

No Compromise Storage

Efficiency

Always-on, inline dedupe

and inline compression

Instantaneous, space

optimized snapshots and

clones

Performance Sensitive Workloads

Support for Databases other performance sensitive workloads

BRKCOM-2004 20


Cisco HyperFlex All Flash Configurations

HXAF240c Cluster

Capacity-heavy 3–8 Node Cluster

(VSI: IT/Biz Apps, Test/Dev)

Per-Node1 x 800 GB WL Cache SSD

up to 10 x 960 GB/3.8 TB SSDs

SD Card/120GB Back SSD

(Boot/Housekeeping)

HXAF220c Cluster

Smallest Footprint 3–8 Node

Cluster (VDI, ROBO)

Per-Node1 x 800 GB WL Cache SSD

6 x 960 GB/3.8 TB SSDs

SD Card/120GB SSD

(Boot/Housekeeping)

Hybrid Clusters

Compute-heavy Hybrid

(Compute Bound Apps/VDI)

Up to 8 Compute Nodes

Blade or RackB200 M4, C220 M4 or C240 M4 (Supported)

Local Disk, SD Card or SAN Boot

3-8 Node HXAF220 or HXAF240 Cluster

4

1

5

2

6

3

7 8 UCSC220 M4

Intel

Inside

XEON

Console!

C220 M4

!

Intel

Inside

XEON

UCSC240 M4

Con

sole

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

1 6 12 18 24

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

UC

S-H

DD

30

0G

I2F

10

5

15

K S

AS

30

0 G

B

!

C240 M4

B200 M4

BRKCOM-2004 21


HX All Flash Improvements

Comparison between HX Hybrid and HX All Flash for 70/30 8K Random Read Write Workload

with 4.8TB Working Set Size, vmdks were initialized with writes

Pre-release Performance Data

Up to 6X IOPS

and 1/5th

Latency

6X Better Perf Consistency

3.2XCapacity / Density

BRKCOM-2004 23


What’s New With Cisco HyperFlex 2.0?

Unified UCS ArchitectureSupport for adding HyperFlex to an

existing UCS Domain

Enhanced User ExperienceImproved Installer User Interface and

Capabilities, Additional Snapshot API and Sizer improvements

New Hardware Qualifications for 3rd Gen Fabric Interconnects and

VIC cards (Post FCS)

HXAF220 and HXAF240 Converged Nodes

BRKCOM-2004 24

© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 25BRKCOM-2004

Adding HX in an existing Cisco UCS Domain

HyperFlex

SLOT

1

SLOT

5

SLOT

3

SLOT

7

SLOT

2

SLOT

6

SLOT

4

SLOT

8

!

UCS 5108

OK FAIL OK FAIL OK FAIL OK FAIL

! ResetConsole

UCS B200 M4

! ResetConsole

UCS B200 M4

! ResetConsole

UCS B200 M4

! ResetConsole

UCS B200 M4

Non-HX

UCS

• Adding HyperFlex to pre-existing

UCS Domains with general UCS

equipment and workloads

• Will be supported 2.0

• HyperFlex QoS requirements

may conflict with existing config

• Consult your CSE and TAC for

guidance and assistance


External Storage Configuration

Added sections for FC / iSCSI storage

• Automates creation of vNICs, vHBA and necessary profiles in UCSM

• Installer automatically fixes VMDirectpath PCI ordering

• Ready for storage configuration inside of ESXi

BRKCOM-2004 26


Sizer Improvements

• End to end sizing guidance: compute, capacity and performance

• Rapid iterative development: Lots of improvements / additions coming

• Next: Support for all flash nodes and support for DBs

Link: https://hyperflexsizer.cloudapps.cisco.com/

(CCO Logon Required)

BRKCOM-2004 28

https://hyperflexsizer.cloudapps.cisco.com/

Cisco HyperFlex Architecture Overview


Building on the Right FoundationCisco HX Data Platform

Built From the Ground Up for

Hyperconvergence

Distributed Log-Structured File

System Designed for Scale-out,

Distributed Storage

Advanced Data Services (Snapshots,

Clones) and Data Optimization

(Inline Dedupe, Compression) Without

Trade-offs

Better Flash Endurance and

Disk Performance

Computing, Storage,

Networking, and Hypervisor Integration

No Reliance on Legacy

Filesystems or

Technology

Distributed File system

Local Disks

Unique

Architecture

Local Disks Local Disks Local Disks

BRKCOM-2004 31


HYPERCONVERGED DATA PLATFORMHYPERCONVERGED DATA PLATFORMHYPERCONVERGED DATA PLATFORMHYPERCONVERGED DATA PLATFORM

Hyperconverged Scale Out and Distributed File System

CONTROLLERVMHYPERVISOR

VM VM VM

Start with as Few as

Three Nodes

Hyperconverged Data Platform

Installs in Minutes

Add Servers, One or More

at a Time

Linearly Scale Compute, Storage

Performance, and Capacity

Distribute and Rebalance Data Across Servers Automatically

Retire Older

Servers


VM VM VM


VM VM VM


VM VM VM


VM VM VM

BRKCOM-2004 32


Software Modules Inside a Server

Controller VM Has Direct Access to Drives

VAAI Plugin Offloads Snapshots and Clone Operations

IO Visor Module Presents NFS to ESX and Stripes IO

DATASTORE/VOLUME

HYPERVISOR

HDD

HDD

SDD

SDD

SSD SSD SSD SSD

CONTROLLERVM

IO Visor

VAAI

VM VM VM VM VM

BRKCOM-2004 33


Read Cache

Log FS

L1

Read

Distributed Objects

All Flash

DRAM

Read Cache

Optimizing for All Flash: Read Path

Log FS

L2

L1

Read

Distributed Objects

Hybrid L1 (DRAM) Cache:

• Metadata and data

• Helps with small bursts

L2 (SSD) Cache:

• De-duplicated Read Cache

• Use aggregate performance of all Read Cache SSDs

SSD

DRAM

No L2 (SSD) Cache:

• No flash on flash

advantage

• Use aggregate

performance of all SSDs

BRKCOM-2004 34

SCVM RAM:

• Unified capacity disk

index cache

• Improvements by

avoiding additional index

reads


Life of a Write IO: Write Log

CONTROLLERVM

IOvisor

VMVM

ESXiDS

CONTROLLERVM

IOvisor

VMVM

ESXiDS

CONTROLLERVM

IOvisor

VMVM

ESXiDS

AA’’ A’

ACKA

1. Application writes data (A) to OS

2. VM writes to VMDK on datastore

3. Write intercepted by IOvisor and sent to primary controller

4. Primary data is committed to WL SSD

5. Simultaneously, data replicated to two other nodes (RF3) and committed to their WL SSD

6. Acknowledgement sent to VM

7. Data is successfully committed and protected

B B’ B’’

CC’’C’

D D’D’’

Write Log Capacity Used

Size: 2G Hybrid / 48G AF

BRKCOM-2004 35


Life of a Write IO: Flush

AA’’ A’

1. When the active write log of a node is full, the active log becomes the secondary log and flushes

2. Primary data copies are deduplicated then compressed

3. Data is then three way written (RF3) to SSD / HDD on 3 separate nodes

4. After all data is successfully committed, data is purged from the secondary log on the caching SSD

5. In hybrid systems, recent writes are added to the read cache area of the SSD during flush

B B’ B’’

CC’’C’

D D’D’’

Write Log Capacity Used

CONTROLLERVM

IOvisor

VMVM

ESXiDS

CONTROLLERVM

IOvisor

VMVM

ESXiDS

CONTROLLERVM

IOvisor

VMVM

ESXiDS

BRKCOM-2004 36

A’A’’AA

BBB’B’’

CC’’C’ D’’ D D’


CONTROLLERVM

IOvisor

VMVM

ESXiDS

CONTROLLERVM

IOvisor

VMVM

ESXiDS

CONTROLLERVM

IOvisor

VMVM

ESXiDS

Life of a Read IO

ESXi

Read A

1. Application reads data from OS filesystem

2. VM reads from VMDK on datastore

3. Read intercepted by IOvisor and sent to primary controllera) First check active write log

b) Next check passive write log (the duplicated write log of the other nodes)

c) Look in L1 (DRAM) cache

d) Look in L2 (SSD) cache [Not in AF models]

e) Retrieve from SSD / HDD – Decompress, read into L1, returned to IOvisor, copied into L2 [Not in AF models]

4. Return data to VM

Active WL

Passive WL

L1

L2

SSD (AF) / HDD (Hybrid)

BRKCOM-2004 37


Integrated Management and Data Services

Dynamic Data Distribution

Continuous Data Optimization

Independent Scaling of Compute and Capacity

HyperFlex Design Principles

BRKCOM-2004 38


Scale Compute

HX Data Platform

Add NodesScale Capacity Within Nodes

HX Data Platform

CONTROLLERHYPERVISORCONTROLLERHYPERVISOR CONTROLLERHYPERVISOR CONTROLLERHYPERVISOR

IOVisorIOVisor

IOVisorIOVisor

IOVisorIOVisor

IOVisorIOVisor

VM VM VM VM

Non-HyperFlex Hosts Can Connect to

Storage with IOVisor

VM VMVM VM VMVM VM VMVM VM VMVM

IOVisor

VM VM

IOVisor

VM VM

Independent Scaling of Compute and Capacity

BRKCOM-2004 39


Dynamic Data Distribution

• HX Data Platform stripes data across all nodes simultaneously, leveraging cache across all SSDs for fast writes, and HDDs for capacity, via the low-latency UCS internal network

• Balanced space utilization: no data migration required following a VM migration, and no hotspots

Systems Built on Conventional File

Systems Write Locally, Then Replicate,

Creating Performance Hotspots

CONTROLLERHYPERVISORHYPERVISOR CONTROLLERHYPERVISOR CONTROLLERHYPERVISOR

VM VMVM VM VMVM VM VMVM

HX Data Platform

VM VMVM

CONTROLLERCONTROLLER

BRKCOM-2004 40


DATASTORE DATASTORE

Non-Disruptive Operations

• Stripe blocks of a file across servers

• Replicate one or two additional copies

to other servers

• Handle entire server or disk failures

• Restore back to original number of copies

• Rebalance VMs and data post

replacement

• Rolling “one-click” software upgrades

CONTROLLERHYPERVISOR

VM VM VM


VM VM VM


VM VM VM


VM VM VM

File.vmdk

D1 E1A1 B1 C1B2 A2 A3C2 C3 D2D3 E2E3 D1E1 B3 B3

EDCBA

BRKCOM-2004 43


Hybrid Capacity Options

3 Node

HX220c

HX240c

4 Node 5 Node 6 Node 7 Node 8 Node

Assumes: Full HDD population at RF3.

Note: The above calculations are before deduplication & compression. Effective capacity will be higher.

Consult with your Cisco CSE for the latest sizing & design guidance.

6.02 TiB 8.03 TiB 10.04 TiB 12.04 TiB 14.05 TiB 16.06 TiB

23.09 TiB 30.79 TiB 38.48 TiB 46.18 TiB 53.88 TiB 61.58 TiB

Usable Cluster Capacity

BRKCOM-2004 45


All Flash Capacity Options

3 Node

HXAF220c

HXAF240c

4 Node 5 Node 6 Node 7 Node 8 Node

Assumes: Full SSD population at RF3.

Note: The above calculations are before deduplication & compression. Effective capacity will be higher.

Consult with your Cisco CSE for the latest sizing & design guidance.

4.82 TiB/ 19.28 TiB

6.43 TiB/ 25.71 TiB

8.03 TiB/ 32.14 TiB

9.64 TiB/ 38.56 TiB

11.25 TiB/ 44.99 TiB

12.85 TiB/ 51.42 TiB

8.03 TiB/ 32.41 TiB

10.71 TiB/ 42.85 TiB

13.39 TiB/ 53.56 TiB

16.07 TiB/ 64.27 TiB

18.75 TiB/ 74.99 TiB

21.42 TiB/ 85.70 TiB

Usable Cluster Capacity with 960GB/3.8TB SSDs

BRKCOM-2004 46


HyperFlex 2.0 Capacity Considerations

• SSD reliability is significantly better than HDD; from 2-10 times lower annual replacement rate in studies by major online providers.

• HyperFlex installer no longer defaults to RF3 for an all-flash system, forcing the end user to make a decision.

• Consider RF2 for all-flash systems that do not require triple data copies or multiple component redundancy.

• RF2 shows 15-40% reduced write latencies in lab testing.

• RF2 can provide clusters up to 128 TiB maximum capacity.

• Keep overall cluster capacity consumed below 70% for best performance.

47BRKCOM-2004

HyperFlex Performance and Benchmarks


HX Hybrid to AF Performance Comparison

0.0

1.0

2.0

3.0

4.0

5.0

6.0

7.0

0.4GB 100GB 800GB 2TB

Per

form

ance

Mu

ltip

le

Effective Working Set Size per NodeTests done on 4 x HX240

Performance With Varying Working Set

All Flash Performance Multiple

BRKCOM-2004 49

HX240 hybrid read cache 1.2 TB per node

• AF offers performance improvements and lower latency even at small working sets

• AF offers huge performance and latency improvements for working sets that do not fit in the hybrid model read cache


Benchmark 1

Base Performance Using vdbench Curve Tests

HX-All Flash HX Hybrid

Version HX 2.0.1 HX 1.8.1c

HW Configuration 4 x HXAF240, each with:

• WL: 1 x 400G

• Data: 8 x 960G

4 x HX240, each with:

• Cache/WL: 1 x 1.6TB SSD

• Data: 23 x 1.2TB 10Krpm SAS

HDD

Benchmark Setup • 3 VMs per host

• Each VM has 1 x 400GB vmdk

• 4.8 TB working set

• Priming: 1MB writes

• Tests with 32 threads

BRKCOM-2004 50


Benchmark 1

Base Performance Using vdbench Curve Tests8K, 100% Random Write Workload

0

5

10

15

20

25

30

35

40

0 5000 10000 15000 20000 25000 30000 35000 40000

Late

ncy

(m

s)

IOPS

8K, 100% Write

HX Hybrid

HX All Flash

3.3X Higher

IOPS at 3.5X

Lower latency

BRKCOM-2004 51


8K, 70% Read, 30% Write, 100% Random Workload

6.3X Higher

IOPS at 5X

Lower latency

0

5

10

15

20

25

30

0 10000 20000 30000 40000 50000 60000 70000 80000 90000

Late

ncy

(m

s)

IOPS

8K, 70% Read, 30% Write

HX Hybrid

HX All Flash

Benchmark 1


BRKCOM-2004 52


8K, 100% Random Read Workload

10.7X Higher

IOPS at 5.7X

Lower latency

0

1

2

3

4

5

6

7

8

9

0 50000 100000 150000 200000 250000 300000

Late

ncy

(m

s)

IOPS

8K, 100% Read

HX Hybrid

HX All Flash

Benchmark 1


BRKCOM-2004 53


Benchmark 1

Base Performance Using vdbench Curve TestsVariability Measured By Standard Deviation of Latency

HX Hybrid HX All Flash Ratio

8K, 100% Write39.4 11.6 3.4

8K, 70 / 30 RW74.3 5.1 14.6

8K, 100% Read16.2 1.1 14.2

Significantly

more

consistent

performance

BRKCOM-2004 54


Benchmark 2

SQL Performance Using vdbench Curve TestsHX-All Flash HX Hybrid

Version HX 2.0.1 HX 1.8.1c


• WL: 1 x 400G

• Data: 8 x 960G

4 x HX240, each with:

• Cache/WL: 1 x 1.6TB SSD

• Data: 23 x 1.2TB 10Krpm SAS

HDD

Benchmark Setup • 3 VMs per host

• Each VM has 1 x 400GB vmdk

• 4.8 TB working set

• Priming: 1MB writes

• Test with 32 threads• Read Block Size Mix: 4k 2.39%, 8k 53.57%,16k 4.84%, 28k 2.64%, 64k 28.75%,120k

1.79%, 220k 1.41%,492k 4.24%, 524k 0.37%

• Write Block Size Mix: 4k 27.85%, 8k 35.24%,16k 4.82%, 28k 2.58%, 64k 26.35%,

116k 0.6%, 244k 1.28%, 488k 0.38%, 536k 0.89%, 1032k 0.01%

• Overall Avg Blocksize: 43.4K; Overall Read %: 68.5%

BRKCOM-2004 55


Benchmark 2

SQL Performance Using vdbench Curve Tests

4.5X Higher IOPS

at 4.6X Lower

latency with 7.3X

better consistency

0

5

10

15

20

25

30

35

0 5000 10000 15000 20000 25000 30000

Late

ncy

(ms)

IOPS

SQL Profile Using vdbench

HX Hybrid HX All Flash

BRKCOM-2004 56


SQL Performance Using HammerDBHX-All Flash

Version HX 2.0.1


• WL: 1 x 800G

• Data: 10 x 960G

Benchmark Setup • 1 SQL Server 2014 VM

• Windows Server 2012 R2

• 28 vCPU

• 64 GB RAM

• ~500 GB DB created by HammerDB with 5000 warehouses

• 500 User Test

• 5 minute ramp up, 60 minute test

BRKCOM-2004 57

© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 58BRKCOM-2004

HammerDB SQL TPC-C OLTP Test Results

HX All-Flash

SQL Transactions Per Minute 1.475 million

Average IOPs 15,141

Average Read Latency 0.78 ms

Read Latency Standard Deviation 0.32

Average Write Latency 1.36 ms

Write Latency Standard Deviation 0.36

Performance limited by CPU

power, all CPU cores running

at ~100% long before

storage performance or

latency becomes an issue.

Competitive Performance and Benchmarks


Cisco HyperFlex HybridRuns more VMs per Cluster

0

20

40

60

80

100

120

140

160

Cisco HyperFlex Vendor X Vendor Y Vendor Z

Total VMs per 4 node cluster 70/30 R/W 4K

Number of VMs per Cluster Write Latency

• Cisco HyperFlex can

run more VMs per

cluster than other

vendors

• Tests found the number

of VMs other vendors

can run with similar

Write Latency as HX

• Simulates heavy VM

load

Up to 3X

the VMs

on HX

BRKCOM-2004 60


Cisco HyperFlex Hybrid140 VMs per 4 Node Cluster Latency Averages

0

5

10

15

20

25

Cisco HyperFlex Vendor X Vendor Y

70/30 R/W 4K

Write Latency Read Latency

• Cisco HyperFlex has

lower R/W latency for

140 VMs per cluster

than other vendors

• Simulated heavy VM

load

• Vendor Z could not run

the load @ 140 VMs

HX has up to

3X + lower

latency

BRKCOM-2004 61


4K Random

70% RD/ 30%

WR

R/W Latency

8K Random

70% RD/ 30%

WR

R/W Latency

HX 136243 1.216 / 4.166 120901 1.208 / 4.936

Vendor Y 121463 1.467 / 5.997 97437 1.704 / 7.148

HX vs Vendor Y 12.2% 20.4 / 30% 24.1% 29.3 / 30.8%

4K Random

70% RD/ 30%

WR

R/W Latency

8K Random

70% RD/ 30%

WR

R/W Latency

HX 136243 1.216 / 4.166 120901 1.208 / 4.936

Vendor Y 99951 1.355 / 6.682 70409 2.889 / 7.910

HX vs Vendor Y 36.3% 7.4 / 37.4% 71.7% 58.8 / 37.9%

Dataset with no dedupe and no compression savings

Dataset with dedupe and compression

• 2.8TB Working Set

• HX significantly better

at write Latency

• HX much better IOPs

with dedupe and

compression

• HX better performance

at 8K block sizes

• As the working set

increases the

performance continues

to drop

Benchmark 3

Base Performance Using vdbench Curve TestsHX All Flash vs Vendor Y All Flash (Unprimed Dataset)

BRKCOM-2004 62


Benchmark 3

Base Performance Using vdbench Curve Tests8K, 100% Random Write Workload

• 61% Higher

IOPS

• 42% Lower

latency

• 60% lower

standard

deviation

0

2

4

6

8

10

12

14

16

18

0 5000 10000 15000 20000 25000 30000 35000 40000

Late

ncy

(m

s)

IOPS

8K, 100% Write

HX All Flash

Competitor Y

BRKCOM-2004 63


8K, 70% Read, 30% Write, 100% Random Workload

• 11% Higher

IOPS

• 24% Lower

latency

• 59% lower

standard

deviation

0

1

2

3

4

5

6

0 10000 20000 30000 40000 50000 60000 70000 80000 90000

Late

ncy

(m

s)

IOPS

8K, 70% Read, 30% Write

HX All FlashCompetitor Y

Benchmark 3


BRKCOM-2004 64


8K, 100% Random Read Workload

• 43% Higher

IOPS

• 34% Lower

latency

• 59% lower

standard

deviation

0

0.5

1

1.5

2

2.5

0 50000 100000 150000 200000 250000 300000

Late

ncy

(m

s)

IOPS

8K, 100% Read

HX All Flash

Competitor Y

Benchmark 3


BRKCOM-2004 65


0

5000

10000

15000

20000

25000

30000

1 7

13

19

25

31

37

43

49

55

61

67

73

79

85

91

97

10

3

10

9

11

5

12

1

12

7

13

3

13

9

14

5

15

1

15

7

16

3

16

9

17

5

18

1

Intervals

Vendor Y

VM1 VM2 VM3 VM4 VM5 VM6


IOPS in a 70/30 Workload – 1 Hour Duration

0

5000

10000

15000

20000

25000

30000

1 8

15

22

29

36

43

50

57

64

71

78

85

92

99

10

6

11

3

12

0

12

7

13

4

14

1

14

8

15

5

16

2

16

9

17

6

Intervals

HX



Benchmark 3


BRKCOM-2004 66


All Flash POC Best PracticesAll Flash Performance Test Setup

1. Initialize the disks• More realistic - Applications don’t read data they haven’t written

2. Longer Test Duration:• SSDs / All flash systems show very high performance initially – which not realistic

• Tests should be long running (12 – 24 hours depending on write load)

3. Larger Working set size• Ability to support larger working set size (several TBs / node) is a key differentiator for all flash

• For smaller working set size hybrid will perform similar to all flash (as data is coming from flash)

4. Enable Dedupe and compression on competition (On by default on HX)• Most enterprise workloads see significant storage efficiency savings – this is not optional

• Expecting users to choose this on a workload by workload basis is not realistic with 100s of VMs

BRKCOM-2004 67


All Flash POC Best PracticesAll Flash Performance Test Setup

5. Workload Read / Write Mix:• Reasonable write percentage (20-35% at least) in the workload

• 100% - 95% read workloads are not realistic in most environments

6. Similar hardware configuration:• Pay careful attention to the BOM – CPU, Memory and SSD (type and count for cache/WL and

persistent data SSDs)

• Additional performance may come at a trade-off: cost or manageability

7. Similar software configuration:• Pay careful attention to the resiliency setting – that needs to be identical

• Comparing Replication Factor = 2 performance with Replication Factor = 3 performance is not comparing apples to apples

BRKCOM-2004 68


All Flash POC Best Practices1. What to look for in performance results:

• IOPS / Throughput

• Latency – not all workloads show sub millisecond latencies (even on all flash!)

• Consistent Latency / standard deviation of latency

Which one would you rather have?

Write Latency as

measured by

esxtop for similar

workloads

BRKCOM-2004 69


All Flash POC Best Practices2. Variance in performance seen across VMs

3. Compare Cost:• Especially important when comparing different hardware configurations

• Metrics: $/GB Effective And $/IOPS

BRKCOM-2004 70


Performance Summary

HX All Flash performance significantly better than both HX Hybrid and other HCI Competitor’s all flash

All Flash performance is more consistent in comparison to HX Hybrid and competition

HX All Flash, unlike competition, shows highly consistent performance across VMs

All realistic workloads should see significantly better performance with all flash than hybrid

BRKCOM-2004 71

Q & A


Complete Your Online Session Evaluation

Don’t forget: Cisco Live sessions will be available for viewing on-demand after the event at CiscoLive.com/Online

• Please complete your Online Session Evaluations after each session

• Complete 4 Session Evaluations & the Overall Conference Evaluation (available from Thursday) to receive your Cisco Live T-shirt

• All surveys can be completed via the Cisco Live Mobile App or the Communication Stations

73BRKCOM-2004

CiscoLive.com/Online


Continue Your Education

• Demos in the Cisco campus

• Walk-in Self-Paced Labs

• Lunch & Learn

• Meet the Engineer 1:1 meetings

• Review related session content:

• BRKINI-4004 What’s New in Cisco HyperFlex 2.0

• BRKVIR-2700 HyperFlex Configuration Best Practices and Troubleshooting

74BRKCOM-2004


Cisco Spark Ask Questions, Get Answers, Continue the Experience

Use Cisco Spark to communicate with the Speaker and fellow participants after the session

Download the Cisco Spark app from iTunes or Google Play

1. Go to the Cisco Live Berlin 2017 Mobile app

2. Find this session

3. Click the Spark button under Speakers in the session description

4. Enter the room, room name = BRKCOM-2004

5. Join the conversation!

The Spark Room will be open for 2 weeks after Cisco Live

75BRKCOM-2004

Cisco HyperFlex 2.0 All Flash Systems Offer High Performance, Low Latency, and Consistent Results

Thank You

Documents

Cisco HyperFlex Deep Dive and Performanceclnv.s3.amazonaws.com/2017/eur/pdf/BRKCOM-2004.pdfmay conflict with existing config • Consult your CSE and TAC for guidance and assistance