All-Flash Solution Invicta - Cisco...Cisco UCS Invicta Series Up to 1.2 Million IOPS Up to 9.6 GBps** Bandwidth Up to 144 TB Raw UCS Invicta Appliance UCS Invicta Scaling System Scalability

All-Flash Solution Invicta

Name Presenter Titel Presenter

Datum

A New Generation of Performance

Cisco Confidential 2© 2013-2014 Cisco and/or its affiliates. All rights reserved.

Overview of Storage Concepts HDD/SSD Basics UCS Invicta OS UCS Invicta Series UCS Invicta Management Cisco UCS Services Use Cases

Topics


Overview of Storage Concepts


Source: Lazard Capital Market Research

Storage Decisions

Tape Capacity Disk Performance Disk Flash Memory/Solid State

Backup and Archiving

Data optimization technologies (e.g.

Backup and Archiving

Data optimization technologies (e.g.

All-Flash-Arrays

Capacity OptimizedPerformance Optimized

Hybrid Arrays

Caching SWCaching SW PCIe Flash Cards/SSD

Enterprise Disk Arrays

PCIe card deployment will require advanced caching in combination with some form of capacity optimized disk array in order to replicate benefits of shared storage.

Most legacy storage vendors now support standard form factor SSDs within disk arrays as Tier-1 performance optimized capacity and could be considered “hybrid” arrays.

SSD in Array Designed for HDD

Tape

Optical

7,200 RPM7,200 RPM 10,000 RPM10,000 RPM 15,0000 RPM15,0000 RPM Solid State Solid State

Overview of Enterprise Storage Concepts


All-Flash-Architectures

Scale Up Array

•Performance is driven by controllers •Adding Storage Shelves Increases capacity

•Can’t Increase IOPS or Throughput without upgrading controllers

•Presents a Single Storage Pool

Scale Up Array

•Performance is driven by controllers •Adding Storage Shelves Increases capacity

•Can’t Increase IOPS or Throughput without upgrading controllers


Scale Up/Out Array

•Leverages the capacity, power and capabilities of each node (appliance)

•Specialized routers manage nodes as one or more storage pools

•Adding Nodes increases capacity and IOPS

•Presents Multiple Storage Pools

Scale Up/Out Array

•Leverages the capacity, power and capabilities of each node (appliance)

•Specialized routers manage nodes as one or more storage pools

•Adding Nodes increases capacity and IOPS

•Presents Multiple Storage Pools

Appliance •Single unit•Does not scale•Can offer a variety of features/capabilities


Appliance •Single unit•Does not scale•Can offer a variety of features/capabilities


Scale Out Storage Cluster • Increase capacity and performance by adding a node

•Each node adds a small amount of capacity, IOPS and throughput to the cluster

•Nodes are generally uniform•Presents a Single Storage Pool

Scale Out Storage Cluster • Increase capacity and performance by adding a node

•Each node adds a small amount of capacity, IOPS and throughput to the cluster

•Nodes are generally uniform•Presents a Single Storage Pool

All Flash Architectures General Rules

All Flash Architectures General Rules


HDD/SSD Basics


HDD Story• Access Time

- Rotational latency- Seek time

• Performance still since 1st

developed 15k rpm disk in 1999- Random r/w performance is

where spinning disks fall short – on average 200 IOPS

• Techniques to optimize performance- Short stroking- Read-ahead, Write-back

Drive level cache• RAID penalty effects

- Disk sprawl• Hard Disk easy to overwrite

- Magnetic process


Flash Cell vs HDD

NAND Flash Cell

Mechanical Hard Drive

Flash memory is a type of EEPROM chip consisting of a Floating gate transistor which uses a dielectric layer between the Control Gate and an additional Floating Gate:• NAND logic: all inputs=‘1’ > output=‘0’; all

other combinations > output=‘1’

Unlike HDD, data cannot be directly overwritten in SSD (P/E Cycle)


NAND Flash Types

Single Layer Cell (SLC)• 100,000 P/E cycles• ExpensiveMulti-layer Cell (MLC)• Used in Invicta• 5000 P/E cycles• 4 times lower costTriple Level (TLC) • 1000-1500 P/E cycles• Low cost and low enduranceeMLC (Enterprise) • Mitigates degradation of

MLC

NAND ARRAYSLC MLC TLC

1 bit/cell 2 states

2 bits/cell 4 states

3bits/cell8 states

34nm 2x nm 2x nm100K (P/E) cycles/cell

3K - 5K (P/E) cycles/cell

1K (P/E) cycles/cell

000

000001

01010011

110

100101

11110111


Nature of flash: P/E cycles and wear out

ERASE

PROGRAM(WRITE)

NAND example


erasing takes much more time than programing

organize erase data in greater units than program (write) data: data unit: 4KB page erase unit: 1MB block (256x 4KB pages) as Erase Block (EB)

Freeing up 4KB pages to an entire EB for erase is so called“garbage collection“.

moving valid data for garbage collection is so called “writeamplification“. Frontend writes will be amplified at thebackend and therefore wearing out cells.

Garbage Collection / Write Amplification


Write Amplification, Garbage Collection, Wear Levelling

• This is what happens in standard Drive controller• In UCS Garbage Collection is background task managed by UCS Invicta OS• Wear Levelling is managed by UCS Invicta OS logging of drive media usage

Write amplification!Causes more wear and write IO than necessary

More data writtenOriginal pages

Marked unusableBlock now full

Initial write toSpare pages

Garbage collection

Wear levelling tries tochoose block for copy with even distribution

1 2 3

Any change In data within Block causes

Data to be moved To new block


UCS Invicta OS


Invicta OS Eliminates Trade-Offs

Data Persistence

Fastest Performance

Highest Protection

Write Protection Buffer

Block Translation Layer

RAID Layer

Flash Media


Invicta Datamanagement (dedupe off)

1. Protect • Store in Write Buffer 1. Protect • Store in Write Buffer

2. Organize • Create Write Blocks2. Organize • Create Write Blocks

3. Optimize • Write Aligned for Flash Media & RAID Protection



1. Protect • Store in Write Buffer 1. Protect • Store in Write Buffer

2. Pattern Match & Organize • Deduplicate 4K Blocks• Create Write Blocks

2. Pattern Match & Organize • Deduplicate 4K Blocks• Create Write Blocks



Invicta Datamanagement (dedupe addon)


4. Optimize Writes• Write speeds are symmetric to read speeds

4. Optimize Writes• Write speeds are symmetric to read speeds

5. Virtual Garbage Collection• Evaluated and Managed by the Invicta OS

5. Virtual Garbage Collection• Evaluated and Managed by the Invicta OS

6. Virtual Garbage Collection• Blocks are invalidated in large chunks to speed up drive level garbage collection

6. Virtual Garbage Collection• Blocks are invalidated in large chunks to speed up drive level garbage collection

Invicta Datamanagement (con’t)


Only unique data segments are written.

AB

CD

Data is already there, so only additional pointers are stored –not the second instance of the data.

A B C D

Blocks of Data Written

E

E New data segment identified and written

First Instance Duplicate Instance Modified Instance

A B

C D

A B

C D

B

C D

E


Invicta OS Data Reduction details

• Every incoming write is hashed a 4K boundary• Hash function is high performance (MM3)

• Hash is compared to all existing hashes in MEMORY• Using the hash as an array index

• Upon match• Media read verifies the duplication

• A miss-match forces a unique store• A full match stores a pointer

* VAAI is only supported in performance nodes


UCS Invicta FeaturesAppliance Scaling System

Architecture INVICTA OS 5.0.0 5.0.1

Symmetric Read/Writes ✔ ✔

RAID Protection ✔ ✔

Asynchronous Replication ✔ ✔

Snapshots – Copy on Write ✔ ✔

Mirroring ✔

Web Based UI/API ✔ ✔

Role Based Access Controls ✔ ✔

Data Reduction Option ✔ ✔

Thin Provisioning ✔ ✔

iSCSI, Fibre Channel ✔ ✔

Enhanced Data Protection ✔ ✔

VAAI Support* ✔ ✔

Ethernet, Fibre Channel ✔ ✔

Increase Performance and Capacity ✔

Scaling System Fabric Point to Point

Max. Routers 2

Max Nodes 6* VAAI is only supported in performance nodes


RAID Effect: Reducing backend IOpS


The RAID EffectProtecting data requires writing data

RAID 1/10 • Comprises mirroring (without parity or striping).

• Write Penalty = 2

RAID 5 • Comprises block-level striping with distributed parity.


RAID 6 • Comprises block-level striping with double distributed parity


Every protected write must be included in the total IOPS consideration

The protected writes are a penalty because they represent overhead subtracted from the IOPS pool


RAID Effect

Storage SystemStorage System

• RAID 5• 2,500 IOPS

ApplicationApplication

•1,000 IOPS• 50/50 Write/Read

Storage SystemStorage System

• RAID 5 • 4,000 IOPS

ApplicationApplication

• 1,000 IOPS• 100% Write

Not @ Invicta!


UCS Invicta Series


Cisco UCS Invicta Series

Up to 1.2 Million IOPSUp to 9.6 GBps** BandwidthUp to 144 TB Raw

UCS Invicta Appliance

UCS InvictaScaling System

Scalability

Modularity

Application Acceleration

Data Optimization

Multiple Workloads

Tuning-Free Performance

250,000 IOPS1.2 GBps Bandwidth3, 6, 12, 24 TB Raw

6 Node Configuration


Single Control PlaneAppliance Breakdown




Mode ApplicationAcceleration

Data Reduction

Throughput (GBps) 1.2 1.2100% Read IOPS 220,000 250,000Blended IOPS (50%read/50%write)

205,000 180,000

100% Write IOPS 200,000 120,000Latency (Microseconds) <100 <100Size 2 RU 2 RUMax Capacity (TB) 24 TB Raw 64 TB**

**addressable capacity


Cisco UCS Invicta Appliance Hardware and Management Connectivity Details

• Sizes are 3TB, 6TB, 12TB and 24TB –always uses 24 drives, no hot spare

• Different part numbers depending on capacity

• Dual redundant 1200W PSU and Fans - 6 internal hot swap fans

• Power draw around 350W

• Hardware is UCS C-240 M3 • 5 x PCIe slots for IO cards and

switching fabric connection• Dedicated CIMC management

port• Use ports 1 and 2 on LOM for

Management GUI access

• Replication software – one part number

24 Drive baysKVM

connectivityPowerbutton

LEDs

Powersupplies 4 x 1Gbps LOM

P1&2 - 1Gbps mgmt.Serial

CIMC.


DistributedControl Plane

Scaling System Breakdown


Adapter Installation for C710R Scaling System Router

User can populate slots 2 and 5 with the following options:• UCSW-PCIE-IM4TG – Interface

Masters 32714L Quad Port 10 GbE Server Adapter SFP+

• UCSW-PCIE-Q2564 – Qlogic QLE2564 Quad Port 8Gb Fibre Channel HBA SFP+

PCIe Slot Description1 Mellanox Infiniband HCA2 Interface Card 13 Mellanox Infiniband HCA4 Mellanox Infiniband HCA5 Interface Card 2


Adapter Installation for UCS Invicta Scaling System Nodes

• No user-selectable interface cards on Nodes

• Mellanox IB HCA Card for connectivity to the low latency backplane to SSRs only

• LSI RAID controller is for connectivity of locally installed SSDs

PCIe Slot Description1 NVRAM Card2 Mellanox Infiniband HCA3 Not Populated (Reserved)4 LSI RAID Controller5 Not Populated (Reserved)


UCS Invicta Scaling System Architecture

Silicon Routers

Workload Acceleration

and Data Reduction

Silicon Nodes

UP TO 6UP

TO 6


SSNSSN

SSNSSN

SSRSSR

SSNSSN

SSRSSR

SSNSSN

SSNSSN

SSNSSN

P

M

P

M

P

M

UCS Invicta Scaling System Mirroring


Flexible Replication Options

Appliances to Scaling Array Scaling Array to Scaling Array

Appliance to Appliance Appliance to Open Target

Linux/WindowsLinux/Windows


UCS InvictaManagement


Administrative GUI

• Main management tool now for Appliance and Scaling System

• Simple intuitive GUI driven by mouse left and right click operations

• Works with a browser

• Actions • Configuration management• Alerting and monitoring• Capacity planning• Protection management• Operational changes


Cisco Management Overview

Manages multiple UCS

Domains

Manages ACI Fabric

UCS CENTRAL

UCS DIRECTOR Centralized infrastructure control point for data center

APIC

INTELLIGENT AUTOMATION FOR CLOUD Private Cloud, PaaS (DevOps), Hybrid Cloud

Manages: UCS Invicta, Nexus, Heterogeneous and

Converged infrastructures

Manages Single UCS domain

UCS MANAGER


Cisco UCS Service


Advanced Service Fixed

Cisco UCS Preproduction Pilot ServiceProvides planning, design, and implementation expertise to conduct a pilot deployment in the customer's environment. The goal is to verify that the project will deliver the expected business and technical objectives within three weeks in one segment of the data center.

Cisco UCS Accelerated Deployment ServiceProvides planning, design, and implementation expertise to bring a unified computing project into production within four weeks in one segment of your customer’s data center.


UCS Customized Services

Cisco UCS Planning, Design, and Implementation Service

Provides field-proven activities and deliverables along with industry-leading expertise that can be customized to a specific scope for Cisco UCS B-Series Blade Servers and C-Series Rack Servers and Cisco UCS Invicta Scaling Systems

Cisco UCS Data Migration Service

Helps customers smoothly migrate workloads to Cisco UCS; includes customizable migration planning and support to enable successful transitions to Cisco UCS solutions, including Cisco UCS Invicta Scaling Systems

Cisco UCS Application Services

Helps customers with a variety of application needs, including portfolio rationalization, cloud enablement, migration, modernization, and management and automation


Technical Service

Cisco SMARTnet® Service for Unified Computing helps customers optimize uptime and availability and quickly resolve issues. Cisco SMARTnet Service for Cisco UCS includes:

• Worldwide access 24 hours a day, 365 days a year to specialized technical experts in the Cisco Technical Assistance Center (TAC)

• Triage support for third-party software• Flexible hardware replacement options including: 8x5xNBD, 8x5x4, 24x7x4

and 24x7x2• Extensive self-help support • All warranty features

Cisco SMARTnet for UCS is required for the first year with the purchase of the Cisco UCS Invicta products


Software Application Support Service SAS

Cisco Software Application Support Service is essential to keeping business-critical applications available, secure, and operating optimally. Cisco SAS includes:

• Registered access to Cisco.com

• 24 x 7 x 365 access to Cisco Technical Assistance Center (TAC), Cisco software specialists

• Maintenance and minor software release updates

Cisco SAS is required for the first year with the purchase of the UCS InvictaDedupe Software and/or the purchase of UCS Invicta Replication software.


Use Cases


What workloads for UCS Invicta

Workload Details Solution

Non Persistent VDI Application StreamingGraphic Intensive VDI

Standard Non PersistentVDI

Appliance

Persistent VDI Standard Persistent VDI Scaling with DeDupe

Databases OracleMicrosoft SQL

Scaling

Analytics Cassandra,Hortonworks, MongoDB

Grid ArchitecturesIndex’s

Appliance

Questions?

Thank you.

Documents

All-Flash Solution Invicta - Cisco...Cisco UCS Invicta Series Up to 1.2 Million IOPS Up to 9.6 GBps** Bandwidth Up to 144 TB Raw UCS Invicta Appliance UCS Invicta Scaling System Scalability