DS8000 - Training

© Copyright IBM Corporation 2001. All rights reserved. © Copyright IBM Corporation 2001. All rights reserved. IBM TotalStorage™

ibm.com/storage

DS8000 Series


Architecture

aspects of the DS8000 series architecture

© Copyright IBM Corporation 2001. All rights reserved. © Copyright IBM Corporation 2005. All rights reserved.

IBM TotalStorage™

DS8000 Frame Possibilities

disk enclosure pair

disk enclosure pair

BatteryBackup unit

Primarypowersupply

BatteryBackup unit

I/O

enclosure 0I/O

Enclosure 1

I/Oenclosure 2

I/O

Enclosure 3

disk enclosure pair

disk enclosure pair

Primarypowersupply

Rack

power

control

eServer p5 570

eServer p5 570

Cooling plenum

disk enclosure pair

disk enclosure pair

disk enclosure pair

disk enclosure pair

disk enclosure pair

disk enclosure pair

disk enclosure pair

disk enclosure pair

Battery

Backup unit

Battery backup unit

Battery

Backup unit

Primarypowersupply

Primarypowersupply

Cooling plenumFan

sensecard

I/Oenclosure 4

I/O

Enclosure 5

I/Oenclosure 6

I/OEnclosure 7

disk enclosure pair

disk enclosure pair

disk enclosure pair

disk enclosure pair

disk enclosure pair

disk enclosure pair

disk enclosure pair

disk enclosure pair

Primarypowersupply

Primarypowersupply

Cooling plenumFan sensecard

BatteryBackup unit


IBM TotalStorage™

SAN fabric

host adapterhost adapter

Persistent memory

VolatilememoryN-way

SMP Persistent memory

Volatilememory N-way

SMP

RIO-G

Complex 0

Complex 1

RIO-G device adapterdevice adapter

Fibre channel switchFront storageenclosure with

16 DDMs

Rear storageenclosure with

16 DDMs

Fibre channel switchFibre channel switchFibre channel switch

DS8000 Architecture


Virtualisation

virtualization concepts as they apply to the DS8000


IBM TotalStorage™

RIO-G

Processor Processor

Memory

Memory

I/O I/O

RIO-G

Processor Processor

Memory Memory

I/O I/ORIO-G

Processor Processor

Memory Memory

I/O I/OLIC LIC LIC LIC

LPAR Hypervisor

Storage Facility image 1

Storage Facility image 2

Physicalview:physicalstorageunit

Logicalview:virtualStorageFacilityimages

takes part of takes part of

takes part of

Storage Facility Virtualisation


IBM TotalStorage™

Server0 Server1

RIO-GI/O Enclosure

DA

HA

HA

HA

HA

DA

Storage enclosure pair

Switched loop 1

Switches

Switched loop 2

I/O Enclosure

DA

HA

HA

HA

HA

DA

Physical Layer


IBM TotalStorage™

Array Site

Switch

Loop 1 Loop 2

Array Site


IBM TotalStorage™

Array Site

RAIDArray

Spare

DataDataDataDataDataDataParity

Creation ofan array

D1 D7 D13 ...

D2 D8 D14 ...

D3 D9 D15 ...

D4 D10 D16 ...

D5 D11 P ...

D6 P D17 ...

P D12 D18 ...

Spare

Creation of an Array


IBM TotalStorage™

FB Rankof 1GBextents

Creation ofa Rank

D1 D7 D13 ...

D2 D8 D14 ...

D3 D9 D15 ...

D4 D10 D16 ...

D5 D11 P ...

D6 P D17 ...

P D12 D18 ...

RAIDArray

Spare

Data

Data

Data

DataData

DataParity

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

1GB 1GB 1GB 1GB

Forming a Rank


IBM TotalStorage™

1GBFB

1GBFB

1GBFB

1GBFB

Server0

1113Cyl.CKD

1113Cyl.CKD

1113Cyl.CKD

1113Cyl.CKD

Server1

1113Cyl.CKD

1113Cyl.CKD

1113Cyl.CKD

1113Cyl.CKD

Extent Pool CKD0 Extent Pool CKD1

Extent Pool FBtest

Extent Pool FBprod

1113Cyl.CKD

1113Cyl.CKD

1113Cyl.CKD

1113Cyl.CKD

1GBFB

1GBFB

1GBFB

1GBFB

1GBFB

1GBFB

1GBFB

1GBFB

1GBFB

1GBFB

1GBFB

1GBFB

1GBFB

1GBFB

1GBFB

1GBFB

Extent Pools

For performanceand ease ofmanagementcreate 1 ExtentPool per Rank


IBM TotalStorage™

1 GB 1 GB 1 GB 1 GBfree

Extent Pool FBprod

Rank-a

Rank-b

3 GB LUN

1 GBfree

1 GBfree

used used Allocate a 3 GB LUN

1 GB 1 GB 1 GB 1 GBused

Extent Pool FBprod

Rank-a

Rank-b

3 GB LUN

1 GBused

1 GBused

used used

Logical 3 GB LUN

2.9 GB LUNcreated

100 MB unused

Creation of an FB LUN


IBM TotalStorage™

LSS X'17'DB2

LSS X'18'DB2-test

Physical Drives Logical VolumesPhysical Drives

Grouping of Volumes in LSSs

Create 1 LSS perExtent Pool

LSS automaticallycreated whencreating FB LUN


IBM TotalStorage™

Logical Storage Subsystems


IBM TotalStorage™

WWPN-1

WWPN-2

Host attachment: AIXprod1

Volume group: DB2-1

WWPN-3

WWPN-4

Host attachment: AIXprod2

Volume group: docs

WWPN-5

WWPN-6WWPN-7

WWPN-8

Host att: Test

Host att: Prog

Volume group: DB2-2

Volume group: DB2-test

Host Attachment & Volume Groups


IBM TotalStorage™

Server0

Array Site

RAIDArray

Spare

Data

Data

Data

Data

Data

Data

Parity

RankType FB

1 GB FB1 GB FB1 GB FB



ExtentPool

LogicalVolume

Virtualisation Hierarchy


Performance Considerations

early performance considerations regarding the DS8000 series


IBM TotalStorage™

LV stripped across 4 LUNs

Extent

Extent PoolRank 5

Rank 6

Rank 7

Rank 8

8GB LUN

Extent

1GB

Rank 1 Extent pool 1

Rank 3

Rank 4

2 GB LUN 1

1GB

Extent Pool 5

Rank 2

Non-balanced method: LUNs across ranks

More than 1 rank per extent pool

2GB LUN 2

2GB LUN 3

2GB LUN 4

Extent pool 2Extent pool 2

Extent pool 3

Extent pool 4

Balanced method: LVM striping

1 rank per extent pool

Spreading Data Across Ranks


IBM TotalStorage™

RIO-G Module

RIO-G Module

HA

Extent pool11

Rank

Extent pool9

Extent pool7

Extent pool5

Extent pool3

Extent pool1

1197

531

8 1210

2 64

Extent pool10

Extent pool8

Extent pool6

Extent pool4

Extent pool2

Extent pool0

RIO-G ModulePOWER5 2-way SMP

Server0

Memory

L3 Memory

Processor

Processor

L1,2 Memory

L1,2 Memor

y

RIO-G ModulePOWER5 2-way SMP

Server1

Memory

L3 Memory

Processor

Processor

L1,2 Memory

L1,2 Memor

y

DAs

Extent Pool Affinity


Implementation Tasks


IBM TotalStorage™

Hardware install - IBM CEDS HMC - interna land external (if fitted)Call Home and Remote SupportDS Storage Manager - CLI

Must use DS CLI package shipped with DS CodeDS Storage Manager - GUI

http://<ipaddress>:8451/DS8000/LogonNote: DS Stoage Manger on Internal HMC can only provide real-time configuration

User MangementDefault logon to DS GUI and CLI is admin/adminPassword must be changed straight away: chuser -pw passw0rd admin

Activation of License CodesRequired before any logical configuration can be doneObtain machine information: serial number, machine signatureEnsure you have the IBM Licensed Function Authorisation DocumentGo to http://www.ibm.com/storage/dsfaFollow instrcutions and note down license actiation codeApply activation codes to storage image using DS GUI

Physical Installation


IBM TotalStorage™

Must use CLI code shipped with DS code

Runs on all OS

Create profiles for each DSWindows - stored in C:\Program Files\IBM\dscli\Profile\dsxxxx.profile## DS CLI Profile## Management Console/Node IP Address(es)# hmc1 and hmc2 are equivalent to -hmc1 and -hmc2 command options.hmc1: 9.149.20.163hmc2: 9.149.20.173## Default target Storage Image ID# "devid" and "remotedevid" are equivalent to # "-dev storage_image_ID" and "-remotedev storeage_image_ID" command options, respectively. devid: IBM.2107-75AZRY1

Start DS CLIdscli -cfg "C:\Program Files\IBM\dscli\profile\FOXDS8300_2.profile"

DS CLI


IBM TotalStorage™

Physical Configuration - CLI

Configure I/O Ports


IBM TotalStorage™

List I/O Ports - lsioport

dscli> lsioport

Date/Time: 14 February 2006 15:19:02 GMT IBM DSCLI Version: 5.1.0.236 DS: IBM.2107-75AZRY1

ID WWPN State Type topo portgrp

===============================================================

I0000 50050763040000D0 Online Fibre Channel-SW SCSI-FCP 0



I0003 500507630400C0D0 Online Fibre Channel-SW SCSI-FCP 0









I0130 50050763040B00D0 Online Fibre Channel-SW SCSI-FCP 0



I0133 50050763040BC0D0 Online Fibre Channel-SW SCSI-FCP 0



.

.



IBM TotalStorage™

Configure I/O Ports - setioport#----------------------------------------------------------------------------------------------------------

# DS8300 Primary setup script

#

# Config

#----------------------------------------------------------------------------------------------------------

#

# SETUP the channel connections FCP

#

#----------------------------------------------------------------------------------------------------------

# IO Port definitions

# Command setioport

# list IOports lsioport

#

# Format of IOport id IEEAP

# I - identifier as IOPORT fixed 'I'

# EE - IO port Enclosure Number (00-17)

# A - Adapter Number (0-7)

# P -Port number (0-3)

#

#----------------------------------------------------------------------------------------------------------

#

setioport -dev IBM.2107-75AZRY1 -topology scsi-fcp I0000 I0001 I0002 I0003 I0030 I0031 I0032 I0033




setioport -dev IBM.2107-75AZRY1 -topology scsi-fcp I0430 I0431 I0432 I0433





IBM TotalStorage™

Logical Configuration

1 Create arrays

2 Create ranks

3 Create extent pool

4 Create logical volumes

5 Create volume groups

6 Create hosts & host groups


IBM TotalStorage™

Create Arrays

RAID5 or RAID10

Formed from Array Site of 8 DDMs

Formed across the FC fabricSimilar to Arrays across loops in an ESS

Explicitly select Array Site or let DS assign automatically

RecommendationForm 1 Array per Array SiteAllows better manipulation of FB volume location


IBM TotalStorage™

Create Arrays (1)- lsarraysite

dscli> lsarraysite


arsite DA Pair dkcap (10^9B) State Array

===========================================

S1 2 146.0 Assigned A0















.

.

.



IBM TotalStorage™

Create Arrays (2)- mkarray

mkarray -raidtype 5 -arsite S1










.

.



IBM TotalStorage™

Create Arrays (3)- lsarray

dscli> lsarray

Date/Time: 14 February 2006 15:37:51 GMT IBM DSCLI Version: 5.1.0.2

Array State Data RAIDtype arsite Rank DA Pair DDMcap (10^9B)

==================================================================

A0 Assigned Normal 5 (6+P+S) S1 R0 2 146.0




A4 Assigned Normal 5 (7+P) S5 R4 2 146.0













.

.

.



IBM TotalStorage™

Create Ranks

Format CKD or FB

Ranks become associated with a server when assigned to an Extent Pool

Important to balance Rank assignments between servers

RecommendationAssign a single Rank to a single Extent Pool


IBM TotalStorage™

Create Ranks (1) - mkrank

mkrank -Array A0 -stgtype fb

















.

.


Rank can be assinged to Extent Pool with mkrankmkrank -Array A0 -stgtype fb -extpool P0Extent Pool must exist


IBM TotalStorage™

Create Ranks (2) - lsrank

dscli> lsrank


ID Group State datastate Array RAIDtype extpoolID stgtype

===========================================================

R0 0 Normal Normal A0 5 P0 fb

















.

.



IBM TotalStorage™

Create Extent Pools

Each Extent Pool is either CKD or FB

RAID types not intermixed in same Extent Pool

Extent Pools associated with a server

Divided into equal sized extentsFB - 1 GB (binary)

Minimum of 2 Extent Pools

Maximum equivalent to number of Ranks

RecommendationsOnly put Ranks of same speed and capacity in same Extent PoolFor performance create Extent Pools of a single rank only


IBM TotalStorage™

Create Extent Pools (1) - mkextpool

mkextpool -rankgrp 0 -stgtype fb FB_0


















.

.


Use Extent pool names that define the type


IBM TotalStorage™

Create Extent Pools (2) - chrank

chrank -extpool P0 R0


















IBM TotalStorage™

Create Extent Pools (3) - lsextpool

dscli> lsextpool


Name ID stgtype rankgrp status availstor (2^30B) %allocated available reserved numvols

========================================================================================

FB_0 P0 fb 0 below 303 61 303 0 14

FB_1 P1 fb 1 below 439 43 439 0 10

FB_2 P2 fb 0 below 167 78 167 0 12

FB_3 P3 fb 1 below 133 82 133 0 11

FB_4 P4 fb 0 below 229 74 229 0 12

FB_5 P5 fb 1 below 161 82 161 0 12

FB_6 P6 fb 0 below 297 67 297 0 10

FB_7 P7 fb 1 below 59 93 59 0 14

FB_8 P8 fb 0 below 439 43 439 0 10

FB_9 P9 fb 1 below 303 61 303 0 14

FB_10 P10 fb 0 below 133 82 133 0 11

FB_11 P11 fb 1 below 99 87 99 0 12

FB_12 P12 fb 0 below 93 89 93 0 13

FB_13 P13 fb 1 below 229 74 229 0 12

FB_14 P14 fb 0 below 263 71 263 0 11

FB_15 P15 fb 1 below 263 71 263 0 11

FB_16 P16 fb 0 below 303 61 303 0 14

.


IBM TotalStorage™

Address Groups

Addresses are grouped into Address Groups of 4096 addresses

Each Address Group must be CKD or FB

No real impact to FB implementations

dscli> lsaddressgrp


ID stgtype basevolnum Vols LSSs confgvols

=========================================

0 fb 0000 4096 16 189

1 fb 1000 4096 16 189

2 fb 2000 4096 16 186


IBM TotalStorage™

Logical Subsystems - LSSNo fixed association with device adapters or Ranks

Server affinity onlyEven numbered LSS associated with server 0Odd numbered LSS associated with server 1

FB LSS creation is implicitLSS selection during volume creation results in LSS creation

Volume addressingFormat is hexadecimal XYZZX = Address group (0-F)XY = LSS Number (00-FE)ZZ = Volume Number (00-FF)

RecommendationsCreate a single LSS for each Extent PoolDefine this during volume creation


IBM TotalStorage™

Create FB Volumes

Logical Volumes formed of increments of extentsMinimum 1 extentMaximum FB is 2TB

Volumes are defined in Decimal sizesEach 1GB FB Binary Extent is 1.073741824 GB Decimal

Ensure OS support exists for larger LUNs

Maximum number of volumes is 65280


IBM TotalStorage™

Create FB Volumes (1) - mkfbvol#

# Define the FB Volumes

#

# 7+p Large Ranks we want 36.5GB (decimal) vols = 34 extents

# 73GB (decimal) vols = 68 extents

# ---------------------------------

# 6+p+s small Ranks we want 36.5GB (decimal) vols = 34 extents

# 73GB (decimal) vols = 68 extents

#

#

#LSS 00

mkfbvol -dev IBM.2107-75AZRY1 -extpool P0 -cap 34 -name P0_#h 0000-000D

#

#LSS 01

mkfbvol -dev IBM.2107-75AZRY1 -extpool P1 -cap 34 -name P1_#h 0100-0107

#

#LSS 02




IBM TotalStorage™

Create FB Volumes (2) - lsfbvol

dscli> lsfbvol -lss 00


Name ID accstate datastate configstate deviceMTM datatype extpool cap (2^30B) cap (10^9B) cap (blocks)

===========================================================================================================

P0_0000 0000 Online Normal Normal 2107-900 FB 512 P0 34.0 - 71303168










P0_000A 000A Online Normal Normal 2107-900 FB 512 P0 34.0 - 71303168

P0_000B 000B Online Normal Normal 2107-900 FB 512 P0 34.0 - 71303168

P0_000C 000C Online Normal Normal 2107-900 FB 512 P0 34.0 - 71303168

P0_000D 000D Online Normal Normal 2107-900 FB 512 P0 34.0 - 71303168


IBM TotalStorage™

Create Volume GroupsProvides LUN masking feature

Volumes can be assigned to one or more volume groups

Up to 8320 Volume Groups

Any Host Port (or Host Group) can only access one Volume Group

Volume Groups can be accessed by multiple hostsProviding they use same blocksize and addressing modeData integrity reposnsibility of the host OS/software

Clustering supportVolumes can be asssigned to multiple Volume GroupsMutliple hosts can be in the same Volume Group

RecommendationsCreate one Volume Group per Hoat GroupName Volume Groups with host names (up to 16 chars)


IBM TotalStorage™

Create Volume Groups (1) - mkvolgrp

mkvolgrp fprdb01_vg

mkvolgrp fprdb02_vg

mkvolgrp fprdb03_vg

mkvolgrp fprdb04_vg

mkvolgrp fprdb05_vg

mkvolgrp fprdb06_vg

mkvolgrp fprdb07_vg

mkvolgrp fprdb08_vg

mkvolgrp fprdb09_vg

mkvolgrp fprdb10_vg

mkvolgrp fprdb11_vg

mkvolgrp fpapp01_vg

mkvolgrp fpapp02_vg

mkvolgrp fpapp03_vg

mkvolgrp fpapp04_vg

mkvolgrp fsrdb01_vg

mkvolgrp fsrdb02_vg

mkvolgrp fsrdb03_vg

mkvolgrp fsrdb04_vg

mkvolgrp futsm01_vg


IBM TotalStorage™

Create Volume Groups (2) - lsvolgrpdscli> lsvolgrp


Name ID Type

=======================================

fprdb01_vg V0 SCSI Mask










All CKD V10 FICON/ESCON All


fpapp01_vg V12 SCSI Mask




fsrdb01_vg V16 SCSI Mask




All Fixed Block-512 V20 SCSI All

futsm01_vg V21 SCSI Mask

All Fixed Block-520 V30 OS400 All


IBM TotalStorage™

Create Hosts and Host Groups

Host attacments assigned as Host Connections

Individual host ports can be grouped into Host Groups

Host Connection created by assigning FC HBA WWN

Profiles used to identify host type

RecommendationsGroup all host connections from a single host into a Host Group


IBM TotalStorage™

Create Hosts and Host Groups (1) - mkhostconnect

# fprdb06

mkhostconnect -dev IBM.2107-75AZRY1 -wwname 10000000C94F16F4 -profile "IBM pSeries - AIX" -portgrp 006 -volgrp V5 fprdb06_p1

mkhostconnect -dev IBM.2107-75AZRY1 -wwname 10000000C94F1663 -profile "IBM pSeries - AIX" -portgrp 006 -volgrp V5 fprdb06_p2



#

#fprdb07






IBM TotalStorage™

Create Hosts and Host Groups (2) - lshostconnect

dscli> lshostconnect


Name ID WWPN HostType Profile portgrp volgrpID ESSIOport

============================================================================

fprdb06_p1 0000 10000000C94F16F4 - IBM pSeries - AIX 5 V5 all

fprdb06_p2 0001 10000000C94F1663 - IBM pSeries - AIX 5 V5 all








IBM TotalStorage™

Assign FB Volumes to Volume Groups

#V5 AADb Staging

chvolgrp -dev IBM.2107-75AZRY1 -action add -volume 0000,0900,0A00,0B00,1200,1300,1A00,1B00,2200,2300,2A00,2B00 V5

#

#V6 AADb Partition

chvolgrp -dev IBM.2107-75AZRY1 -action add -volume 0200,0300,0500,0600,0C00,0D00,1400,1500,1C00,1D00,2400,2500,2C00,2D00 V6

Easier to manage volume assingmentsEnsures even spread of volumes over multiple extent pools


IBM TotalStorage™

Copy services in DS8000


IBM TotalStorage™

Copy Service• Duplication of Data

• You can create back up data with little or no disruption to your application

• Allow you to create immediate copy of production data• Mission Critical Application

• Supports• Open Systems• Z Series

• Advantage • Disaster Recovery

– Faster restore• Data Migration• Data Duplication


IBM TotalStorage™

Types of Copy services• Copy Functions

– FlashCopy ( Point-in-time Copy )– Global Copy

• Mirror Functions– Metro Mirror– Global Mirror


IBM TotalStorage™

Multiple Relationship FlashCopy

• Can have multiple Target volumes

( Maximum 12)

• Only one volume can act as source volume for any target volume

• Single volume can never act as Source & Target


IBM TotalStorage™

Thank You

Documents

DS8000 - Training