DS 8000 Monitoring Applications

Front cover

DS8000 Performance Monitoring and TuningUnderstand the performance aspects of the DS8000 architecture Configure the DS8000 to fully exploit its capabilities Use planning and monitoring tools with the DS8000

Bert Dufrasne Brett Allison John Barnes Jean Iyabi Rajesh Jeyapaul Peter Kimmel

Chuck Laing Anderson Nobre Rene Oehme Gero Schmidt Paulus Usong

ibm.com/redbooks

International Technical Support Organization DS8000 Performance Monitoring and Tuning March 2009

SG24-7146-01

Note: Before using this information and the product it supports, read the information in Notices on page xiii.

Second Edition (March 2009) This edition applies to the IBM System Storage DS8000 with Licensed Machine Code 5.4.1.xx.xx (Code bundles 64.1.x.x).

Copyright International Business Machines Corporation 2009. All rights reserved. Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.

ContentsNotices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiv Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii The team that wrote this IBM Redbooks publication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii Special thanks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix Become a published author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xx Comments welcome. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xx Chapter 1. DS8000 characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1 The storage server challenge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1.1 Performance numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1.2 Recommendations and rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1.3 Modeling your workload . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1.4 Allocating hardware components to workloads. . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Meeting the challenge: DS8000 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2.1 DS8000 models and characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 DS8000 performance characteristics overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3.1 Advanced caching techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3.2 IBM System Storage multipath Subsystem Device Driver (SDD) . . . . . . . . . . . . . . 1.3.3 Performance characteristics for System z. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 2 2 2 3 3 3 4 5 6 6 6

Chapter 2. Hardware configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.1 Storage system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.2 Processor memory and cache . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 2.2.1 Cache and I/O operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 2.2.2 Determining the right amount of cache storage . . . . . . . . . . . . . . . . . . . . . . . . . . 16 2.3 RIO-G interconnect and I/O enclosures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 2.3.1 RIO-G loop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 2.3.2 I/O enclosures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 2.4 Disk subsystem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.4.1 Device adapters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.4.2 Fibre Channel disk architecture in the DS8000. . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.4.3 Disk enclosures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 2.4.4 Fibre Channel drives compared to FATA and SATA drives . . . . . . . . . . . . . . . . . 20 2.4.5 Arrays across loops . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 2.4.6 Order of installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 2.4.7 Performance Accelerator feature (Feature Code 1980) . . . . . . . . . . . . . . . . . . . . 23 2.5 Host adapters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 2.5.1 Fibre Channel and FICON host adapters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 2.5.2 ESCON host adapters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 2.5.3 Multiple paths to Open Systems servers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 2.5.4 Multiple paths to System z servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 2.5.5 Spreading host attachments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 2.6 Tools to aid in hardware planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 2.6.1 White papers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 2.6.2 Disk Magic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 2.6.3 Capacity Magic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

Copyright IBM Corp. 2009. All rights reserved.

iii

Chapter 3. Understanding your workload . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1 General workload types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.1 Standard workload . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.2 Read intensive cache unfriendly workload . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.3 Sequential workload . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.4 Batch jobs workload . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.5 Sort jobs workload . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Database workload . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.1 DB2 query workload . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.2 DB2 logging workload . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.3 DB2 transaction environment workload . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.4 DB2 utilities workload . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Application workload . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.1 General file serving . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.2 Online transaction processing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.3 Data mining . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.4 Video on demand . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.5 Data warehousing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.6 Engineering and scientific applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.7 Digital video editing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4 Profiling workloads in the design phase . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5 Understanding your workload type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5.1 Monitoring the DS8000 workload . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5.2 Monitoring the host workload . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter 4. Logical configuration concepts and terminology . . . . . . . . . . . . . . . . . . . . 4.1 RAID levels and spares. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1.1 RAID 5 overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1.2 RAID 6 overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1.3 RAID 10 overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1.4 Spare creation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 The abstraction layers for logical configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.1 Array sites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.2 Arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.3 Ranks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.4 Extent pools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.5 Logical volumes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.6 Space Efficient volumes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.7 Allocation, deletion, and modification of LUNs and CKD volumes . . . . . . . . . . . . 4.2.8 Logical subsystems (LSS). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.9 Address groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.10 Volume access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.11 Summary of the logical configuration hierarchy . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Understanding the array to LUN relationship . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.1 How extents are formed together to make DS8000 LUNs . . . . . . . . . . . . . . . . . . 4.3.2 Understanding data I/O placement on ranks and extent pools . . . . . . . . . . . . . . . Chapter 5. Logical configuration performance considerations . . . . . . . . . . . . . . . . . . 5.1 Basic configuration principles for optimal performance. . . . . . . . . . . . . . . . . . . . . . . . . 5.1.1 Workload isolation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1.2 Workload resource-sharing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1.3 Workload spreading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1.4 Using workload isolation, resource-sharing, and spreading . . . . . . . . . . . . . . . . . 5.2 Analyzing application workload characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ivDS8000 Performance Monitoring and Tuning

29 30 30 30 30 30 30 31 32 32 32 32 32 33 34 34 34 34 35 35 35 38 38 38 41 42 42 43 43 44 45 45 45 46 47 48 49 50 53 53 54 54 55 56 61 63 64 64 65 66 67 68

5.2.1 Determining isolation requirements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 5.2.2 Reviewing remaining workloads for feasibility of resource-sharing. . . . . . . . . . . . 70 5.3 Planning allocation of disk and host connection capacity . . . . . . . . . . . . . . . . . . . . . . . 70 5.3.1 Planning DS8000 hardware resources for isolated workloads . . . . . . . . . . . . . . . 70 5.3.2 Planning DS8000 hardware resources for resource-sharing workloads . . . . . . . . 70 5.4 Planning volume and host connection spreading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 5.4.1 Spreading volumes for isolated and resource-sharing workloads. . . . . . . . . . . . . 71 5.4.2 Spreading host connections for isolated and resource-sharing workloads . . . . . . 72 5.5 Planning array sites. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 5.5.1 DS8000 configuration example 1: Array site planning considerations . . . . . . . . . 73 5.5.2 DS8000 configuration example 2: Array site planning considerations . . . . . . . . . 75 5.5.3 DS8000 configuration example 3: Array site planning considerations . . . . . . . . . 77 5.6 Planning RAID arrays and ranks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 5.6.1 RAID-level performance considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 5.6.2 RAID array considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 5.6.3 Rank considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 5.7 Planning extent pools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 5.7.1 Single-rank and multi-rank extent pools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 5.7.2 Extent allocation methods for multi-rank extent pools. . . . . . . . . . . . . . . . . . . . . . 96 5.7.3 Balancing workload across available resources . . . . . . . . . . . . . . . . . . . . . . . . . 103 5.7.4 Assigning workloads to extent pools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 5.7.5 Planning for multi-rank extent pools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 5.7.6 Planning for single-rank extent pools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 5.8 Plan address groups, LSSs, volume IDs, and CKD PAVs . . . . . . . . . . . . . . . . . . . . . 118 5.8.1 Volume configuration scheme using application-related LSS/LCU IDs . . . . . . . 120 5.8.2 Volume configuration scheme using hardware-bound LSS/LCU IDs . . . . . . . . . 124 5.9 Plan I/O port IDs, host attachments, and volume groups . . . . . . . . . . . . . . . . . . . . . . 131 5.9.1 DS8000 configuration example 1: I/O port planning considerations . . . . . . . . . . 133 5.9.2 DS8000 configuration example 2: I/O port planning considerations . . . . . . . . . . 136 5.9.3 DS8000 configuration example 3: I/O port planning considerations . . . . . . . . . . 140 5.10 Implement and document DS8000 logical configuration . . . . . . . . . . . . . . . . . . . . . . 143 Chapter 6. Performance management process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2 Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3 Operational performance subprocess . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3.1 Inputs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3.2 Outputs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3.3 Tasks, actors, and roles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3.4 Performance troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4 Tactical performance subprocess . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4.1 Inputs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4.2 Outputs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4.3 Tasks, actors, and roles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.5 Strategic performance subprocess . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.5.1 Inputs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.5.2 Outputs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.5.3 Tasks, actors, and roles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter 7. Performance planning tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1 Disk Magic. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1.1 The need for performance planning and modeling tools. . . . . . . . . . . . . . . . . . . 7.1.2 Overview and characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 148 149 151 152 153 153 154 155 156 156 157 158 158 158 159 161 162 162 163

Contents

v

7.1.3 Output information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1.4 Disk Magic modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2 Disk Magic for System z (zSeries) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2.1 Process the DMC file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2.2 zSeries model to merge the two ESS-800s to a DS8300 . . . . . . . . . . . . . . . . . . 7.2.3 Disk Magic performance projection for zSeries model . . . . . . . . . . . . . . . . . . . . 7.2.4 Workload growth projection for zSeries model . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3 Disk Magic for Open Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3.1 Process the TotalStorage Productivity Center csv output file . . . . . . . . . . . . . . . 7.3.2 Open Systems model to merge the two ESS-800s to a DS8300 . . . . . . . . . . . . 7.3.3 Disk Magic performance projection for an Open Systems model . . . . . . . . . . . . 7.3.4 Workload growth projection for an Open Systems model . . . . . . . . . . . . . . . . . . 7.4 Workload growth projection. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.5 Input data needed for Disk Magic study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.5.1 z/OS environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.5.2 Open Systems environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.6 Configuration guidelines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter 8. Practical performance management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.1 Introduction to practical performance management . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2 Performance management tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2.1 TotalStorage Productivity Center overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2.2 TotalStorage Productivity Center data collection . . . . . . . . . . . . . . . . . . . . . . . . 8.2.3 TotalStorage Productivity Center measurement of DS8000 components. . . . . . 8.2.4 General TotalStorage Productivity Center measurement considerations . . . . . . 8.3 TotalStorage Productivity Center data collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.3.1 Timestamps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.3.2 Duration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.3.3 Intervals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.4 Key performance metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.4.1 DS8000 key performance indicator thresholds . . . . . . . . . . . . . . . . . . . . . . . . . . 8.5 TotalStorage Productivity Center reporting options . . . . . . . . . . . . . . . . . . . . . . . . . . 8.5.1 Alerts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.5.2 Predefined performance reports in TotalStorage Productivity Center. . . . . . . . . 8.5.3 Ad hoc reports. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.5.4 Batch reports. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.5.5 TPCTOOL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.5.6 Volume Planner . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.5.7 TPC Reporter for Disk. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.6 Monitoring performance of a SAN switch or director. . . . . . . . . . . . . . . . . . . . . . . . . . 8.6.1 SAN configuration examples. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.6.2 TotalStorage Productivity Center for Fabric alerts . . . . . . . . . . . . . . . . . . . . . . . 8.6.3 TotalStorage Productivity Center for Fabric reporting. . . . . . . . . . . . . . . . . . . . . 8.6.4 TotalStorage Productivity Center for Fabric metrics . . . . . . . . . . . . . . . . . . . . . . 8.7 End-to-end analysis of I/O performance problems . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.7.1 Performance analysis examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.8 TotalStorage Productivity Center for Disk in mixed environment . . . . . . . . . . . . . . . . Chapter 9. Host attachment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.1 DS8000 host attachment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2 Attaching Open Systems hosts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2.1 Fibre Channel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2.2 SAN implementations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

163 163 165 165 170 177 179 180 181 188 194 195 197 197 198 198 199 203 204 204 205 205 207 212 214 214 216 217 218 221 222 223 228 229 232 236 239 240 242 243 246 247 248 249 257 263 265 266 266 267 267

vi

DS8000 Performance Monitoring and Tuning

9.2.3 Multipathing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.3 Attaching IBM System z and S/390 hosts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.3.1 ESCON . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.3.2 FICON . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.3.3 FICON configuration and performance considerations . . . . . . . . . . . . . . . . . . . . 9.3.4 z/VM, z/VSE, and Linux on System z attachment. . . . . . . . . . . . . . . . . . . . . . . . Chapter 10. Performance considerations with Windows Servers . . . . . . . . . . . . . . . 10.1 General Windows performance tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.2 I/O architecture overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.3 Windows Server 2008 I/O Manager enhancements . . . . . . . . . . . . . . . . . . . . . . . . . 10.4 Filesystem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.4.1 Windows filesystem overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.4.2 NTFS guidelines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.5 Volume management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.5.1 Microsoft Logical Disk Manager (LDM) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.5.2 Microsoft LDM software RAID. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.5.3 Veritas Volume Manager (VxVM) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.5.4 Determining volume layout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.6 Multipathing and the port layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.6.1 SCSIport scalability issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.6.2 Storport scalability features. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.6.3 Subsystem Device Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.6.4 Subsystem Device Driver Device Specific Module . . . . . . . . . . . . . . . . . . . . . . 10.6.5 Veritas Dynamic MultiPathing (DMP) for Windows . . . . . . . . . . . . . . . . . . . . . . 10.7 Host bus adapter (HBA) settings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.8 I/O performance measurement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.8.1 Key I/O performance metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.8.2 Windows Performance console (perfmon) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.8.3 Performance log configuration and data export . . . . . . . . . . . . . . . . . . . . . . . . 10.8.4 Collecting configuration data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.8.5 Correlating performance and configuration data. . . . . . . . . . . . . . . . . . . . . . . . 10.8.6 Analyzing performance data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.8.7 Windows Server Performance Analyzer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.9 Task Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.9.1 Starting Task Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.10 I/O load testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.10.1 Types of tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.10.2 Iometer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter 11. Performance considerations with UNIX servers . . . . . . . . . . . . . . . . . . . 11.1 Planning and preparing UNIX servers for performance . . . . . . . . . . . . . . . . . . . . . . 11.1.1 UNIX disk I/O architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 AIX disk I/O components. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2.1 AIX Journaled File System (JFS) and Journaled File System 2 (JFS2) . . . . . . 11.2.2 Veritas File System (VxFS) for AIX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2.3 General Parallel FileSystem (GPFS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2.4 IBM Logical Volume Manager (LVM) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2.5 Veritas Volume Manager (VxVM) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2.6 IBM Subsystem Device Driver (SDD) for AIX . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2.7 MPIO with SDDPCM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2.8 Veritas Dynamic MultiPathing (DMP) for AIX . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2.9 FC adapters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

270 275 275 276 279 279 281 282 283 283 284 284 284 285 285 286 286 287 288 289 289 290 290 291 291 291 292 294 296 296 297 297 300 301 301 304 304 305 307 308 309 311 312 315 315 316 320 321 321 322 322

Contents

vii

11.2.10 Virtual I/O Server (VIOS). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.3 AIX performance monitoring tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.3.1 AIX vmstat. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.3.2 pstat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.3.3 AIX iostat. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.3.4 lvmstat. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.3.5 topas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.3.6 nmon . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.3.7 fcstat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.3.8 filemon . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.4 Solaris disk I/O components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.4.1 UFS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.4.2 Veritas FileSystem (VxFS) for Solaris. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.4.3 SUN Solaris ZFS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.4.4 Solaris Volume Manager (formerly Solstice DiskSuite). . . . . . . . . . . . . . . . . . . 11.4.5 Veritas Volume Manager (VxVM) for Solaris . . . . . . . . . . . . . . . . . . . . . . . . . . 11.4.6 IBM Subsystem Device Driver for Solaris . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.4.7 MPxIO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.4.8 Veritas Dynamic MultiPathing (DMP) for Solaris. . . . . . . . . . . . . . . . . . . . . . . . 11.4.9 Array Support Library (ASL) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.4.10 FC adapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.5 Solaris performance monitoring tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.5.1 fcachestat and directiostat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.5.2 Solaris vmstat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.5.3 Solaris iostat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.5.4 vxstat. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.5.5 dtrace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.6 HP-UX Disk I/O architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.6.1 HP-UX High Performance File System (HFS). . . . . . . . . . . . . . . . . . . . . . . . . . 11.6.2 HP-UX Journaled File System (JFS). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.6.3 HP Logical Volume Manager (LVM) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.6.4 Veritas Volume Manager (VxVM) for HP-UX . . . . . . . . . . . . . . . . . . . . . . . . . . 11.6.5 PV Links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.6.6 Native multipathing in HP-UX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.6.7 Subsystem Device Driver (SDD) for HP-UX . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.6.8 Veritas Dynamic MultiPathing (DMP) for HP-UX . . . . . . . . . . . . . . . . . . . . . . . 11.6.9 Array Support Library (ASL) for HP-UX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.6.10 FC adapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.7 HP-UX performance monitoring tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.7.1 HP-UX sar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.7.2 vxstat. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.7.3 GlancePlus and HP Perfview/Measureware . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.8 SDD commands for AIX, HP-UX, and Solaris . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.8.1 HP-UX SDD commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.8.2 Sun Solaris SDD commands. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.9 Testing and verifying DS8000 Storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.9.1 Using the dd command to test sequential rank reads and writes . . . . . . . . . . . 11.9.2 Verifying your system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter 12. Performance considerations with VMware . . . . . . . . . . . . . . . . . . . . . . . 12.1 Disk I/O architecture overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.2 Multipathing considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.3 Performance monitoring tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

323 325 326 328 329 334 335 336 339 339 343 343 344 345 346 347 348 348 349 349 350 350 350 351 352 353 354 356 356 357 357 362 362 362 363 363 363 363 363 363 366 366 366 371 373 375 376 377 383 384 386 389

viii


12.3.1 Virtual Center Performance Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.3.2 Performance monitoring with esxtop. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.3.3 Guest-based performance monitoring. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.4 VMware specific tuning for maximum performance . . . . . . . . . . . . . . . . . . . . . . . . . 12.4.1 Workload spreading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.4.2 Virtual Machines sharing the same LUN. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.4.3 ESX filesystem considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.4.4 Aligning partitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.5 Tuning of Virtual Machines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter 13. Performance considerations with Linux. . . . . . . . . . . . . . . . . . . . . . . . . . 13.1 Supported platforms and distributions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.2 Linux disk I/O architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.2.1 I/O subsystem architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.2.2 Cache and locality of reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.2.3 Block layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.2.4 I/O device driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.3 Specific configuration for storage performance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.3.1 Host bus adapter for Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.3.2 Multipathing in Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.3.3 Software RAID functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.3.4 Logical Volume Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.3.5 Tuning the disk I/O scheduler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.3.6 Filesystems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.4 Linux performance monitoring tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.4.1 Disk I/O performance indicators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.4.2 Finding disk bottlenecks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter 14. IBM System Storage SAN Volume Controller attachment . . . . . . . . . . . 14.1 IBM System Storage SAN Volume Controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.1.1 SAN Volume Controller concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.1.2 SAN Volume Controller multipathing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.1.3 SVC Advanced Copy Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.2 SAN Volume Controller performance considerations . . . . . . . . . . . . . . . . . . . . . . . . 14.3 DS8000 performance considerations with SVC . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.3.1 DS8000 array . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.3.2 DS8000 rank format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.3.3 DS8000 extent pool implications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.3.4 DS8000 volume considerations with SVC. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.3.5 Volume assignment to SAN Volume Controller . . . . . . . . . . . . . . . . . . . . . . . . 14.3.6 Managed Disk Group for DS8000 Volumes . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.4 Performance monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.4.1 Using TotalStorage Productivity Center for Disk to monitor the SVC . . . . . . . . 14.5 Sharing the DS8000 between a server and the SVC . . . . . . . . . . . . . . . . . . . . . . . . 14.5.1 Sharing the DS8000 between Open Systems servers and the SVC . . . . . . . . 14.5.2 Sharing the DS8000 between System i server and the SVC . . . . . . . . . . . . . . 14.5.3 Sharing the DS8000 between System z server and the SVC . . . . . . . . . . . . . . 14.6 Advanced functions for the DS8000 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.6.1 Cache-disabled VDisks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.7 Configuration guidelines for optimizing performance . . . . . . . . . . . . . . . . . . . . . . . .

389 390 391 392 392 392 393 396 399 401 402 402 403 404 405 405 406 406 407 409 410 412 414 417 417 418 421 422 422 425 426 427 429 429 429 430 434 434 435 436 436 437 437 438 438 438 438 439

Chapter 15. System z servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 441 15.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 442 15.2 Parallel Access Volumes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 442Contents

ix

15.2.1 Static PAV, Dynamic PAV, and HyperPAV . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.2.2 HyperPAV compared to dynamic PAV test . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.2.3 PAV and large volumes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.3 Multiple Allegiance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.4 How PAV and Multiple Allegiance work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.4.1 Concurrent read operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.4.2 Concurrent write operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.5 I/O Priority Queuing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.6 Logical volume sizes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.6.1 Selecting the volume size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.6.2 Larger volume compared to smaller volume performance . . . . . . . . . . . . . . . . 15.6.3 Planning the volume sizes of your configuration. . . . . . . . . . . . . . . . . . . . . . . . 15.7 FICON . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.7.1 Extended Distance FICON . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.7.2 High Performance FICON . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.7.3 MIDAW . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.8 z/OS planning and configuration guidelines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.8.1 Channel configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.8.2 Extent pool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.8.3 Considerations for mixed workloads . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.9 DS8000 performance monitoring tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.10 RMF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.10.1 I/O response time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.10.2 I/O response time components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.10.3 IOP/SAP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.10.4 FICON host channel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.10.5 FICON director . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.10.6 Processor complex . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.10.7 Cache and NVS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.10.8 DS8000 FICON/Fibre port and host adapter. . . . . . . . . . . . . . . . . . . . . . . . . . 15.10.9 Extent pool and rank . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.11 RMF Magic for Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.11.1 RMF Magic analysis process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.11.2 Data collection step. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.11.3 RMF Magic reduce step . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.11.4 RMF Magic analyze step. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.11.5 Data presentation and reporting step . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.11.6 Hints and tips . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter 16. Databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16.1 DB2 in a z/OS environment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16.1.1 Understanding your database workload . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16.1.2 DB2 overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16.1.3 DB2 storage objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16.1.4 DB2 dataset types. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16.2 DS8000 considerations for DB2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16.3 DB2 with DS8000 performance recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . 16.3.1 Know where your data resides . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16.3.2 Balance workload across DS8000 resources . . . . . . . . . . . . . . . . . . . . . . . . . . 16.3.3 Take advantage of VSAM data striping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16.3.4 Large volumes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16.3.5 Modified Indirect Data Address Words (MIDAWs) . . . . . . . . . . . . . . . . . . . . . . 16.3.6 Adaptive Multi-stream Prefetching (AMP) . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

442 443 445 446 446 447 448 449 449 450 451 453 454 455 455 457 459 459 461 463 464 464 464 466 468 468 469 470 470 472 474 476 477 478 479 479 479 482 485 486 486 487 487 488 489 489 489 490 490 490 491 491

x


16.3.7 DB2 burst write . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16.3.8 Monitoring DS8000 performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16.4 DS8000 DB2 UDB in an Open Systems environment . . . . . . . . . . . . . . . . . . . . . . . 16.4.1 DB2 UDB storage concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16.5 DB2 UDB with DS8000 performance recommendations. . . . . . . . . . . . . . . . . . . . . . 16.5.1 Know where your data resides . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16.5.2 Balance workload across DS8000 resources . . . . . . . . . . . . . . . . . . . . . . . . . . 16.5.3 Use DB2 to stripe across containers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16.5.4 Selecting DB2 logical sizes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16.5.5 Selecting the DS8000 logical disk sizes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16.5.6 Multipathing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16.6 IMS in a z/OS environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16.6.1 IMS overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16.6.2 IMS logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16.7 DS8000 considerations for IMS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16.8 IMS with DS8000 performance recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . 16.8.1 Know where your data resides . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16.8.2 Balance workload across DS8000 resources . . . . . . . . . . . . . . . . . . . . . . . . . . 16.8.3 Large volumes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16.8.4 Monitoring DS8000 performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter 17. Copy Services performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17.1 Copy Services introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17.2 FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17.2.1 FlashCopy performance considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17.2.2 Performance planning for IBM FlashCopy SE . . . . . . . . . . . . . . . . . . . . . . . . . 17.3 Metro Mirror. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17.3.1 Metro Mirror configuration considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17.3.2 Metro Mirror performance considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17.3.3 Scalability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17.4 Global Copy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17.4.1 Global Copy configuration considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17.4.2 Global Copy performance consideration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17.4.3 Scalability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17.5 Global Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17.5.1 Global Mirror performance considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17.5.2 Global Mirror Session parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17.5.3 Avoid unbalanced configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17.5.4 Growth within Global Mirror configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17.6 z/OS Global Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17.6.1 z/OS Global Mirror control dataset placement . . . . . . . . . . . . . . . . . . . . . . . . . 17.6.2 z/OS Global Mirror tuning parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17.6.3 z/OS Global Mirror enhanced multiple reader. . . . . . . . . . . . . . . . . . . . . . . . . . 17.6.4 zGM enhanced multiple reader performance improvement . . . . . . . . . . . . . . . 17.6.5 XRC Performance Monitor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17.7 Metro/Global Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17.7.1 Metro/Global Mirror performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17.7.2 z/OS Metro/Global Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17.7.3 z/OS Metro/Global Mirror performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

491 491 492 492 497 497 498 499 499 500 502 502 502 503 504 504 504 505 505 506 507 508 509 511 516 518 519 524 526 526 527 529 530 530 533 535 538 541 543 545 545 549 550 552 552 553 553 554

Appendix A. Logical configuration examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 555 A.1 Considering hardware resource availability for throughput. . . . . . . . . . . . . . . . . . . . . 556 A.2 Resource isolation or sharing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 556

Contents

xi

Scenario 1: Spreading everything with no isolation . . . . . . . . . . . . . . . . . . . . . . . . . . . Scenario 2: Spreading data I/O with partial isolation . . . . . . . . . . . . . . . . . . . . . . . . . . Scenario 3: Grouping unlike RAID types together in the extent pool . . . . . . . . . . . . . . Scenario 4: Grouping like RAID types in the extent pool . . . . . . . . . . . . . . . . . . . . . . . Scenario 5: More isolation of RAID types in the extent pool . . . . . . . . . . . . . . . . . . . . . Scenario 6: Balancing mixed RAID type ranks and capacities . . . . . . . . . . . . . . . . . . . Appendix B. Windows server performance log collection . . . . . . . . . . . . . . . . . . . . . B.1 Windows Server 2003 log file configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring logging of disk metrics Windows Server 2003 . . . . . . . . . . . . . . . . . . . . . . Saving counter log settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Importing counter logs properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Analyzing disk performance from collected data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Retrieving data from a counter log file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Exporting logged data on Windows Server 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.2 Windows Server 2008 log file configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Windows Server 2008 Export . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Appendix C. UNIX shell scripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C.2 vgmap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C.3 lvmap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C.4 vpath_iostat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C.5 ds_iostat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C.6 test_disk_speeds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C.7 lsvscsimap.ksh . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C.8 mkvscsimap.ksh . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Appendix D. Post-processing scripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.2 Dependencies. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.2.1 Running the scripts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Appendix E. Benchmarking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E.1 Goals of benchmarking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E.2 Requirements for a benchmark . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Define the benchmark architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Define the benchmark workload . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Monitoring the performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Define the benchmark time frame . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E.3 Caution using benchmark results to design production . . . . . . . . . . . . . . . . . . . . . . . Related publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IBM Redbooks publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Other publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Online resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . How to get IBM Redbooks publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Help from IBM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

556 561 562 563 565 566 571 572 572 575 576 576 576 578 580 584 587 588 588 589 590 594 597 598 602 607 608 608 609 623 624 624 625 625 626 626 627 629 629 630 630 630 631

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 633

xii


NoticesThis information was developed for products and services offered in the U.S.A. IBM may not offer the products, services, or features discussed in this document in other countries. Consult your local IBM representative for information on the products and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any IBM intellectual property right may be used instead. However, it is the user's responsibility to evaluate and verify the operation of any non-IBM product, program, or service. IBM may have patents or pending patent applications covering subject matter described in this document. The furnishing of this document does not give you any license to these patents. You can send license inquiries, in writing, to: IBM Director of Licensing, IBM Corporation, North Castle Drive, Armonk, NY 10504-1785 U.S.A. The following paragraph does not apply to the United Kingdom or any other country where such provisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you. This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice. Any references in this information to non-IBM Web sites are provided for convenience only and do not in any manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the materials for this IBM product and use of those Web sites is at your own risk. IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you. Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. This information contains examples of data and reports used in daily business operations. To illustrate them as completely as possible, the examples include the names of individuals, companies, brands, and products. All of these names are fictitious and any similarity to the names and addresses used by an actual business enterprise is entirely coincidental. COPYRIGHT LICENSE: This information contains sample application programs in source language, which illustrate programming techniques on various operating platforms. You may copy, modify, and distribute these sample programs in any form without payment to IBM, for the purposes of developing, using, marketing or distributing application programs conforming to the application programming interface for the operating platform for which the sample programs are written. These examples have not been thoroughly tested under all conditions. IBM, therefore, cannot guarantee or imply reliability, serviceability, or function of these programs.


xiii

TrademarksIBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. These and other IBM trademarked terms are marked on their first occurrence in this information with the appropriate symbol ( or ), indicating US registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at http://www.ibm.com/legal/copytrade.shtml The following terms are trademarks of the International Business Machines Corporation in the United States, other countries, or both:AIX 5L AIX alphaWorks CICS DB2 Universal Database DB2 DS4000 DS6000 DS8000 ECKD Enterprise Storage Server ESCON eServer FICON FlashCopy GDPS Geographically Dispersed Parallel Sysplex GPFS HACMP i5/OS IBM iSeries Iterations OMEGAMON OS/390 Parallel Sysplex POWER5 POWER5+ POWER6 PowerHA PowerPC PowerVM POWER pSeries Rational Redbooks Redbooks (logo) RS/6000 S/390 Sysplex Timer System i System p5 System p System Storage System x System z10 System z9 System z Tivoli Enterprise Console Tivoli TotalStorage xSeries z/Architecture z/OS z/VM z/VSE z9 zSeries

The following terms are trademarks of other companies: Acrobat, and Portable Document Format (PDF) are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, other countries, or both. Disk Magic, IntelliMagic, and the IntelliMagic logo are trademarks of IntelliMagic BV in the United States, other countries, or both. ITIL is a registered trademark, and a registered community trademark of the Office of Government Commerce, and is registered in the U.S. Patent and Trademark Office. Novell, SUSE, the Novell logo, and the N logo are registered trademarks of Novell, Inc. in the United States and other countries. Oracle, JD Edwards, PeopleSoft, Siebel, and TopLink are registered trademarks of Oracle Corporation and/or its affiliates. QLogic, and the QLogic logo are registered trademarks of QLogic Corporation. SANblade is a registered trademark in the United States. SAP R/3, SAP, and SAP logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries. VMotion, VMware, the VMware "boxes" logo and design are registered trademarks or trademarks of VMware, Inc. in the United States and/or other jurisdictions. J2EE, Java, JNI, S24, Solaris, Solstice, Sun, ZFS, and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. Excel, Internet Explorer, Microsoft, MS-DOS, MS, PowerPoint, SQL Server, Visual Basic, Windows NT, xivDS8000 Performance Monitoring and Tuning

Windows Server, Windows, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. Intel, Intel logo, Intel Inside logo, and Intel Centrino logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States, other countries, or both. UNIX is a registered trademark of The Open Group in the United States and other countries. Linux is a trademark of Linus Torvalds in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of others.

Notices

xv

xvi


PrefaceThis IBM Redbooks publication provides guidance about how to configure, monitor, and manage your IBM System Storage DS8000 to achieve optimum performance. It describes the DS8000 performance features and characteristics and how they can be exploited with the various server platforms that attach to the DS8000. Then, in separate chapters, we detail specific performance recommendations and discussions that apply for each server environment, as well as for database and DS8000 Copy Services environments. We also outline the various tools available for monitoring and measuring I/O performance for different server environments, as well as describe how to monitor the performance of the entire DS8000 subsystem.

The team that wrote this IBM Redbooks publicationThis book was produced by a team of specialists from around the world working with the International Technical Support Organization, San Jose Center at the ESCC lab in Mainz, Germany. Bertrand Dufrasne is an IBM Certified Consulting I/T Specialist and Project Leader for System Storage disk products at the International Technical Support Organization, San Jose Center. He has worked at IBM in various I/T areas. Bertrand has written many IBM Redbooks publications and has also developed and taught technical workshops. Before joining the ITSO, he worked for IBM Global Services as an Application Architect in the retail, banking, telecommunication, and healthcare industries. He holds a Masters degree in Electrical Engineering from the Polytechnic Faculty of Mons (Belgium). Brett Allison has performed distributed systems performance-related work since 1997, including performance analysis of J2EE applications, UNIX/Windows NT systems, and SAN Storage technologies. He is currently the storage performance and capacity technical focal point for IBM Global Services Technology Delivery. He has designed and developed tools, processes, and service offerings to support storage performance and capacity. He has spoken at a number of conferences and is the author of several White Papers on performance. John Barnes is a Senior IT Specialist in IBM Global Services in the UK. John started his IBM career 30 years ago as a Large Systems hardware CE. After an assignment to the UK Hardware Support Centre, he moved to a career in Availability Management. John then joined the UK Storage and SAN Services Team in 2000, specializing in IBM TotalStorage Enterprise Storage Server (ESS) and SAN implementations. He now works in the UK STG Storage Services team, specializing in DS8000 and SAN implementations, including SAN Volume Controller (SVC) and Copy Services. Jean Iyabi is an active member of the IBM ESCC (European Storage Competence Center) in Mainz, Germany since 2001. As a Product Field Engineer, he acted as last level support for High End storage disk. Jean has extensive experience in DS8000 support and focuses on Host Attachment (System z), Extended Copy Services Functions, and Geographically Dispersed Parallel Sysplex (GDPS). He was assigned for two years as the EMEA field support interface with the DS8000 development and test teams in Tucson, AZ. He holds a degree in Electrical Engineering from the University of Applied Sciences of Wiesbaden (Germany).


xvii

Rajesh Jeyapaul is an AIX Development Support Specialist in IBM India. He has nine years of experience in AIX, specializing in investigating the performance impact of processes running in AIX. Currently, he is leading a technical team responsible for providing Development support to various AIX components. He holds a Masters Degree in Software Systems from the University of BITS, India, and an MBA from University of MKU, India. His areas of expertise include System p, AIX, and High-Availability Cluster Multi-Processing (HACMP). Peter Kimmel is an IT Specialist and the ATS team lead of the Enterprise Disk Performance team at the European Storage Competence Center in Mainz, Germany. He joined IBM Storage in 1999 and since then worked with SSA, VSS, the various ESS generations, and DS8000/DS6000. He has been involved in all Early Shipment Programs (ESPs), early installs for the Copy Services rollouts, and has co-authored several DS8000 IBM Redbooks publications so far. Peter holds a Diploma (MSc) degree in Physics from the University of Kaiserslautern. Chuck Laing is a Senior IT Architect and Master Certified IT Specialist with The Open Group. He is also an IBM Certified IT Specialist, specializing in IBM Enterprise Class and Midrange Disk Storage Systems/Configuration Management in the Americas ITD. He has co-authored eight previous IBM Redbooks publications about the IBM TotalStorage Enterprise Storage Server and the DS8000/6000. He holds a degree in Computer Science. He has worked at IBM for over ten years. Before joining IBM, Chuck was a hardware CE on UNIX systems for ten years and taught Computer Science at Midland College for six and a half years in Midland, Texas. Anderson Ferreira Nobre is a Certified IT Specialist and Certified Advanced Technical Expert - IBM System p5 in Strategic Outsourcing in Hortolndia (Brazil). He has 10 years of experience with UNIX (mainly with AIX). He was assigned to the UNIX team in 2005 to plan, manage, and support the UNIX, SAN, and Storage environments for IBM Outsourcing clients. Rene Oehme is an IBM Certified Specialist for High-End Disk Solutions, working for the Germany and CEMAAS Hardware Support Center in Mainz, Germany. Rene has more than six years of experience in IBM hardware support, including Storage Subsystems, SAN, and Tape Solutions, as well as System p and System z. Currently, he provides support for clients and service representatives with High End Disk Subsystems, such as the DS8000, DS6000, and ESS. His main focus is Open Systems attachment of High-End Disk Subsystems, including AIX, Windows, Linux, and VMware. He holds a degree in Information Technology from the University of Cooperative Education (BA) Stuttgart. Gero Schmidt is an IT Specialist in the IBM ATS technical sales support organization in Germany. He joined IBM in 2001 working at the European Storage Competence Center (ESCC) in Mainz, providing technical support for a broad range of IBM storage products (SSA, ESS, DS4000, DS6000, and DS8000) in Open Systems environments with a primary focus on storage subsystem performance. During his seven years of experience with IBM storage products, he participated in various beta test programs for ESS 800 and especially in the product rollout and beta test program for the DS6000/DS8000 series. He holds a degree in Physics (Dipl.-Phys.) from the Technical University of Braunschweig, Germany. Paulus Usong started his IBM career in Indonesia decades ago. He rejoined IBM at the Santa Teresa Lab (now known as the Silicon Valley Lab). In 1995, he joined the Advanced Technical Support group in San Jose. Currently, he is a Certified Consulting I/T Specialist and his main responsibilities are handling mainframe DASD performance critical situations

xviii


and performing Disk Magic study and remote copy sizing for clients who want to implement the IBM solution for their disaster recovery system.

The team: Rene, Bert, John, Brett, Gero, Anderson, Jean, Paulus, and Peter

Special thanksFor hosting this residency at the ESCC in Mainz, Germany, we want to thank: Rainer Zielonka - Director ESCC Dr. Friedrich Gerken - Manager Services and Technical Sales Support Rainer Erkens - Manager ESCC Service & Support Management Bernd Mller - Manager Enterprise Disk High-End Solutions Europe, for dedicating so many resources to this residency Stephan Weyrich - Opportunity Manager ESCC Workshops We especially want to thank Lee La Frese (IBM, Tucson) for being our special advisor and development contact for this book. Many thanks to those people in IBM in Mainz, Germany, who helped us with access to equipment as well as technical information and review: Uwe Heinrich Mueller, Uwe Schweikhard, Guenter Schmitt, Joerg Zahn, Werner Deul, Mike Schneider, Markus Oscheka, Hartmut Bohnacker, Gerhard Pieper, Alexander Warmuth, Kai Jehnen, Frank Krueger, and Werner Bauer Special thanks to: John Bynum DS8000 World Wide Technical Support Marketing Lead

Preface

xix

Thanks to the following people for their contributions to this project: Mary Anne Bromley Garry Bennet Jay Kurtz Rosemary McCutchen Brian J. Smith Sonny Williams IBM US Nick Clayton Patrick Keyes Andy Wharton Barry Whyte IBM UK Brian Sherman IBM Canada

Become a published authorJoin us for a two-week to six-week residency program. Help write an IBM Redbooks publication dealing with specific products or solutions, while getting hands-on experience with leading edge technologies. You will team with IBM technical professionals, IBM Business Partners, and clients. Your efforts will help increase product acceptance and client satisfaction. As a bonus, you will develop a network of contacts in IBM development labs, and increase your productivity and marketability. Find out more about the residency program, browse the residency index, and apply online at: ibm.com/redbooks/residencies.html

Comments welcomeYour comments are important to us. We want our IBM Redbooks publications to be as helpful as possible. Send us your comments about this or other IBM Redbooks publications in one of the following ways: Use the online Contact us review IBM Redbooks publication form found at: ibm.com/redbooks Send your comments in an e-mail to: [email protected] Mail your comments to: IBM Corporation, International Technical Support Organization Dept. HYTD Mail Station P099 2455 South Road Poughkeepsie, NY 12601-5400

xx


1

Chapter 1.

DS8000 characteristicsThis chapter contains a high level discussion and introduction to the storage server performance challenge. Then, we provide an overview of the DS8000 model characteristics that allow the DS8000 to meet this performance challenge.


1

1.1 The storage server challengeOne of the primary criteria in judging a storage server is performance: how fast it responds to a read or write request from an application server. How well a storage server accomplishes this task is dependent on the design of its hardware and firmware. Data continually moves from one component to another component within a storage server. The objective of server design is to have hardware of sufficient throughput to keep that data flowing smoothly without having to wait because a component is busy. When data stops flowing because a component is busy, a bottleneck has formed. Obviously, it is desirable to minimize the frequency and severity of bottlenecks. The ideal storage server is one in which all components are well utilized and bottlenecks are few. This scenario is the case if: The machine is designed well, with all hardware components in balance. To provide this balance over a range of workloads, a storage server must allow a range of hardware component options. The machine is sized well for the clients workload. That is, where options exist, the right quantities of each option were chosen. The machine is set up well. That is, where options exist in hardware installation and logical configuration, these options are chosen correctly.

1.1.1 Performance numbersRaw performance numbers provide evidence that a particular storage server is better than the previous generation model, or better than the competitions product. But isolated performance numbers are often out of line with a production environment. It is important to understand how raw performance numbers relate to the performance of the storage server in processing a particular production workload. Throughput numbers are usually achieved in controlled tests, which have the objective of pushing as much data as possible through the storage server as a whole, or perhaps through just a single component. At the point of maximum throughput, the system is usually so overloaded that response times are greatly extended. Trying to achieve such throughput numbers in a normal business environment brings protests from the users of the system, because response times are extremely poor. To assure yourself that the DS8000 offers the latest and fastest technology, take a look at the performance numbers for the individual disks, adapters, and other components of the DS8000, as well as for the total device. You will find that the DS8000 uses the most current technology available. But, use a more rigorous approach when planning the DS8000 hardware configuration to meet the requirements of a specific environment.

1.1.2 Recommendations and rulesHardware selections are sometimes based on general recommendations and rules. A general rule is a simple guideline for making a selection based on limited information. The advantage is that it allows you to make a quick decision, with little effort, that provides a solution that works acceptably well most of the time. The disadvantage is that it does not work all the time; sometimes, the solution is not at all what the client needs. You can increase the chances that the solution will work by making it more conservative. However, a conservative solution generally involves more hardware, which means a more expensive solution. In this chapter, we will provide recommendations and general rules for different hardware

2


components. Just remember, only use general rules when there is no information available to make a more informed decision.

1.1.3 Modeling your workloadA much better way to determine the hardware requirements for your workload is to run a Disk Magic model. Disk Magic is a modeling tool, which shows the throughput and response time of a storage server based on workload characteristics and the hardware resources of the storage server. By converting the results of performance runs into mathematical formulas, Disk Magic allows the results to be applied to a wide range of workloads. Disk Magic allows many variables of hardware to be brought together so the effect of each variable is integrated, producing a result that shows the overall performance of the storage server. For additional information about this tool, refer to 7.1, Disk Magic on page 162.

1.1.4 Allocating hardware components to workloadsThere are two contrasting methods to allocate the use of hardware components to workloads. The first method is spreading the workloads across components, which means that you try to share the use of hardware components across all, or at least many, workloads. The more hardware components are shared among multiple workloads, the more effectively the hardware components are utilized, which reduces total cost of ownership (TCO). For example, to attach multiple hosts, you can use the same host adapters for all hosts instead of acquiring a separate set of host adapters for each host. However, the more that components are shared, the more potential there is that one workload will dominate use of the component. The second method is isolating workloads to specific hardware components, which means that specific hardware components are used for one workload, and other hardware components are used for different workloads. The downside of isolating workloads is that certain components are unused when their workload is not demanding service. On the upside, it means that when that workload does demand service, the component is available immediately, and the workload does not have to contend with other workloads for that resource. Spreading the workload maximizes the utilization and performance of the storage server as a whole. Isolating a workload is a way to maximize that individual workloads performance, making it run as fast as possible. For a detailed discussion, refer to 5.1, Basic configuration principles for optimal performance on page 64.

1.2 Meeting the challenge: DS8000The DS8000 is a member of the DS product family. It offers disk storage servers with a wide range of hardware component options to fit many workload requirements, in terms of both type and size. It has the capability to scale very well to the highest disk storage capacities. The scalability is supported by design functions that allow installation of additional components without disruption. The IBM System Storage DS8000 has the performance to allow multiple workloads to be easily consolidated into a single storage subsystem.

Chapter 1. DS8000 characteristics

3

1.2.1 DS8000 models and characteristicsThe DS8000 series currently has three Turbo models available: the DS8100 Turbo Model 931 and the DS8300 Turbo Models 932 and 9B2. The difference in models is in the processors and in the capability of storage system logical partitions (LPARs). The predecessors of the DS8000 series Turbo models were the DS8000 series Models 921, 922, and 9A2. The base frame houses the processor complexes, including system memory, up to 16 host adapters, and up to 128 disk modules. The first expansion frame houses up to 16 additional host adapters (for a total of 32) and up to 256 additional disk modules (for a total of 384). A second expansion frame houses up to another 256 disk modules (for a grand total of 640). The third and fourth expansion frame houses up to 256 and 128 (for a grand total of 1024) additional disk modules respectively. There are no additional host adapters installed for the second, third, and fourth expansion frames. Table 1-1 provides an overview of the DS8000 models, including processor, memory, host adapter, and disk specifications for each model. Note that the DS8300 LPAR model is essentially the same as the non-LPAR model in terms of hardware components. However, the LPAR model provides a 50/50, a 75/25, or a 25/75 split of processors and system memory, and up to half the maximum number of host adapters and disk modules on each system image.Table 1-1 DS8000 processor models overview DS8100 Turbo model 931 Number of processor complexes Number of processors per complex Number of Storage Facility Images (SFIs) 2 2 1 DS8300 Turbo model 932 2 4 1 DS8300 Turbo LPAR model 9B2 2 4 2 (Each SFI has half the total processor resources) 2.2 GHz 32 GB 64 GB 128 GB 256 GB (Each SFI has half the total memory) 0-4 Model 9AE 4 - 32 (Each SFI can have 2 - 16) 4

Processor speed Processor Memory options (cache)

2.2 GHz 16 GB 32 GB 64 GB 128 GB

2.2 GHz 32 GB 64 GB 128 GB 256 GB

Expansion frames, minimum - maximum Expansion frame model Host adapters, minimum - maximum

0-1 Model 92E 2 - 16

0-4 Model 92E 2 - 32

Ports per Fibre Channel Protocol (FCP)/ Fibre Channel connection (FICON) host adapter Ports per Enterprise Systems Connection (ESCON) host adapter

4

4

2

2

2

4


DS8100 Turbo model 931 Disk drive modules (DDMs), minimum maximum 16 - 384

DS8300 Turbo model 932 16 - 1024

DS8300 Turbo LPAR model 9B2 32 - 1024

Next, we provide a short description of the main hardware components.

POWER5+ processor technologyThe DS8000 series exploits the IBM POWER5+ technology, which is the foundation of the storage system LPARs. The DS8100 Model 931 utilizes the 64-bit microprocessors dual 2-way processor complexes, and the DS8300 Model 932/9B2 uses the 64-bit dual 4-way processor complexes. Within the POWER5+ servers, the DS8000 series offers up to 256 GB of cache, which is up to four times as much as the previous ESS models.

Internal fabricThe DS8000 comes with a high bandwidth, fault tolerant internal interconnection, which is also used in the IBM System p servers. It is called RIO-2 (Remote I/O) and can operate at speeds up to 1 GHz and offers a 2 GB/s sustained bandwidth per link.

Switched Fibre Channel Arbitrated Loop (FC-AL)The disk interconnection has changed in comparison to the previous ESS. Instead of the Serial Storage Architecture (SSA) loops, there is now a switched FC-AL implementation. This implementation offers a point-to-point connection to each drive and adapter, so that there are four paths available from the controllers to each disk drive.

Disk drivesThe DS8000 offers a selection of industry standard Fibre Channel (FC) disk drives. There are 15k rpm FC drives available with 146 GB, 300 GB, or 450 GB capacity. The 500 GB Fibre Channel Advanced Technology Attachment (FATA) drives (7200 rpm) allow the system to scale up to 512 TB of capacity.

Host adaptersThe DS8000 offers enhanced connectivity with the availability of four-port Fibre Channel/FICON host adapters. The 4 Gb/s Fibre Channel/FICON host adapters, which are offered in longwave and shortwave, can also auto-negotiate to 2 Gb/s or 1 Gb/s link speeds. This flexibility enables immediate exploitation of the benefits offered by the higher performance, 4 Gb/s storage area network (SAN)-based solutions, while also maintaining compatibility with existing 2 Gb/s infrastructures. In addition, the four ports on the adapter can be configured with an intermix of Fibre Channel Protocol (FCP) and FICON, which can help protect your investment in Fibre adapters, and increase your ability to migrate to new servers. The DS8000 also offers two-port ESCON adapters. A DS8000 can support up to a maximum of 32 host adapters, which provide up to 128 Fibre Channel/FICON ports.

1.3 DS8000 performance characteristics overviewThe IBM System Storage DS8000 offers optimally balanced performance, which is over six times the throughput of the Enterprise Storage Server Model 800. This throughput is possible, because the DS8000 incorporates many performance enhancements, such as the dual-clustered POWER5+ servers, four-port 4 Gb Fibre Channel/FICON host adapters, new Fibre Channel disk drives, and the high-bandwidth, fault-tolerant internal interconnections.Chapter 1. DS8000 characteristics

5

With all these new components, the DS8000 is positioned at the top of the high performance category. As previously mentioned in this chapter, the following components contribute to the high performance of the DS8000: Redundant Array of Independent Disks (RAID), array across loops (AAL), POWER5+ processors, RIO, and the FC-AL implementation with a truly switched FC back end. In addition to these, there are even more contributions to performance as illustrated in the following sections.

1.3.1 Advanced caching techniquesThe DS8000 benefits from advanced caching techniques.

Sequential Prefetching in Adaptive Replacemen

Documents

DS 8000 Monitoring Applications