Implementation Guide X6 1.9.96 13

7/24/2019 Implementation Guide X6 1.9.96 13

1/244

Lenovo X6 Systems Solution for SAP HANA

Implementation Guide for System x X6 Servers

Lenovo Development for SAP Solutions

In cooperation with: SAP AG

Created on 3rd July 2015 15:13 Version 1.9.96-13

Copyright Lenovo, 2015


2/244

Technical Documentation

X6 Systems Solution for SAP HANA Platform Edition

Dear Reader,

We wish to explicitly announce that this guide book is for the System X6 based servers for SAP HANAPlatform Edition (Type 6241 Model AC3/AC4/Hxx) based on Intel Xeon IvyBridge or Haswell EXFamily of Processors.

Type 3837 of System X6 based servers and the System eX5 based servers for SAP HANA PlatformEdition (models 7147-H** and 7143-H**) are not discussed in this manual.

The Lenovo Systems X6 solution for SAP HANA Platform Edition is based on System X6 Architecturebuilding blocks that provide a highly scalable infrastructure for the SAP HANA Platform Edition ap-pliance software. The Systems x3850 X6, x3950 X6 and software, such as IBM General Parallel FileSystem(GPFS) will be used to run SAP HANA Platform Edition appliance software.

Lenovo has created orderable models upon which you may install and run the SAP HANA Platform Edi-tion appliance software according to the sizing charts coordinated with SAP AG. For each workload type,special ordering options for the System x3850 X6 and System x3950 X6 Type 6241 Models AC3/AC4/H xxhave been approved by SAP and Lenovo to accommodate the requirements for the SAP HANA PlatformEdition appliance software.

The Lenovo SAP HANA Development Team

X6 Implementation Guide1.9.96-13

Lenovo Systems Solution for SAP HANA

Copyright Lenovo, 2015I


3/244


Copyrights and Trademarks

Copyright 2010-2015 Lenovo.

Lenovo may not offer the products, services, or features discussed in this document in all countries.Consulty our local Lenovo representative for information on the products and services currently availablein your area. Any reference to a Lenovo product, program, or service is not intended to state or implythat only that Lenovo product, program, or service may be used. Any functionally equivalent product,program, or service that does not infringe any Lenovo intellectual property right may be used instead.However, it is the users responsibility to evaluate and verify the operation of any other product, program,or service.

Lenovo may have patents or pending patent applications covering subject matter described in this doc-ument. The furnishing of this document does not give you any license to these patents. You can sendlicense inquiries, in writing, to:

Lenovo (United States), Inc.

1009 Think Place - Building OneMorrisville, NC 27560U.S.A.Attention: Lenovo Director of Licensing

LENOVO PROVIDES THIS PUBLICATION AS IS WITHOUT WARRANTY OF ANY KIND, EI-THER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WAR-RANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULARPURPOSE.

Neither this documentation nor any part of it may be copied or reproduced in any form or by any meansor translated into another language, without the prior consent of Lenovo.

This document could include technical inaccurancies or errors. The information contained in this doc-ument is subject to change without any notice. Lenovo reserves the right to make any such changeswithout obligation to notify any person of such revision or changes. Lenovo makes no commitment tokeep the information contained herein up to date.

Any references in this publication to non-Lenovo Web sites are provided for convenience only and do notin any manner serve as an endorsement of those Web sites. The materials at those Web sites are not partof the materials for this Lenovo product, and use of those Web sites is at your own risk.

Information concerning non-Lenovo products was obtained from a supplier of these products, publishedannouncement material, or other publicly available sources and does not constitute an endorsementof such products by Lenovo. Sources for non-Lenovo list prices and performance numbers are takenfrom publicly available information, including vendor announcements and vendor worldwide home pages.

Lenovo has not tested these products and cannot confirm the accuracy of performance, capability, or anyother claims related to non-Lenovo products. Questions on the capability of non-Lenovo products shouldbe addressed to the supplier of those products.

Edition Notice: 3rd July 2015

This is the thirteenth published edition of this document. The online copy is the master.



Copyright Lenovo, 2015II


4/244


Lenovo, the Lenovo logo, System x and For Those Who Do are trademarks or registered trademarksof Lenovo in the United States, other countries, or both. Other product and service names might betrademarks of Lenovo or other companies.

A current list of Lenovo trademarks is available on the web at:http://www.lenovo.com/legal/copytrade.html.

IBM, the IBM logo, and ibm.com are trademarks of International Business Machines Corp., registered inthe United States and/or other countries.

Adobe and PostScript are either registered trademarks or trademarks of Adobe Systems Incorporated inthe United States and/or other countries.

Fusion-io is a registered trademark of Fusion-io, in the United States.

Intel, Intel Xeon, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation orits subsidiaries in the United States and other countries.

UNIX is a registered trademark of The Open Group in the United States and other countries.

Linux is a trademark of Linus Torvalds in the United States, other countries, or both.

SAP HANA is a trademark of SAP Corporation in the United States, other countries, or both.

Other company, product or service names may be trademarks or service marks of others.



Copyright Lenovo, 2015III
http://www.lenovo.com/legal/copytrade.htmlhttp://www.lenovo.com/legal/copytrade.htmlhttp://www.lenovo.com/legal/copytrade.html


5/244


Contents

1 Abstract 11.1 Preface & Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.2 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.3 Feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.4 Disclaimer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.5 Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2 Introduction 62.1 Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62.2 Applicability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.2.1 SAP HANA Platform Edition Versions. . . . . . . . . . . . . . . . . . . . . . . . . 62.3 Exclusions and Exceptions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62.4 Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.4.1 Icons Used . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.4.2 Code Snippets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

3 Solution Overview 73.1 The SAP HANA Appliance Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73.2 Definition of SAP HANA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

4 Hardware Configurations 94.1 SAP HANA Platform Edition T-Shirt Sizes . . . . . . . . . . . . . . . . . . . . . . . . . . 104.2 Single Node versus Clustered Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . 10

4.2.1 Network Switch Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114.3 SAP HANA Optimized Hardware Configurations . . . . . . . . . . . . . . . . . . . . . . . 13

4.3.1 System x3850 X6 Single Node Configurations . . . . . . . . . . . . . . . . . . . . . 134.3.2 System x3950 X6 Single Node Configurations . . . . . . . . . . . . . . . . . . . . . 13

4.3.3 System x3850 X6 Single Node Four Socket Configurations with Storage Expansion 144.3.4 System x3950 X6 SAP ERP on SAP HANA Single Node Configurations . . . . . . 144.3.5 System x3850 X6 Cluster Node Configurations with Storage Expansion . . . . . . 144.3.6 System x3950 X6 Cluster Node Configurations . . . . . . . . . . . . . . . . . . . . 15

4.4 Card Placement. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154.4.1 Network Interface Cards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154.4.2 Slots for additional Network Interface Cards. . . . . . . . . . . . . . . . . . . . . . 154.4.3 RAID Adapter Cards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

5 Networking 215.1 Networking Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215.2 Jumbo Frames . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215.3 Network Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225.4 Network Switch Configuration For Clustered Installations . . . . . . . . . . . . . . . . . . 235.5 Customer Site Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245.6 Network Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

5.6.1 Numbering conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245.6.2 Internal Networks Option 1 G8264 RackSwitch 10Gbit. . . . . . . . . . . . . . . 245.6.3 Internal Networks Option 2 G8124 RackSwitch 10Gbit. . . . . . . . . . . . . . . 265.6.4 Internal Networks Option 3 G8272 RackSwitch 10Gbit. . . . . . . . . . . . . . . 275.6.5 Internal Networks Option 4 G8296 RackSwitch 10Gbit. . . . . . . . . . . . . . . 285.6.6 Administrative, SAP-Access and Backup Networks Option G8052 RackSwitch

1Gbit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295.6.7 Network Configurations in a Clustered Environment . . . . . . . . . . . . . . . . . 30

5.7 Setting up the Switches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31



Copyright Lenovo, 2015IV


6/244


5.7.1 Basic Switch Configuration Setup. . . . . . . . . . . . . . . . . . . . . . . . . . . . 315.7.2 Advanced Setup of the Switches . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325.7.3 Disable Spanning Tree Protocol. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

5.7.4 Disable Default IP Address . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335.7.5 Enable L4Port Hash . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335.7.6 Disable Routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335.7.7 Add Networking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335.7.8 VLAN configurations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335.7.9 Save changes to switch FLASH memory . . . . . . . . . . . . . . . . . . . . . . . . 36

5.8 Inter-Site Portchannel Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365.8.1 Static Trunk over one Inter-Site Link. . . . . . . . . . . . . . . . . . . . . . . . . . 365.8.2 Portchannel over two Inter-Site Links . . . . . . . . . . . . . . . . . . . . . . . . . 375.8.3 Portchannel over four Inter-Site Links . . . . . . . . . . . . . . . . . . . . . . . . . 385.8.4 Save and Restore Switch Configuration . . . . . . . . . . . . . . . . . . . . . . . . 38

5.9 Generation of Switch Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395.9.1 Script Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

5.9.2 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395.9.3 Input Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

6 Guided Install of the Lenovo Solution 416.1 Preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

6.1.1 Firewall Preparations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 426.1.2 Lenovo Systems solution for SAP HANA Additional Software Stack . . . . . . . . 426.1.3 Software, Firmware and Drivers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 436.1.4 Card Placement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 456.1.5 Hardware UEFI Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

6.2 Phase 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 486.2.1 Storage Configuration RAID Setup. . . . . . . . . . . . . . . . . . . . . . . . . . 486.2.2 Mounting Installation Images using the IMM Virtual Media Center. . . . . . . . . 516.2.3 Starting the Automatic Installation Process . . . . . . . . . . . . . . . . . . . . . . 52

6.3 Phase 2 SLES for SAP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 536.4 Phase 2 RHEL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 586.5 Interim Check. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

6.5.1 Installation of Mandatory Packages . . . . . . . . . . . . . . . . . . . . . . . . . . 616.5.2 Installation without Network Connectivity. . . . . . . . . . . . . . . . . . . . . . . 62

6.6 Phase 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 626.6.1 Verification of RAID Controller and HDD/SSD Firmware . . . . . . . . . . . . . . 626.6.2 HANA Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 626.6.3 Single Node with HA Installation with Side-car Quorum Solution. . . . . . . . . . 65

7 After Installation 66

7.1 Actions to insure the correctness of the installation . . . . . . . . . . . . . . . . . . . . . . 667.2 HANA Network Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

8 Disaster Recovery 688.1 Architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

8.1.1 Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 688.1.2 Architectural overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 698.1.3 Three site/Tiebreaker node architecture . . . . . . . . . . . . . . . . . . . . . . . . 71

8.2 Mixing eX5/X6 Server in a DR Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . 718.3 Hardware Setup. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

8.3.1 Site A and B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 718.3.2 Tiebreaker Site C (optional). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 718.3.3 Acquire TCP/IP addresses and host names . . . . . . . . . . . . . . . . . . . . . . 72



Copyright Lenovo, 2015V


7/244


8.3.4 Network switch setup (GPFS and SAP HANA network) . . . . . . . . . . . . . . . 728.3.5 Link between site A and B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 728.3.6 Network integration into customer infrastructure . . . . . . . . . . . . . . . . . . . 73

8.3.7 Setup network connection to tiebreaker node at site C (optional) . . . . . . . . . . 738.4 Software Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 738.4.1 GPFS configuration prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . 748.4.2 GPFS Server configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 768.4.3 GPFS Disk configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 778.4.4 Filesystem Creation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 788.4.5 SAP HANA appliance installation . . . . . . . . . . . . . . . . . . . . . . . . . . . 798.4.6 Tiebreaker node setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 818.4.7 Verify Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

8.5 Extending a DR-Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 838.6 Mixing eX5/X6 Server in a DR Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

8.6.1 Hardware Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 838.6.2 GPFS Part 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

8.6.3 HANA Backup Node Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 858.6.4 GPFS Part 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 868.6.5 HANA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

8.7 Using Non Productive Instances on Inactive DR Site . . . . . . . . . . . . . . . . . . . . . 878.7.1 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 878.7.2 Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

9 Mixed eX5/X6 Environments 919.1 Mixed eX5/X6 HA Clusters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

9.1.1 Definition & Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 919.1.2 Prerequisites & Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 919.1.3 New Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 929.1.4 Existing Cluster Extension/Node Replacement . . . . . . . . . . . . . . . . . . . . 949.1.5 Deviating Operation Instructions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

9.2 Mixed eX5/X6 DR Clusters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 979.2.1 Definition & Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 979.2.2 Prerequisites & Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 979.2.3 New Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 989.2.4 Existing Cluster Extension/Node Replacement . . . . . . . . . . . . . . . . . . . . 999.2.5 Deviating Operation Instructions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

10 Special Single Node Installation Scenarios 10310.1 Single Node with HA Installation with Side-car Quorum Solution . . . . . . . . . . . . . . 103

10.1.1 Installation of SAP HANA appliance single node with HA . . . . . . . . . . . . . . 10410.1.2 Prepare quorum node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

10.1.3 Quorum Node Network Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10610.1.4 Adapt hosts file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10710.1.5 SSH configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10710.1.6 Quorum Node IBM GPFS setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10810.1.7 Quorum Node IBM GPFS installation . . . . . . . . . . . . . . . . . . . . . . . . . 10810.1.8 Add quorum node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11010.1.9 Create descriptor disk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11010.1.10 Add disk to file system. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11010.1.11 Verify Cluster Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11010.1.12 Installation of SAP HANA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

10.2 Single Node with stretched HA Installation . . . . . . . . . . . . . . . . . . . . . . . . . . 11110.2.1 Installation and configuration of SLES and IBM GPFS . . . . . . . . . . . . . . . 11210.2.2 Installation of SAP HANA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113



Copyright Lenovo, 2015VI


8/244


10.3 Single Node with DR Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11310.3.1 Installation and configuration of SLES and IBM GPFS . . . . . . . . . . . . . . . 11510.3.2 Optional: Expansion Storage Setup for Non-Production Instance . . . . . . . . . . 115

10.4 Single Node with HA and DR Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . 11510.4.1 Installation and configuration of SLES and IBM GPFS . . . . . . . . . . . . . . . 11610.4.2 Optional: Expansion Storage Setup for Non-Production Instance . . . . . . . . . . 118

10.5 Single Node DR Installation with SAP HANA System Replication . . . . . . . . . . . . . 11910.5.1 Installation and configuration of SLES and IBM GPFS . . . . . . . . . . . . . . . 12010.5.2 Installation of SAP HANA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12110.5.3 Optional: Expansion Storage Setup for Non-Production Instance . . . . . . . . . . 121

10.6 Single Node with HA using IBM GPFS Storage Replication and DR using System Replication12210.6.1 Installation and configuration of SLES and IBM GPFS . . . . . . . . . . . . . . . 12410.6.2 Installation of SAP HANA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125

10.7 Expansion Storage Setup for Non-productive SAP HANA Instance . . . . . . . . . . . . . 126

11 Virtualization 128

11.1 Getting Started . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12811.1.1 Memory Overhead . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12811.1.2 Configure UEFI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12911.1.3 Start Embedded VMware ESXi Hypervisor . . . . . . . . . . . . . . . . . . . . . . 12911.1.4 Configure Management Network of ESXi Hypervisor . . . . . . . . . . . . . . . . . 12911.1.5 Enable SSH on VMware ESXi Hypervisor . . . . . . . . . . . . . . . . . . . . . . . 13311.1.6 StorCLI on VMware ESXi 5.5. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13311.1.7 Setting up ESXi Storage in CLI. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13511.1.8 setting up vswitches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13511.1.9 setting up nic bonding(teaming) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13611.1.10 Setting Storage for SLES and HANA ISO . . . . . . . . . . . . . . . . . . . . . . . 13611.1.11 Restart VMware ESXi Hypervisor . . . . . . . . . . . . . . . . . . . . . . . . . . . 13711.1.12 Installing VMware vSphere Client . . . . . . . . . . . . . . . . . . . . . . . . . . . 137

11.2 Configuring and Starting VMs with vSphere Client . . . . . . . . . . . . . . . . . . . . . . 13811.3 Operating System (SLES for SAP 11 SP3) Installation . . . . . . . . . . . . . . . . . . . . 15011.4 Operating System (Red Hat Enterprise Server 6.5 and 6.6) Installation . . . . . . . . . . . 150

11.4.1 Changes after Red Hat Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . 15111.5 Tuning of Operating System and VM. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152

11.5.1 Tuning of OS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15211.5.2 Tuning of ESXi and VM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152

12 Upgrading the Hardware Configuration 15412.1 Power Policy Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15412.2 Reboot Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15412.3 Adding storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156

12.3.1 Adding storage via EXP2524 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15612.3.2 Adding storage on second internal M5210 controller . . . . . . . . . . . . . . . . . 15612.3.3 Configure RAID array(s) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15712.3.4 Deciding for a CacheCade RAID Level . . . . . . . . . . . . . . . . . . . . . . . . . 15812.3.5 Configuring RAID array when CacheCade is not yet configured . . . . . . . . . . . 15812.3.6 Configuring RAID array with existing CacheCade . . . . . . . . . . . . . . . . . . 15812.3.7 Changing the CacheCade RAID Level . . . . . . . . . . . . . . . . . . . . . . . . . 15812.3.8 Configuring GPFS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159

12.4 Adding memory. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16012.5 Adding CPU Books . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161

13 Software Updates 16213.1 Warning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162



Copyright Lenovo, 2015VII


9/244


13.2 Update Variants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16213.2.1 General per node update procedure . . . . . . . . . . . . . . . . . . . . . . . . . . 16213.2.2 Disruptive Cluster Update. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164

13.2.3 Full Cluster Rolling Update . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16413.3 RHEL versionlock . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16413.4 Linux Kernel Update . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165

13.4.1 SLES Kernel Update Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16513.4.2 RHEL Kernel Update Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16613.4.3 Kernel Update Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166

13.5 Updating GPFS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16713.5.1 Disruptive GPFS Cluster Update . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168

13.6 Upgrading from GPFS 3.5 to 4.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17113.6.1 Disruptive Upgrade from GPFS 3.5 to 4.1 . . . . . . . . . . . . . . . . . . . . . . . 17213.6.2 Rolling upgrade per node from GPFS 3.5 to 4.1. . . . . . . . . . . . . . . . . . . . 174

13.7 Update Mellanox Network Cards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17613.8 SAP HANA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177

13.9 Upgrade VMware ESXi 5.5 to 5.5U2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177

14 Operating System Upgrade 17814.1 Upgrade RHEL 6.5 to 6.6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17814.2 Rolling Upgrade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17814.3 Upgrade Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17814.4 Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17914.5 Shutting down services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17914.6 Upgrade of IBM GPFS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17914.7 Update Mellanox Drivers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18014.8 Upgrading Red Hat. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18014.9 Mandatory Kernel Update . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18114.10Update of nss-softokn packages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18114.11Recompile Linux Kernel Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18114.12Adapting Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18114.13Start IBM GPFS and HANA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182

15 System Check and Support 18315.1 System Login . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18315.2 Basic System Check . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18315.3 System Support. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18615.4 Additional Tools for System Checks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187

15.4.1 Lenovo Advanced Settings Utility. . . . . . . . . . . . . . . . . . . . . . . . . . . . 18715.4.2 ServeRAID StorCLI Utility for Storage Management . . . . . . . . . . . . . . . . . 18715.4.3 SSD Wear Gauge CLI utility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187

15.5 Getting Support (IBM PMR, SAP OSS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188

16 Backup and Restore of the Primary Partition 18916.1 Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189

16.1.1 Boot Loader . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19016.1.2 Drive Partitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191

16.2 Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19116.2.1 Correcting the backup fstab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19216.2.2 Add boot loader entry for backup partition . . . . . . . . . . . . . . . . . . . . . . 193

16.3 Backup of the Linux operating system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19516.4 Restoring the operating system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195

17 SAP HANA Backup and Recovery 197



Copyright Lenovo, 2015VIII


10/244


17.1 Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19717.2 Backup of SAP HANA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19717.3 Restore of SAP HANA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201

18 Troubleshooting 20418.1 Adding SAP HANA Worker/Standby Nodes in a Cluster. . . . . . . . . . . . . . . . . . . 20418.2 GPFS mount points missing after Kernel Update . . . . . . . . . . . . . . . . . . . . . . . 20418.3 Degrading disk I/O throughput . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20418.4 SAP HANA will not install after a system board exchange . . . . . . . . . . . . . . . . . . 20518.5 Known Kernel Updates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20518.6 Important SAP Notes (SAP Service Marketplace ID required) . . . . . . . . . . . . . . . . 205

18.6.1 SAP Note 1641148 HANA server hang caused by GPFS issue . . . . . . . . . . . . 205

Appendices 207

A GPFS Disk Descriptor Files 207

B Topology Vectors (GPFS 3.5 failure groups) 207

C Quotas 209C.1 Quota Calculation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209C.2 Quota Calculation Script . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209

D Performance Settings 211

E Lenovo X6 Server MTM List & Model Overview 214

F Frequently Asked Questions 216F.1 FAQ #1: SAP HANA Memory Limits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216F.2 FAQ #2: GPFS parameter readReplicaPolicy . . . . . . . . . . . . . . . . . . . . . . . . . 216F.3 FAQ #3: SAP HANA Memory Limit on XS sized Machines . . . . . . . . . . . . . . . . . 216F.4 FAQ #4: Overlapping NSDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217F.5 FAQ #5: Missing RPMs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217F.6 FAQ #6: CPU Governor set to ondemand . . . . . . . . . . . . . . . . . . . . . . . . . . . 220F.7 FAQ #7: No disk space left bug (Bug IV33610) . . . . . . . . . . . . . . . . . . . . . . . . 220F.8 FAQ #8: Setting C-States . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221F.9 FAQ #9: ServeRAID M5120 RAID Adapter FW Issues . . . . . . . . . . . . . . . . . . . 221

F.9.1 Changing Queue Depth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222F.9.2 Use recommended Firmware version . . . . . . . . . . . . . . . . . . . . . . . . . . 222

F.10 FAQ #10: GPFS Parameter enableLinuxReplicatedAIO . . . . . . . . . . . . . . . . . . . 223F.11 FAQ #11: GPFS NSD on Devices with GPT Labels . . . . . . . . . . . . . . . . . . . . . 223F.12 FAQ #12: GPFS pagepool should be set to 4GB . . . . . . . . . . . . . . . . . . . . . . . 224

F.13 FAQ #13: Limit Page Cache Pool to 4GB (SAP Note #1557506 . . . . . . . . . . . . . . 225F.14 FAQ #14: restripeOnDiskFailure and start-disks-on-startup . . . . . . . . . . . . . . . . . 225

G References 226G.1 Lenovo References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226G.2 IBM References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226G.3 SAP General Help (SAP Service Marketplace ID required). . . . . . . . . . . . . . . . . . 226G.4 SAP Notes (SAP Service Marketplace ID required) . . . . . . . . . . . . . . . . . . . . . . 227G.5 Novell SUSE Linux Enterprise Server References . . . . . . . . . . . . . . . . . . . . . . . 228G.6 Red Hat Enterprise Linux References (Red Hat account required). . . . . . . . . . . . . . 228

H Changelog 229



Copyright Lenovo, 2015IX


11/244


List of Figures

1 Current SAP HANA Appliance Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 Hardware Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 SAP HANA Multiple Single Node Example . . . . . . . . . . . . . . . . . . . . . . . . . . 104 SAP HANA Clustered Example with Backup . . . . . . . . . . . . . . . . . . . . . . . . . 115 Workload Optimized System x3850 X6 2 Socket Rear View . . . . . . . . . . . . . . . . . 176 Workload Optimized System x3850 X6 4 Socket Rear View . . . . . . . . . . . . . . . . . 187 Workload Optimized System Storage Book. This contains slots 11, 12 and slots 43, 44 on

x3950 X6 in an additional Storage Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188 Workload Optimized System x3950 X6 8 Socket Rear View . . . . . . . . . . . . . . . . . 209 G8264 RackSwitch front view . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2510 G8124 RackSwitch front view . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2611 G8272 RackSwitch front view . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2712 G8296 RackSwitch front view . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2913 G8052 RackSwitch front view . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

14 Cluster Node Network Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3115 Cluster Switch Networking Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3216 License Agreement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5317 Hostname and Domain Name . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5418 Network Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5419 Cluster Node NIC Configuration dialog bond0. . . . . . . . . . . . . . . . . . . . . . . . . 5620 Clock and Time Zone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5721 Advanced NTP Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5722 Password for the System Administrator . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5823 Installation Mode Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6324 GPFS IP Configuration Dialog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6325 HANA Password Input Dialog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6426 DR Architectural Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6827 DR Data Distribution in a Four Node Cluster . . . . . . . . . . . . . . . . . . . . . . . . . 6928 Logical DR Network Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7029 DR Networking View (with no client uplinks shown) . . . . . . . . . . . . . . . . . . . . . 7030 SAP HANA DR using storage expansion - architectural overview . . . . . . . . . . . . . . 8831 Single Node with High Availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10332 File System Layout - Single Node HA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10433 Network Switch Setup for Single Node with HA . . . . . . . . . . . . . . . . . . . . . . . . 10734 Single Node with stretched HA - Two Site Approach . . . . . . . . . . . . . . . . . . . . . 11235 Single Node with stretched HA - Three Site Approach . . . . . . . . . . . . . . . . . . . . 11236 File System Layout - Single Node stretched HA . . . . . . . . . . . . . . . . . . . . . . . . 11337 Single Node with Disaster Recovery - Two Site Approach . . . . . . . . . . . . . . . . . . 11438 Single Node with Disaster Recovery - Three Site Approach . . . . . . . . . . . . . . . . . 114

39 File System Layout - Single Node with DR with Storage Expansion. . . . . . . . . . . . . 11540 Single Node with HADR using IBM GPFS Storage Replication . . . . . . . . . . . . . . . 11641 File System Layout - Single Node HADR . . . . . . . . . . . . . . . . . . . . . . . . . . . 11742 File System Layout - Single Node HADR with Storage Expansion. . . . . . . . . . . . . . 11943 Single Node DR with SAP System Replication . . . . . . . . . . . . . . . . . . . . . . . . 12044 Single Node DR with SAP System Replication . . . . . . . . . . . . . . . . . . . . . . . . 12045 File System Layout of Single Node DR with SAP System Replication. . . . . . . . . . . . 12146 File System Layout of Single Node DR with SAP System Replication with Storage Expansion12247 Single Node with HA using IBM GPFS Storage Replication and DR using System Replication12348 Single Node with HA using IBM GPFS Storage Replication and DR using System Repli-

cation without remote site Switches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12449 File System of Single Node with HA and DR with System Replication . . . . . . . . . . . 125

50 File System of Single Node with HA and DR with System Replication and Storage Expansion126



Copyright Lenovo, 2015X


12/244


51 login to ESXi . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12952 configure management network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13053 display network adapters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130

54 display network adapters 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13155 IP configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13156 Set IP,NETMASK,GW . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13257 Set DNS and Hostname . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13258 Set DNS suffix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13359 ESXi 5.x filesystems on a System x3850 X6. The VFAT filesystems belong to the USB

device . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13560 ESXi5.5 Storage Path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13761 ESXi 5.1 WEB Welcome . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13862 Create new virtual machine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13963 Choose custom configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13964 Choose a name . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14065 Choose disk storage for VM files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140

66 Newest virtual machine hardware version . . . . . . . . . . . . . . . . . . . . . . . . . . . 14167 Configure the use of more than 32 CPUs . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14168 Choose Operating System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14269 Choose number of CPUs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14270 Choose Memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14371 Choose Network Cards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14372 Choose SCSI controller. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14473 Create new HANA datastore . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14474 Choose datastore size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14575 Choose datastore . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14576 Choose SCSI Node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14677 Add a new CD/DVD device . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146

78 Select ISO image . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14779 Select IDE device 0:0. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14780 Finish creation of SLES ISO mount. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14881 Upgrade virtual hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14882 Confirm upgrade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14983 Upgrade virtual hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14984 Changing the autoyast parameter for installation . . . . . . . . . . . . . . . . . . . . . . . 15085 Adding kickstart parameter for install . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15186 Overview of Backup/Restore Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19087 Sample GRUB boot loader screen. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196



Copyright Lenovo, 2015XI


13/244


List of Tables

1 Network Switch Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122 System x3850 X6 Single Node Configurations . . . . . . . . . . . . . . . . . . . . . . . . . 133 IBM System x3950 X6 Single Node Four Socket Configurations . . . . . . . . . . . . . . . 134 System x3950 X6 Single Node Eight Socket Configurations. . . . . . . . . . . . . . . . . . 135 System x3950 X6 Single Node Four Socket Configurations with Storage Expansion . . . . 146 System x3950 X6 SAP ERP on SAP HANA Single Node Configurations . . . . . . . . . . 147 System x3850 X6 Cluster Node Configurations with Storage Expansion . . . . . . . . . . 148 System x3950 X6 Cluster Node Configurations . . . . . . . . . . . . . . . . . . . . . . . . 159 Slots which may be used for additional NICs . . . . . . . . . . . . . . . . . . . . . . . . . 1610 Card assignments for a two socket x3850 X6. . . . . . . . . . . . . . . . . . . . . . . . . . 1611 Card assignments for a four socket x3850 X6 . . . . . . . . . . . . . . . . . . . . . . . . . 1712 Network interface card assignments for an eight socket x3950 X6 . . . . . . . . . . . . . . 1913 Card placement for x3950 X6 four socket and eight socket . . . . . . . . . . . . . . . . . . 1914 Customer infrastructure addresses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

15 IP address configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2316 Numbering conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2417 G8264 RackSwitch port assignments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2518 G8124 RackSwitch port assignments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2719 G8272 RackSwitch port assignments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2820 G8296 RackSwitch port assignments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2921 G8052 RackSwitch port assignments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3022 Installation Process and Phases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4123 SAP HANA references . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4224 DVD Part Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4325 Supported Firmware, Software and Driver Levels . . . . . . . . . . . . . . . . . . . . . . . 4426 Required Operation Modes UEFI settings . . . . . . . . . . . . . . . . . . . . . . . . . . . 4627 Required Processors UEFI settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4728 Required Power UEFI settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4729 Required Memory UEFI settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4730 Boot options and boot loaders used. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4831 x3850 X6 RAID Controller Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . 4932 x3950 X6 RAID Controller Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . 5033 Partition Scheme for Single Nodes and Cluster Installations . . . . . . . . . . . . . . . . . 5134 DVD/ISO Media Install Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5235 Hostname Settings for DR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7236 Extra Network Settings for DR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7237 GPFS Settings for DR Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7638 eX5 T-Shirt Size to X6 Model Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9139 Stanza file for X6 servers in eX5 clusters . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

40 Stanza file for X6 servers in eX5 clusters . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9541 eX5 T-Shirt Size to X6 Model Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9842 Stanza file for X6 servers in eX5 clusters . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10043 Stanza file for X6 servers in eX5 clusters . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10144 IBM System x3550 M4 GPFS quorum node . . . . . . . . . . . . . . . . . . . . . . . . . . 10545 Single Node with HA OS Partitioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10646 Single Node with HA OS Networking Setup . . . . . . . . . . . . . . . . . . . . . . . . . . 10647 Single Node with HA Network Switch Definitions . . . . . . . . . . . . . . . . . . . . . . . 10748 SAP HANA Virtual Machine Sizes by Lenovo . . . . . . . . . . . . . . . . . . . . . . . . . 12849 RAID array and RAID controller overview. . . . . . . . . . . . . . . . . . . . . . . . . . . 15550 x3850 X6 Memory DIMM Placement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16051 x3950 X6 Memory DIMM Placement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160

52 Upgrade GPFS Portability Layer Checklist . . . . . . . . . . . . . . . . . . . . . . . . . . 166



Copyright Lenovo, 2015XII


14/244


53 Upgrade GPFS Portability Layer Checklist . . . . . . . . . . . . . . . . . . . . . . . . . . 16854 GPFS Upgrade Checklist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17255 Required SAP HANA directories for restore . . . . . . . . . . . . . . . . . . . . . . . . . . 200

56 Topology Vectors in a 8 node DR-cluster. . . . . . . . . . . . . . . . . . . . . . . . . . . . 20857 Lenovo MTM Mapping & Model Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 21458 Lenovo MTM Mapping & Model Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 21559 ServeRAID M5120 Firmware Issues. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222

List of Listings

1 SSH login screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1832 Support script usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1833 Support script output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1844 Example SUSE fstab entries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1915 Example Red Hat fstab entries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192

6 Creating a copy of the motd file. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1927 Example SLES primary fstab file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1928 Example SLES backup fstab file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1929 Example RHEL backup fstab file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19310 Changing files for backup partition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19311 Example UEFI Configuration for Primary Partition . . . . . . . . . . . . . . . . . . . . . 19312 Example GRUB Configuration for Primary Partition . . . . . . . . . . . . . . . . . . . . . 19413 Example GRUB Configuration for Backup Partition . . . . . . . . . . . . . . . . . . . . . 19514 Example rsync command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19515 Example rsync command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19616 Example SLES primary fstab file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19617 Example RHEL primary fstab file. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196



Copyright Lenovo, 2015XIII


15/244


List of Abbreviations

ASU Lenovo Advanced Settings Utility

BIOS Basic Input / Output System

DR Disaster Recovery (previously SAP Disaster Tolerance)

DT SAP Dynamic Tiering (not to be confused with Disaster Recovery (DR), previouslyDisaster Tolerance (DT))

ELILO EFI Linux Loader

IBM GPFS IBM General Parallel File System

GRUB Grand Unified Bootloader

GSS GPFS Storage Server

IMM Integrated Management Module

LILO Linux Loader

MTM Machine Type Model

NIC Network Interface Controller

OLAP On Line Analytical Processing

OLTP On Line Transaction Processing

OS Operating System

RHEL Red Hat Enterprise Linux

SAP HANA SAP HANA Platform Edition

SLES SUSE Linux Enterprise Server

SLES for SAP SUSE Linux Enterprise Server for SAP Applications

UEFI Unified Extensible Firmware Interface

UUID Universally Unique Identifier

VLAG Virtual Link Aggregation Group

VLAN Virtual Local Area Network



Copyright Lenovo, 2015XIV


16/244


1 Abstract

This document provides general information specific to the Lenovo Systems Solution for SAP HANA

Platform Edition (short: Lenovo Solution). This document assumes that the reader understands thebasic structure and components of the SAP HANA Platform Edition (SAP HANA) software, that hehas a solid understanding of Linux administration processes, and that he has been instructed how toinstall theSAP HANA1 software on Lenovo Systems hardware.

Lenovo Solution is built with Lenovo Systems hardware based on Intel Xeon Architecture as buildingblocks for a scale-up or scale-outSAP HANA system. These provide a highly-scalable infrastructure forSAP HANA. The Lenovo Systems servers with local storage and Lenovo Systems Networking switcheswill be used to runSAP HANA.

Lenovo has created orderable models upon which you may install and run the SAP HANAaccording tothe sizing charts coordinated with SAP AG. For each workload type, special ordering options for theLenovo System servers, storage and switches have been approved by SAP and Lenovo to accommodatethe requirements for theSAP HANA.

Attention

IMPORTANT: Please do not attempt to install a system without having been instructedabout the content of this document.

Note

It is considered best practice to create backups before and recover the SAP HANAsystemafter a major failure instead of relying on a fresh install with the help of this document. Fordetails on Backup and Recovery please refer to the Lenovo Solution Backup & Restore Guideas well as the Lenovo Solution Hardware, Operating System & GPFS Operations Guide (SAPNote 1650046).

Copyright 2014-2015 Lenovo. All Rights Reserved.Neither this documentation nor any part of it may be copied or reproduced in any form or by any meansor translated into another language, without the prior consent of Lenovo.

Lenovo makes no warranties or representations with respect to the content hereof and specifically disclaimsany implied warranties of merchantability or fitness for any particular purpose. Lenovo assumes noresponsibility for any errors that may appear in this document. The information contained in thisdocument is subject to change without any notice. Lenovo reserves the right to make any such changeswithout obligation to notify any person of such revision or changes. Lenovo makes no commitment tokeep the information contained herein up to date.

Edition Notice: 3rd July 2015

This is the published edition of this document. The online copy is the master.

1

SAP HANA Platform Edition





17/244


1.1 Preface & Scope

The objective of this paper is to document the installation and configuration of the SAP HANA Platform

Edition (SAP HANA) on System x hardware using a managed set up rather than manually installingeach node from scratch. The major products installed here areSAP HANA, IBM General Parallel FileSystem (IBM GPFS)and the operating systems SUSE Linux Enterprise Server for SAP Applications(SLES for SAP), or Red Hat Enterprise Linux (RHEL).

For instructions how to administrate SAP HANA Platform Edition (SAP HANA) please refer to theSAP HANA Technical Operations Manual2. Instructions how to administrate and maintain the othercomponents delivered with the System x solution can be found in the SAP Note1650046Lenovo SystemsSolution Hardware, Operating System & GPFS Operations Guide. The Lenovo System x solution forSAP HANAQuick Start Guideprovides an overview of the complete solution and instructions how tofind service and support for your Lenovo Solution.

1.2 AcknowledgementsThe authors of this document are:

Martin Bachmaier, Lenovo Development for SAP Solutions, Germany

Florian Bausch, Lenovo Development for SAP Solutions, Germany

Hans-Peter Droste, Lenovo Development for SAP Solutions, Germany

Detlev Freund, Lenovo Development for SAP Solutions, Germany

Patrick Hartman, Lenovo Development for SAP Solutions, Germany

Guido Kampe, Lenovo Development for SAP Solutions, Germany

Nils Knig, Lenovo Development for SAP Solutions, Germany Christoph Nelles, Lenovo Development for SAP Solutions, Germany

Richard Ott, Lenovo Development for SAP Solutions, Germany

Volker Pense, Lenovo Development for SAP Solutions, Germany

Michael Reumann, Lenovo Development for SAP Solutions, Germany

The authors would like to thank the following Lenovo and IBM colleagues:

Herbert Diether, Lenovo Development for SAp Solutions, Germany

Oliver Rettig, Lenovo Development for SAP Solutions, Germany

Keith Frisby, Lenovo Systems Lab Services, US

Thorsten Nitsch, IBM GTS, Germany

Alexander Trefs, Lenovo Technical Sales, Germany

And many people at SAP Development, Walldorf, Germany; specifically:

Abdelkader Sellami, SAP HANA Support, Walldorf, Germany

Adolf Brosig, SAP HANA Development, Walldorf, Germany

Helmut Cossmann, SAP HANA Development, Walldorf, Germany

Henning Sackewitz, SAP Development, Walldorf, Germany

2http://help.sap.com/hana_platform



http://help.sap.com/hana_platformhttps://service.sap.com/sap/support/notes/1650046http://www.ibm.com/support/entry/portal/docdisplay?lndocid=MIGR-5087035#APPLYCOUNTRYhttp://help.sap.com/hana_platformhttp://help.sap.com/hana_platformhttp://www.ibm.com/support/entry/portal/docdisplay?lndocid=MIGR-5087035#APPLYCOUNTRYhttps://service.sap.com/sap/support/notes/1650046http://help.sap.com/hana_platform


18/244


Michael Becker, SAP HANA Support Development, Walldorf, Germany

Oliver Rebholz, SAP HANA Development, Walldorf, Germany

1.3 Feedback

We are interested in your comments and feedback. Please send it to [email protected]. The fullguidebook can be downloaded, depending on its version, from following community (SAP HANA SupportDocument section) SAP Solutions at Lenovo Community.

1.4 Disclaimer

This document is subject to change without notification and will not cover the issues encountered inevery customer situation. It should be used only in conjunction with the official product literature. Theinformation contained in this document has not been submitted to any formal test and is distributed ASIS.

All statements regarding Lenovo future direction and intent are subject to change or withdrawal withoutnotice, and represent goals and objectives only. Contact your local Lenovo office or Lenovo authorizedreseller for the full text of the specific Statement of Direction.

Some information addresses anticipated future capabilities. Such information is not intended as a defini-tive statement of a commitment to specific levels of performance, function or delivery schedules withrespect to any future products. Such commitments are only made in Lenovo product announcements.The information is presented here to communicate Lenovos current investment and development activitiesas a good faith effort to help with our customers future planning.

This document is for educated service personnel only. If you are not familiar with the described system,we will ask you to restrain from trying to apply what is described herein you could void the preloadedsystem installation and void the SAP certified configuration. This will void the warranty and support ofsaid machine. Please contact [email protected] get enrolled for education prior to installingan Lenovo Solution appliance.

In case of issues with theSAP HANAappliance, the customer is asked to open a SAP Help Desk request(OSS ticket) first and foremost. Only by following this path, can we ensure the proper configuration ofthe Lenovo Solution. If the customer would open an Lenovo support ticket for the system, he might berequested to perform system upgrades to firmware or software to the latest available levels which mightnot be supported with the SAP HANA appliance. If identified as a hardware or file system issue, theticket will be forwarded to the Lenovo support team and handled appropriately. Although this may becontrary to standard Lenovo Support processes, it is the approved and accepted support process for allSAP Appliances including theSAP HANAappliance.

1.5 Support

The System xSAP HANAdevelopment team provides new images for theSAP HANAappliance at reg-ular intervals. These images have dependencies regarding the hardware, operating systems, and hardwaredrivers. The use of the latest image for maintenance and installation ofSAP HANAappliance is highlyrecommended.

Whenever the firmware level recommendations (fixes known firmware issues) for the Lenovo componentsof the SAP HANA appliance are given by the individual System x support representatives, it is thecustomers responsibility to upgrade (or downgrade) to the recommended levels as instructed by Systemx support representatives. A list of the minimally required versions can be found in SAP Note1880960 Lenovo Systems Solution for SAP HANA Platform Edition FW/OS/Driver Maintenance.



mailto:[email protected]://apps.na.collabserv.com/wikis/home?lang=de-de#!/wiki/Wd1cffd934cc7_4a4f_97bf_d6e0e396ad09https://apps.na.collabserv.com/wikis/home?lang=de-de#!/wiki/Wd1cffd934cc7_4a4f_97bf_d6e0e396ad09mailto:[email protected]://service.sap.com/sap/support/notes/1880960https://service.sap.com/sap/support/notes/1880960mailto:[email protected]://apps.na.collabserv.com/wikis/home?lang=de-de#!/wiki/Wd1cffd934cc7_4a4f_97bf_d6e0e396ad09mailto:[email protected]


19/244


Whenever the operating systems recommendations (fixes known operating systems issues) for the SUSELinux components of the SAP HANAappliance are given by the SAP, SUSE, or IBM/Lenovo supportrepresentatives, it is the customers responsibility to upgrade (or downgrade) to the recommended levels

as instructed by SAP through an explicit SAP Note or a Customer OSS Message. SAP describes theiroperational concept, including updating of the operating system components in SAP Note 1599888SAPHANA: Operational Concept. If the Linux kernel is updated, you have to recompile IBM GPFS softwareas well.

Whenever other hardware or software recommendations (that fix known issues) for components of theSAP HANAappliance are given by the individual Lenovo support representatives, it is the customersresponsibility to upgrade (or to downgrade) to the recommended levels as instructed by Lenovo supportrepresentatives.

If software and documentation updates are available, you can download them from the respective Lenovo,IBM, SUSE or SAP website. To check for updates, go to the following websites. Follow the procedure inthe included documentation to update the software.

Firmware and drivers for System X6 Servers You can obtain updates for System x3850/x3950 X6 servers on the IBM support website (Fix

Central) at http://www.ibm.com/support/fixcentralusing the the Find product tab.

IBM General Parallel File System (IBM GPFS3) and IBM Spectrum Scale updates

You can obtain updates for GPFS on the IBM support website for GPFS 3.5.0, GPFS 4.1.0andIBM Spectrum Scale/GPFS 4.1.1

SUSE Linux Enterprise Server for SAP Applications 11 SP3

You can download the installation package from the SUSE website at http://download.novell.com/Download?buildid=XL0RqEykZpc~

SUSE Linux patches and updates

You can obtain the latest code updates for SUSE from the SUSE website at http://download.novell.com/patch/finder/

Red Hat Enterprise Linux 6.5 and 6.6

You can download the installation package from the Red Hat website at http://www.redhat.com/en/technologies/linux-platforms/enterprise-linux

VMware ESX Server patches and updates

You can obtain the latest code updates for vSphere ESX server from the VMware website athttp://www.vmware.com/support/

SAP HANA appliance updates

You can obtain the latest code updates from SAP at the SAP Service Marketplace at http://service.sap.com/swdc

Lenovo recommends that customers follow the software upgrade recommendations set out by SAP in theSAP HANATechnical Operations Manual4 (TOM). It is important to understand that the correctionslisted in this note are those known to be a solution to a definite problem when running SAP HANAappliance on the System x solutions. This knowledge was derived from internal testing, or customers whoran into a specific problem. In parallel, the organizations owning the individual products provide a lotmore fixes that are unknown to the Lenovo-SAP team, yet are recommend to be applied, nevertheless. Inparticular, there are fixes that IBM/Lenovo recommends to install that are not listed here. It is expected

3IBM General Parallel File System4http://help.sap.com/hana/SAP_HANA_Technical_Operations_Manual_en.pdf



https://service.sap.com/sap/support/notes/1599888http://www.ibm.com/support/fixcentralhttp://www.ibm.com/support/fixcentral/swg/selectFixes?parent=Cluster+software&product=ibm/power/IBM+General+Parallel+File+System&release=3.5.0&platform=Linux+64-bit,x86_64&function=allhttp://www.ibm.com/support/fixcentral/swg/selectFixes?parent=Cluster+software&product=ibm/power/IBM+General+Parallel+File+System&release=3.5.0&platform=Linux+64-bit,x86_64&function=allhttp://www.ibm.com/support/fixcentral/swg/selectFixes?parent=Cluster+software&product=ibm/power/IBM+General+Parallel+File+System&release=4.1.0&platform=Linux+64-bit,x86_64&function=allhttp://www-933.ibm.com/support/fixcentral/swg/selectFixes?parent=Software%2Bdefined%2Bstorage&product=ibm/StorageSoftware/IBM+Spectrum+Scale&release=4.1.1&platform=Linux+64-bit,x86_64&function=allhttp://download.novell.com/Download?buildid=XL0RqEykZpc~http://download.novell.com/Download?buildid=XL0RqEykZpc~http://download.novell.com/patch/finder/http://download.novell.com/patch/finder/http://www.redhat.com/en/technologies/linux-platforms/enterprise-linuxhttp://www.redhat.com/en/technologies/linux-platforms/enterprise-linuxhttp://www.vmware.com/support/http://service.sap.com/swdchttp://service.sap.com/swdchttp://help.sap.com/hana/SAP_HANA_Technical_Operations_Manual_en.pdfhttp://help.sap.com/hana/SAP_HANA_Technical_Operations_Manual_en.pdfhttp://help.sap.com/hana/SAP_HANA_Technical_Operations_Manual_en.pdfhttp://help.sap.com/hana/SAP_HANA_Technical_Operations_Manual_en.pdfhttp://service.sap.com/swdchttp://service.sap.com/swdchttp://www.vmware.com/support/http://www.redhat.com/en/technologies/linux-platforms/enterprise-linuxhttp://www.redhat.com/en/technologies/linux-platforms/enterprise-linuxhttp://download.novell.com/patch/finder/http://download.novell.com/patch/finder/http://download.novell.com/Download?buildid=XL0RqEykZpc~http://download.novell.com/Download?buildid=XL0RqEykZpc~http://www-933.ibm.com/support/fixcentral/swg/selectFixes?parent=Software%2Bdefined%2Bstorage&product=ibm/StorageSoftware/IBM+Spectrum+Scale&release=4.1.1&platform=Linux+64-bit,x86_64&function=allhttp://www.ibm.com/support/fixcentral/swg/selectFixes?parent=Cluster+software&product=ibm/power/IBM+General+Parallel+File+System&release=4.1.0&platform=Linux+64-bit,x86_64&function=allhttp://www.ibm.com/support/fixcentral/swg/selectFixes?parent=Cluster+software&product=ibm/power/IBM+General+Parallel+File+System&release=3.5.0&platform=Linux+64-bit,x86_64&function=allhttp://www.ibm.com/support/fixcentralhttps://service.sap.com/sap/support/notes/1599888


20/244


that you contact your IBM/Lenovo service contact to get a list of those fixes as well as a reasonablycurrent service level in general.





21/244


2 Introduction

2.1 Purpose

This document is intended to provide a single point of reference for techniques and product behaviorswhen dealing withSAP HANA.

2.2 Applicability

The techniques and product behaviours outlined in this document apply to:

SAP HANA appliance Platform Edition v1.0

SLES for SAP5 11 SP3

RHEL6 6.5 and 6.6

IBM GPFS3.5 and 4.1

Lenovo Systems solution for SAP HANA appliance based on the:

System x3850/x3950 X6 Workload Optimized Server

2.2.1 SAP HANA Platform Edition Versions

In this document, we reference to several different versions of the Lenovo Solution guided installationsoftware. The following numbering refers to the corresponding SAP HANA Platform Edition version.

1.7.x SAP HANA Platform Edition v 1.0 SPS07 - First release on IBM/Lenovo Systems X6 hardware

1.8.x SAP HANA Platform Edition v 1.0 SPS081.9.x SAP HANA Platform Edition v 1.0 SPS09

2.3 Exclusions and Exceptions

The techniques and product behaviours outlined in this document may not be applicable to future releases.

2.4 Conventions

This guide uses several conventions to improve the readers experience and the ease of understanding.

2.4.1 Icons Used

The following information boxes indicate important information you should follow according to the levelof importance.

Attention

ATTENTION pay close attention to the instructions given

5SUSE Linux Enterprise Server for SAP Applications6Red Hat Enterprise Linux





22/244


Warning

WARNING this is something to take into consideration

NoteINFORMATION extra information describing in detail

2.4.2 Code Snippets

When reading code snippets you have to note the following: Lines of code that are too long to be shownin one line will be automatically broken. This line break is indicated by an arrow at the end of the firstand an arrow at the start of the second line:

1 This is a code snippet that is too long to be printed in one single line, therefore

you will see an automatic line break.

There are also line numbers at the left side of each code snippet to improve the readability.

Code examples that contain commands that have to be executed on a command line follow these rules:

Lines beginning with a # indicate commands to be executed by the rootuser.

Lines beginning with a $ indicate commands to be executed by an arbitraryuser.

3 Solution Overview

This document provides general information specific to the Lenovo Solution. This document assumesthat the reader understands the basic structure and components of the SAP HANA Platform Edition.

SAP HANAshould be installed on hardware that has been specifically certified forSAP HANAby SAP.This hardware may not be configured from individual parts, rather it is to be ordered and delivered as asingle unit using an Lenovo manufacturer type/model number specified later.

3.1 The SAP HANA Appliance Software

The Lenovo Solution is based on building blocks that provide a highly scalable infrastructure forSAP HANAbased on the System x architecture: x3850/x3950 X6 as well as software, such as IBM GPFS, that willbe used to runSAP HANA.

Lenovo has created several system models upon which you may install and runSAP HANAaccording tothe sizing charts coordinated with SAP. For each workload type a special System x type/model has been

approved by SAP and Lenovo to accommodate the requirements for the SAP HANA Platform Edition.

3.2 Definition of SAP HANA

The following picture defines the currentSAP HANAscenarios that can be leveraged through the Systemx solution for the SAP HANA Platform Edition.





23/244


Figure 1: Current SAP HANA Appliance Scenarios





24/244


4 Hardware Configurations

The System X6 Workload Optimized servers for SAP HANA are based upon two building blocks that

can be used to fulfill the hardware requirements for SAP HANA. TheSAP HANA appliance softwaremust be installed only on a certified and tested hardware configuration based on one of these two models.Lenovo provides a model/type number for four (4) socket and eight (8) socket systems that are to besetup for each certified model by SAP. A customer needs only to choose the model and the extra optionsto fulfill their requirements. Models created manually will neither be supported by Lenovo nor SAP dueto the high-performance criteria set out by SAP during certification.

(a) System x3850 X6 (b) System Storage EXP2524 (c) System x3950 X6

Figure 2: Hardware Overview

System x3850 X6 Wo

Documents

Implementation Guide X6 1.9.96 13