Upload
others
View
21
Download
0
Embed Size (px)
Citation preview
VMware Virtualized SAP HANA with Dell EMC Storage
Storage configuration and connectivity
Scale-up and scale-out deployments November 2018
H14721.4
Deployment Guide
Abstract
This deployment guide describes best practices for deploying an SAP HANA system on Dell EMC storage arrays in VMware vSphere virtualized environments.
Dell EMC Solutions
Copyright
2 VMware Virtualized SAP HANA with Dell EMC Storage Storage configuration and connectivity; Scale-up and scale-out deployments Deployment Guide
The information in this publication is provided as is. Dell Inc. makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose.
Use, copying, and distribution of any software described in this publication requires an applicable software license.
Copyright © 2018 Dell Inc. or its subsidiaries. All Rights Reserved. Dell Technologies, Dell, EMC, Dell EMC and other trademarks are trademarks of Dell Inc. or its subsidiaries. Intel, the Intel logo, the Intel Inside logo and Xeon are trademarks of Intel Corporation in the U.S. and/or other countries. Other trademarks may be trademarks of their respective owners. Published in the USA 11/18 Deployment Guide H14721.4.
Dell Inc. believes the information in this document is accurate as of its publication date. The information is subject to change without notice.
Contents
3 VMware Virtualized SAP HANA with Dell EMC Storage Storage configuration and connectivity; Scale-up and scale-out deployments
Deployment Guide
Contents
Executive summary ....................................................................................................................... 4
Deploying VMware vSphere virtualized SAP HANA on Dell EMC storage ................................ 6
SAP HANA storage devices in virtualized environments ......................................................... 11
Connecting an ESXi host to Dell EMC storage arrays .............................................................. 17
Creating data and log devices on vSphere datastores ............................................................. 19
Setting up an SAP HANA virtual machine in vSphere .............................................................. 21
References ................................................................................................................................... 24
Executive summary
4 VMware Virtualized SAP HANA with Dell EMC Storage Storage configuration and connectivity; Scale-up and scale-out deployments Deployment Guide
Executive summary
Since the initial release of the SAP HANA in-memory, real-time platform as a physical
appliance, updates to the platform have provided more flexible deployment and scalability
for customers while reducing the total cost of ownership (TCO). The enhancements
include:
SAP HANA scale-outBreaks the RAM barrier of a single server by functioning as
a multinode cluster.
SAP HANA tailored datacenter integration (TDI)―Broadens the appliance model
toward shared enterprise storage arrays.
Virtualized SAP HANAEnables the installation of SAP HANA single-node (scale-
up) and multinode (scale-out) clusters in VMware virtualized environments. For
more information, see SAP Note 1788665: SAP HANA Support for
virtualized/partitioned (multitenant) environments (access requires an SAP
registration ID).
While these updates make the SAP HANA deployment easier and help customers with
the integration into their existing data center, organizations must still meet strict SAP
HANA requirements relating to their compute, network, and storage environments.
This VMware Virtualized SAP HANA with Dell EMC Storage solution offers SAP
customers a data center platform that provides agility, high availability (HA), cost savings,
and easy provisioning. The solution enables SAP customers to provision SAP HANA
instances more quickly in virtual machines.
Using the SAP HANA platform with the VMware vSphere virtualization infrastructure
provides the best environment for achieving a unique, cost-effective solution. This solution
is enabled by VMware vSphere vMotion, VMware vSphere Distributed Resource
Scheduler (DRS), and VMware vSphere High Availability, ensuring the best operational
performance and availability.
VMware provides best practices for deploying SAP HANA scale-up and scale-out systems
under vSphere. This deployment guide can be used in vSphere virtualized environments
as a supplement to the VMware documents and other Dell EMC documents that describe
storage configuration best practices for SAP HANA in physical TDI environments. See
References on page 24 for the titles of these documents.
VMware virtualized SAP HANA scale-up and scale-out systems can be deployed in TDI
environments on certified Dell EMC storage arrays. When running an SAP HANA
production system, the environment must meet SAP hardware and performance
requirements. You can use the SAP HANA hardware configuration check tool (hwcct) to
validate that the environment is correctly configured and that the storage performance
meets the SAP key performance indicators (KPIs). For more information, see SAP Note
1943937: Hardware Configuration Check Tool - Central Note (access requires an SAP
registration ID).
Note: The SAP performance requirements apply to production systems only.
Business
challenge
Solution
overview
Executive summary
5 VMware Virtualized SAP HANA with Dell EMC Storage Storage configuration and connectivity; Scale-up and scale-out deployments
Deployment Guide
This guide describes best practices for implementing SAP HANA in vSphere virtualized
environments when using the following Dell EMC storage systems in TDI deployments:
VNX series unified and block storage arrays
VMAX, VMAX3, and VMAX All Flash
PowerMax
XtremIO
VPLEX
SC Series
Unity
This guide describes vSphere-specific storage connectivity and configuration procedures
to follow when connecting vSphere virtualized SAP HANA systems on ESXi hosts with
certified Dell EMC enterprise storage arrays.
Use this guide with the following VMware guide for virtualized SAP HANA scale-up and
scale-out deployments: Architecture Guidelines and Best Practices for Deployments of
SAP HANA on VMware vSphere.
This deployment guide is intended for administrators who are responsible for
implementing, maintaining, and protecting robust and virtualized SAP HANA in-memory
databases and storage systems on vSphere. It assumes that readers have some
familiarity with SAP HANA in-memory databases, Dell EMC storage hardware and
software, and vSphere virtual infrastructures.
Table 1 provides explanations for some abbreviations used in this deployment guide.
Table 1. Terminology
Term Explanation
RDM Raw Device Mapping
RHEL Red Hat Enterprise Linux
SLES SUSE Linux Enterprise Service
VMDK Virtual Machine Disk
VMFS Virtual Machine File System
Dell EMC and the authors of this document welcome your feedback on the solution and the
solution documentation. Contact the Dell EMC Solutions Team with your comments.
Authors: Werner Katzenberger, Aighne Kearney
Document
purpose
Audience
Terminology
We value your
feedback
Deploying VMware vSphere virtualized SAP HANA on Dell EMC storage
6 VMware Virtualized SAP HANA with Dell EMC Storage Storage configuration and connectivity; Scale-up and scale-out deployments Deployment Guide
Deploying VMware vSphere virtualized SAP HANA on Dell EMC storage
Following the enhancement of SAP HANA deployment options with the TDI model, Dell
EMC certified the VMAX, VMAX3, VMAX All Flash, PowerMax, VNX, XtremIO, Unity,
VPLEX, and SC Series block storage platforms for production use in TDI deployments.
Tests with SAP HANA-certified Dell EMC block storage systems confirmed that these
arrays can be used in VMware virtualized environments with only a minor impact on
performance when you follow the guidelines provided n this deployment guide.
As in physical environments, you can install an SAP HANA system in virtualized
environments as a single-node (scale-up) system or as a multinode (scale-out) system.
The number of SAP HANA virtual machines that you can install on a single physical
server depends on the number of CPU sockets, the server RAM, the SAP HANA
installation use case (production or non-production), and the maximum scalability
supported by SAP and VMware, among other things.
Consult the relevant VMware best practices and recommendations when you create an
SAP HANA virtual machine and need to size and configure memory, CPU,
hyperthreading, and non-uniform memory access (NUMA) nodes for the machine.
Scale-up systems
Scale-up systems are primarily used for transactional SAP applications such as the SAP
Business Suite. The SAP HANA database is stored in the RAM of a single virtual
machine, while the SAP HANA persistence is stored on a data and log file system
belonging to this virtual machine.
Figure 1 shows the virtual disk devices used in SAP HANA virtual machines. The virtual
disk that holds the operating system is created during the initial machine configuration and
can reside on any available vSphere data store. Installing the SAP HANA software
requires a /hana/shared mount point. In scale-up installations, this mount point is
private to the virtual machine and can be a directory under the operating system root
device (if the size meets the requirements), or a separate virtual disk mounted under
/hana/shared. This virtual disk does not have specific performance requirements and
can reside on any multipurpose data store.
Overview
SAP HANA
deployment
options
Deploying VMware vSphere virtualized SAP HANA on Dell EMC storage
7 VMware Virtualized SAP HANA with Dell EMC Storage Storage configuration and connectivity; Scale-up and scale-out deployments
Deployment Guide
Figure 1. Virtual disks in an SAP HANA scale-up scenario
Scale-out systems
Scale-out systems are used primarily for analytic applications such as SAP Business
Warehouse. The SAP HANA database is distributed across multiple virtual machines,
either on a single physical server or across a vSphere cluster consisting of multiple
physical servers. The system uses a “shared-nothing cluster architecture,” in which every
node in a scale-out deployment stores its database persistence on its own data and log
file systems. In a scale-out deployment, the /hana/shared file system must be shared
across all SAP HANA nodes that are part of the scale-out cluster. For configuration
guidelines, see SAP HANA shared file system on page 11.
High availability in a vSphere cluster
In a vSphere cluster, you must enable the vSphere High Availability (HA) feature. HA handles the restart of virtual machines on another host that has enough free resources if one of the hosts in the cluster fails. To ensure that SAP HANA starts automatically with the virtual machine, enable the SAP HANA service autostart feature, either at the time of
the SAP HANA installation or by setting the autostart option to 1 in the following file:
/hana/shared/<SID>/profile/<SID>_HDB<InstNo>_<hostname>
Figure 2 shows the minimum required vSphere HA settings.
High availability
considerations
Deploying VMware vSphere virtualized SAP HANA on Dell EMC storage
8 VMware Virtualized SAP HANA with Dell EMC Storage Storage configuration and connectivity; Scale-up and scale-out deployments Deployment Guide
Figure 2. vSphere HA Settings
Using vSphere host groups, VM groups, and affinity rules, the administrator can control which hosts in the vSphere cluster are allowed to run SAP HANA virtual machines.
HA for SAP HANA virtual machines
The SAP HANA service auto-restart watchdog function monitors the SAP HANA
application and the associated services within a virtual machine. The watchdog function
automatically detects a failure and restarts the corresponding SAP HANA process
(nameserver, indexserver, and so on).
The VMware HA monitoring feature (Guest not heartbeating) handles operating system
(OS) failures. The monitoring feature restarts the guest OS of the virtual machine as well
as SAP HANA on the same host, provided the SAP HANA autostart option is enabled.
Enable the monitoring feature when you enable vSphere HA, as shown in Figure 2.
Dell EMC recommends setting Heartbeat monitoring sensitivity to High, as shown in
Figure 3.
Deploying VMware vSphere virtualized SAP HANA on Dell EMC storage
9 VMware Virtualized SAP HANA with Dell EMC Storage Storage configuration and connectivity; Scale-up and scale-out deployments
Deployment Guide
Figure 3. Enabling VMware HA VM Monitoring
To enable heartbeat monitoring, install and run VMware Tools in the virtual machine.
VMware Tools are installed either as part of the OS (open-vm-tools) or by using the
vSphere Web client and the context menu. Select Guest OS > Install VMware Tools.
In addition to heartbeat polling, VM Monitoring monitors the virtual machine’s I/O activity.
When heartbeats are not received and no disk or network activity has occurred over the
last 120 seconds, the virtual machine is reset by default. The administrator can change
the advanced setting das.iostatsInterval to modify this 120-second interval. Dell
EMC recommends aligning das.iostatsInterval with the failure interval you
selected in the vSphere Web client’s vSphere HA VM Monitoring section.
SAP HANA high availability in scale-out installations
In an SAP HANA scale-out installation on physical servers, you can deploy a standby
server to enable the SAP HANA host auto-failover functionality. If one of the active SAP
HANA servers (workers) fails, the SAP HANA nameserver triggers a failover to the
standby host.
In VMware virtualized environments, an SAP HANA standby host is not required and does
not work with the default SAP HANA storage connector. Instead, the VMware HA feature
restarts the SAP HANA virtual machines on another host in the cluster when a physical
host fails or on the same host if just the virtual machine operating system fails.
SAP HANA split-brain condition and host isolation
In physical SAP HANA scale-out installations with shared storage, the first installed node
becomes the master nameserver, Master 1, and the second and the third installed nodes
become master candidates Master 2 and Master 3. SAP HANA experiences a split-brain
situation if multiple hosts try to become the master nameserver or indexserver and to
access the same data (persistence) from disk. Such a situation might occur if the host of
the master nameserver becomes isolated from the network but can still access the shared
disk storage and a master candidate tries to take over the master role. Physical SAP
HANA installations use techniques such as SCSI-3 persistent reservation to prevent this
situation.
Deploying VMware vSphere virtualized SAP HANA on Dell EMC storage
10 VMware Virtualized SAP HANA with Dell EMC Storage Storage configuration and connectivity; Scale-up and scale-out deployments Deployment Guide
The standby master nameserver candidates are not required in VMware virtualized
environments. VMware HA restarts the existing master nameserver quickly enough that
promoting a new node to the master role is not necessary. To prevent SAP HANA from
using Master 2 and Master 3 candidates in scale-out installations, set the following entries
in the global.ini file:
[communication]
listeninterface = .global
[persistence]
basepath_shared = no
When installing SAP HANA using hdbinst, add these parameters to the
/usr/sap/<SID>/SYS/global/hdb/custom/config/global.ini file after the first
node is installed.
When using hdblcm for a SAP HANA scale-out installation, create a default
global.ini file with the above entries in a temporary directory. The --custom_cfg
installation parameter must point to this temporary directory.
In existing SAP HANA installations that already have multiple master candidates (Master
2 and Master 3), disable the failover mechanism by removing the nameserver roles
Master 2 and Master 3. Follow these steps:
1. Select SAP HANA Studio > Administration-Perspective > Landscape >
Hosts.
2. Select Configure Hosts for Failover Situation and change the master role to
Slave.
If a network failure occurs whereby a physical ESXi host becomes disconnected from the
network but all virtual machines on the host are still running, the VMware HA Host
Isolation rule and the associated response are applied. The default response leaves all
virtual machines powered on, as shown in Figure 2. Because a virtualized SAP HANA
scale-out cluster does not have a standby node to take over the function (master
nameserver) of the unresponsive node on the isolated host, a split-brain situation cannot
occur. The default response allows the administrator to investigate and fix the network
issue, or to power off the isolated host, which then causes VMware HA to restart the
virtual machines on another host.
You can also set the host isolation response to Power off and restart VMs. The host is
powered off and the virtual machines are restarted automatically on another host that still
has network connectivity.
With only one SAP HANA master nameserver candidate and the VMware HA Host
Isolation rules, a master nameserver can exist only once and therefore a split-brain
condition cannot occur.
SAP HANA storage devices in virtualized environments
11 VMware Virtualized SAP HANA with Dell EMC Storage Storage configuration and connectivity; Scale-up and scale-out deployments
Deployment Guide
SAP HANA storage devices in virtualized environments
SAP HANA installations on VMware vSphere require the following storage devices:
OS disk (root device) ‒ approximately 10 GB
The disk is created when the virtual machine is deployed. The virtual disk file
(VMDK) can reside on any data store that meets the installation requirements, that
is, has vMotion support.
SAP home directory: /usr/sap ‒ approximately 50 GB
The /usr/sap directory can be a mount point under the OS root directory. Adjust
the size of the root device correspondingly. Alternatively, you can use and mount a
separate virtual disk under /usr/sap/<SID>. Dell EMC recommends this method
for storage-based cloning and when using SAP instance cloning tools such as SAP
Landscape Management (LaMa).
SAP HANA shared file system: /hana/shared
The /hana/shared file system stores the SAP HANA installation binaries,
configuration files, and trace files. The size of the file system depends on the size of
the SAP HANA installation―that is, database size and number of nodes. For
guidelines about capacity sizing, see SAP HANA Storage Requirements. The SAP
HANA shared file system section on page 11 of this guide provides configuration
details.
SAP HANA database persistence (data and log)
Each active SAP HANA node, be it a scale-up node or worker node in scale-out
deployments, requires two devices to persist the in-memory database to disk and to
write the transaction log. Because the SAP performance requirements for the
storage devices apply only to these two file systems, it is important to consider
storage configuration best practices when you set up an SAP HANA production
environment. The size of the data and log devices depends on the actual database
size. For more information, see SAP HANA Storage Requirements.
SAP HANA scale-out scenarios require a shared file system that is mounted on every
SAP HANA node within the cluster. The file system stores the installation binaries,
configuration files, and trace files. In physical environments, this file system is generally
provided as a network file system (NFS) share by storage systems with network-attached
storage (NAS) capabilities, such as VNX series unified, Unity, SC Series storage, and
VMAX or PowerMax with eNAS.
If a NAS array is not available, vSphere with native Linux functionality offers an
alternative. A Linux virtual machine (non-SAP HANA node) running an NFS server
process provides the NFS share. This process exports a file system that is mounted on all
the SAP HANA cluster nodes. Reliability is achieved by using vSphere Fault Tolerance.
Overview
SAP HANA
shared file
system
SAP HANA storage devices in virtualized environments
12 VMware Virtualized SAP HANA with Dell EMC Storage Storage configuration and connectivity; Scale-up and scale-out deployments Deployment Guide
Figure 4. A /hana/shared file system using Linux NFS and vSphere fault tolerance
Setting up the NFS server
Follow these steps:
1. Create a Linux virtual machine (SLES or RHEL) and install the NFS server
package. See the operating system documentation for instructions. The Linux
virtual machine is not required to meet the high CPU and memory requirements of
an SAP HANA node.
2. Add a new VMDK to the virtual machine. The VMDK can reside on any vSphere
datastore. The size of the disk must be the required size of the shared file system
for the SAP HANA cluster.
3. Format the disk using the XFS file system.
4. Create a mount point in the format /hana_<sid>, replacing <sid> with the
system ID of the SAP HANA cluster that is using the export as the shared file
system. Then mount the disk under the new mount point.
5. Open the /etc/exports file. For each SAP HANA node (NFS client) in the
cluster, add the following line:
/hana_<sid> <fully qualified file server client host
name>(rw,sync,no_root_squash,no_subtree_check)
6. Restart the NFS server process to activate the changes
Note: The virtual machine running the NFS server can easily provide multiple SAP HANA
share exports for different SAP HANA installations.
Setting up the NFS client on the SAP HANA nodes
To set up the NFS client:
1. Ensure that the Linux NFS client is installed and running on each SAP HANA
node.
2. Open the /etc/fstab file in each SAP HANA node and add the following line:
<fully qualified file server host name>:/hana_<sid>
/hana/shared nfs hard,intr,nolock,proto=tcp 0 0
SAP HANA storage devices in virtualized environments
13 VMware Virtualized SAP HANA with Dell EMC Storage Storage configuration and connectivity; Scale-up and scale-out deployments
Deployment Guide
3. Create the local mount point on each SAP HANA node by using the following
command:
mkdir -p /hana/shared
4. Mount the shared file system on each node using the following command:
mount –a
Using fault tolerance for the NFS server virtual machine in vSphere
Because the SAP HANA nodes depend on the shared file system, Dell EMC recommends
that you turn on the fault tolerance function for the NFS server VM. To do this, go to the
vSphere Web client, highlight the NFS Server virtual machine, and select and enable
Fault Tolerance in the context menu.
Note: The fault-tolerance feature may require additional licenses in your vSphere environment.
Also, it presents some limitations on the number of virtual CPUs per protected virtual machine.
vSphere fault tolerance provides continuous availability for the SAP HANA share during
an NFS server failure. vSphere creates a live-shadow virtual machine instance that is
always up to date with the primary virtual machine. If a hardware outage occurs, vSphere
automatically triggers failover, ensuring zero downtime for the SAP HANA share and
preventing data loss. After failover, vSphere creates a secondary virtual machine to
deliver continuous protection for the SAP HANA share.
SAP HANA persists the in-memory database to the /hana/data file system at regular
intervals and writes the transaction log to the /hana/log file system. These two file
systems are the SAP HANA database persistence. The storage performance
requirements for SAP HANA TDI certification apply to the SAP HANA persistence for
production systems only. Dedicate the underlying storage device (RDM or a VMware
VMFS data store) to the persistence and then configure it using the best practices for SAP
HANA on that storage array. For more information, see the relevant document in
References on page 24 of this deployment guide.
SAP HANA production on vSphere
In production environments, a virtual SAP HANA node requires the same storage
performance and bandwidth as a physical node. The configuration of the LUNs and the
underlying pools or RAID groups must follow the same rules. Because the additional
virtualization layer affects the bandwidth and latency, the scalability of a particular Dell
EMC array for virtualized SAP HANA nodes might be lower than the number of SAP
HANA nodes supported in a physical environment.
For non-production virtualized SAP HANA environments, the storage performance does
not have to meet the SAP KPIs. Therefore, you have more flexibility when you configure
the virtual environment: RAID protection, disk types, pools, RAID groups, and so on. Dell
EMC recommends monitoring the storage performance and making adjustments if the
performance does not meet your requirements.
SAP HANA
database
persistence
SAP HANA storage devices in virtualized environments
14 VMware Virtualized SAP HANA with Dell EMC Storage Storage configuration and connectivity; Scale-up and scale-out deployments Deployment Guide
Data stores and raw device mappings
When creating LUNs for virtualized SAP HANA deployments in production environments,
follow the storage configuration best practices for physical SAP HANA environments for
the corresponding SAP TDI-certified Dell EMC storage array. See Dell EMC
documentation on page 24 for links to these documents.
When Dell EMC block storage devices (LUNs) are connected to vSphere ESXi hosts, the
SAP HANA virtual machines can use them for the database persistence, either as data
stores formatted using a VMFS cluster or as RDMs. VMFS data stores are preferred
because they are easier to manage and provide more data growth flexibility.
This guide does not address the use of VMware virtual volumes (vvols).
Using VMFS datastores
Creating a VMFS data store on a block storage LUN is the preferred configuration option
for SAP HANA virtual machines. Creating dedicated and isolated datastores for SAP
HANA data and log file systems is a VMware best practice, which means the datastores
must be created on different LUNs in the storage array. Create all data devices (VMDKs)
of an SAP HANA system on one datastore, and all log devices on a second datastore.
When running multiple SAP HANA databases in a vSphere cluster, you can achieve
further workload isolation with dedicated data and log datastores for each SAP HANA
database system ID SID.
Each SAP HANA worker virtual machine has a data and a log file system residing on
VMDK files created on the corresponding VMFS datastore.
Figure 5 shows an example of an SAP HANA three-node scale-out deployment on a
vSphere cluster using VMFS datastores.
SAP HANA storage devices in virtualized environments
15 VMware Virtualized SAP HANA with Dell EMC Storage Storage configuration and connectivity; Scale-up and scale-out deployments
Deployment Guide
Figure 5. Deploying SAP HANA persistence on VMFS datastores
Using raw device mappings
RDMs are an alternative option for the SAP HANA database persistence. With RDMs,
every data and log file system resides on a dedicated LUN in the storage array.
RDMs enable you to configure the LUNs for virtualized SAP HANA nodes the same way
as for physical environments. RDMs also enable you to use the LUNs for physical SAP
HANA installations when the nodes are migrated to and from a virtualized environment.
Because the high number of LUNs might become difficult to manage for larger SAP HANA
deployments, consider VMFS datastores instead.
Figure 6 shows an example of an SAP HANA 3+1 scale-out deployment on RDMs.
SAP HANA storage devices in virtualized environments
16 VMware Virtualized SAP HANA with Dell EMC Storage Storage configuration and connectivity; Scale-up and scale-out deployments Deployment Guide
Figure 6. SAP HANA persistence deployed on RDMs
Sizing storage LUNs for SAP HANA persistence
This section provides guidelines for correctly sizing the LUNs used as RDMs or the
storage LUNs used for the vSphere VMFS datastores. See SAP HANA Storage
Requirements for the capacity sizing for SAP HANA devices. Because the requirements
have been updated several times, use the most recent version of the guide when sizing a
new SAP HANA installation.
The examples in this guide use the following capacity sizing:
SAP HANA data file system: 1.2x net data of the SAP HANA node
SAP HANA log file system: 0.5x RAM of the SAP HANA node (maximum 512 GB)
Sizing storage LUNs for VMFS datastores
The LUNs for the VMFS datastores contain the SAP HANA persistence. When sizing the
LUNS, provide the following information:
Number of virtualized SAP HANA nodes, both scale-up and scale-out
RAM size of the SAP HANA nodes
Figure 5 shows a three-node SAP HANA scale-out cluster. Assuming each node has 128
GB RAM and holds approximately 50 GB of data, the capacity calculation is:
Data: 50 GB x 1.2 x 3 nodes = 180 GB
Log: 128 GB x 0.5 x 3 nodes = 194 GB
Connecting an ESXi host to Dell EMC storage arrays
17 VMware Virtualized SAP HANA with Dell EMC Storage Storage configuration and connectivity; Scale-up and scale-out deployments
Deployment Guide
All VMFS datastores must retain a level of free space to ensure that virtual machines
function correctly. Dell EMC recommends adding 25 percent free space to the previous
calculation. The size of the LUNs should be:
Data: 225 GB LUN size
Log: 242 GB LUN size
These are the minimum capacity requirements for the SAP HANA persistence. They do
not take account of future growth. With virtualized SAP HANA, you can increase the RAM
capacity of the SAP HANA nodes, requiring larger file systems for the database
persistence and therefore more capacity in the VMFS datastores. It is more efficient to
create larger LUNs at the beginning to prepare for future growth. It is also better to avoid
adding another LUN to a VMFS datastore because performance does not scale when
LUNs are concatenated.
Unlike physical environments, the underlying disk configuration of virtual environments,
that is, RAID groups, thin pools, and so on, does not change. You must still adjust the size
and configuration of the LUNs to meet the capacity requirements for datastores.
Sizing storage LUNs for RDMs
When using RDMs for the SAP HANA persistence, you can size the storage LUNs in
accordance with the SAP minimum requirements described in SAP HANA Storage
Requirements.
For the example shown in Figure 6, we created the following LUNs:
Three data LUNs: 50 GB x 1.2 = 60 GB for each LUN
Three log LUNs: 128 GB x 0.5 = 64 GB for each LUN
Connecting an ESXi host to Dell EMC storage arrays
To connect Dell EMC storage to VMware ESXi servers, follow the best practices
described in the EMC Host Connectivity Guide for VMware ESX Server. When running
virtualized SAP HANA systems on vSphere, consider the additional connectivity options
that this section of the deployment guide describes.
Host bus adapters
For redundancy reasons, use at least two Fibre Channel (FC) host bus adapters (HBAs)
connected to the storage array over two switches. If you deploy multiple virtualized SAP
HANA production nodes on a single ESXi host, the storage bandwidth and latency must
fulfill the SAP TDI KPIs for each SAP HANA system1. In TDI deployments, a single SAP
HANA production node requires 400 MB/s maximum bandwidth. With four SAP HANA
production nodes on a single physical ESXi host, the combined bandwidth of all HBAs on
the host must be at least 4 x 400 MB/s (1600 MB/s)—for example, a single 8 Gb/s FC
HBA provides approximately 750 MB/s bandwidth.
1 See the relevant SAP and VMWare documentation for information about how many SAP HANA
production systems can be deployed per physical host.
Overview
ESXi
considerations
Connecting an ESXi host to Dell EMC storage arrays
18 VMware Virtualized SAP HANA with Dell EMC Storage Storage configuration and connectivity; Scale-up and scale-out deployments Deployment Guide
ESXi HBA queue depth
Dell EMC recommends increasing the HBA queue depth on the ESXi host to 128:
1. Run the following command on the ESXi host:
esxcli system module parameters set -p
lpfc_lun_queue_depth=128 -m lpfc
2. To display the HBA queue depth setting, run the following command:
esxcli system module parameters list -m lpfc|grep
lpfc_lun_queue_depth
3. Restart the ESXi host to activate the changes.
vSphere Native Multipathing
If the ESXi host is using vSphere Native Multipathing (NMP), Dell EMC recommends the
round-robin policy for the multipath setting for VMAX, VMAX3, VMAX All Flash,
PowerMax, VNX, Unity, SC Series, and XtremIO storage arrays. For best performance,
set the multipath round-robin policy for the I/O operation limit to 1. For the SC Series, the
I/O operation limit must be set to 3. The current VMware default value for this setting is
1000.
To change the default policy setting from fixed to round-robin and the host I/O operation
limit to 1, follow these steps:
1. Open the vSphere CLI (recommended) or the service console.
2. Run the following command:
For VMAX, VMAX3, VMAX All Flash and PowerMax: esxcli storage nmp satp set --default-psp=VMW_PSP_RR --
satp=VMW_SATP_SYMM
For VNX2 and Unity using the ALUA protocol: esxcli storage nmp satp set --default-psp=VMW_PSP_RR --
satp=VMW_SATP_ALUA_CX
For XtremIO: esxcli storage nmp satp rule add -c tpgs_off -e "XtremIO
Active/Active" –M XtremApp -P VMW_PSP_RR -O iops=1 -s
VMW_SATP_DEFAULT_AA –t vendor -V XtremIO
3. To set the host I/O operation limit to 1, run the following commands:
List all devices connected to the ESXi host:
# esxcli storage nmp device list
Set the I/O operation limit to 1 for all Dell EMC storage devices used for the
SAP HANA persistence (either as datastores or as RDMs):
esxcli storage nmp psp roundrobin deviceconfig set -I 1 -
t iops -d <naa.deviceid>
Verify the device settings:
Creating data and log devices on vSphere datastores
19 VMware Virtualized SAP HANA with Dell EMC Storage Storage configuration and connectivity; Scale-up and scale-out deployments
Deployment Guide
esxcli storage nmp psp roundrobin deviceconfig get -d
<naa.deviceid>
Dell EMC PowerPath/VE
Dell EMC PowerPath/VE for VMware vSphere can be used instead of or with vSphere
NMP. PowerPath/VE is a multipathing extensions module for vSphere, providing software
that works with SAN storage to intelligently manage FC, iSCSI, and FC over Ethernet
(FCoE) I/O paths. PowerPath/VE is installed on the vSphere host and scales to the
maximum number of virtual machines on the host, thus improving I/O performance. The
virtual machines do not have PowerPath/VE installed and are not aware that
PowerPath/VE is managing I/O to storage. PowerPath/VE dynamically load-balances I/O
requests and automatically detects and recovers from path failures.
PowerPath/VE automatically assigns an optimized load-balancing policy for all devices
that it manages. Dell EMC recommends that you use the default optimized load-balancing
and failover policies that PowerPath/VE assigns to the devices based on the underlying
storage array type.
The number of front-end ports that are used for the ESXi hosts cluster depends on the
number of virtualized SAP HANA production nodes. Treat a virtualized SAP HANA node
like a physical SAP HANA node. For more information, see the storage configuration best
practices document for the corresponding Dell EMC storage array.
With ESXi hosts, use single-initiator zoning or single-initiator-single-target zoning. Single-
initiator-single-target zoning is preferred.
Note: Dell EMC does not support multi-initiator zones in a VMware ESX Server fabric
environment.
To perform zoning on the fabric, create zone sets that contain the initiator and the targets.
Creating data and log devices on vSphere datastores
VMDKs on VMFS datastores are the preferred storage for the SAP HANA persistence on
SAP HANA virtual machines. After you have correctly sized and created the storage
LUNs, as described in Sizing storage LUNs for SAP HANA persistence on page 16, and
formatted VMFS datastores on these LUNs, you can create data and log devices for the
SAP HANA virtual machines on the datastores.
Each SAP HANA node uses its own data and log device and has access only to its pair of
devices, as Figure 1 shows. For performance reasons, VMware recommends provisioning
the SAP HANA data and log devices as “thick provision eager zeroed.” VMware also
recommends assigning dedicated SCSI controllers for each data and log device. See
vSphere SCSI controller on page 20 for more information.
Front-end ports
for storage
arrays
Fabric zoning
Introduction
Scale-up and
scale-out
Creating data and log devices on vSphere datastores
20 VMware Virtualized SAP HANA with Dell EMC Storage Storage configuration and connectivity; Scale-up and scale-out deployments Deployment Guide
For information about creating data and log VMDKs, see Setting up an SAP HANA virtual
machine in vSphere on page 21.
VMware recommends using multiple vSphere SCSI controllers for each SAP HANA virtual
machine. Assign SCSI controller 0 to the operating system disk, SCSI controller 1 to the
SAP HANA data devices, and SCSI controller 2 to the SAP HANA log devices.
vSphere offers several different types of SCSI controller to connect disks to a virtual
machine:
LSI Logic Parallel
LSI Logic SAS
VMware Paravirtual
BusLogic Parallel (this does not work with Dell EMC storage)
For best performance, VMware recommends using one VMware Paravirtual SCSI
controller for all data devices and one for all log devices.
With the VMWare Paravirtual SCSI controller used for the SAP HANA persistence, set the
following kernel parameters for the virtual machine running SAP HANA:
elevator=noop
vmw_pvscsi.cmd_per_lun=254
vmw_pvscsi.ring_pages=32
For information about how to set these kernel parameters, see:
Red Hat knowledge base article 5427: Suggested I/O scheduler to improve disk
performance when using Red Hat Enterprise Linux with virtualization (applies also
to SUSE).
Novell Hat knowledge base article 7009616: Virtual SLES11SP1 shows bad IO
performance when accessing raw disks (applies also to SLES versions other than
those mentioned).
vSphere SCSI
controller
Prepare an SAP
HANA virtual
machine in Linux
Setting up an SAP HANA virtual machine in vSphere
21 VMware Virtualized SAP HANA with Dell EMC Storage Storage configuration and connectivity; Scale-up and scale-out deployments
Deployment Guide
Setting up an SAP HANA virtual machine in vSphere
This section describes the configuration steps that you must complete when creating an
SAP HANA virtual machine with data and log persistence on Dell EMC storage as one of
the following:
Single-node scale-up system
Multinode scale-out system without a standby host
When deploying a new SAP HANA virtual machine, ensure that the operating system
version is supported by SAP and that you follow SAP and VMware best practices when
configuring the memory and CPU of the virtual machine.
Operating system virtual disk
The virtual disk on which the operating system is installed can reside on any datastore
that meets the functional requirements, namely, support for vSphere vMotion. This device
does not have to meet the SAP HANA performance requirements.
vSphere compatibility mode
It is important to use the correct vSphere compatibility mode (virtual-machine version). We
tested and validated compatibility mode ESXi 5.5 and later versions, virtual-machine
version 10.
To configure the disk:
1. In the vSphere Web client, edit the settings in the SAP HANA virtual machine.
2. Add two additional VMware Paravirtual SCSI controllers to the virtual machine.
Set SCSI Bus Sharing to None.
The virtual machine should now have three SCSI controllers assigned:
SCSI controller 0Used for the system device
SCSI controller 1To be used for the SAP HANA data devices
SCSI controller 2To be used for the SAP HANA log devices
3. Add a new SAP HANA data or log disk.
The SAP HANA persistence can be a vSphere VMDK residing on a VMFS
datastore, or an RDM pointing directly to the underlying storage LUN.
VMDK—Add a new hard disk to the virtual machine on the desired datastore
using the thick provision eager zeroed option. Assign an SAP HANA data
device to SCSI controller 1, as shown in Figure 7, and an SAP HANA log
device to SCSI controller 2, as shown in Figure 8.
Overview
Deploying an SAP
HANA virtual
machine
Assigning the
persistence to
the SAP HANA
virtual machine
Setting up an SAP HANA virtual machine in vSphere
22 VMware Virtualized SAP HANA with Dell EMC Storage Storage configuration and connectivity; Scale-up and scale-out deployments Deployment Guide
Figure 7. Creating a new data hard disk
The virtual-machine data device is assigned to SCSI device 1:0
and the log device is assigned to SCSI device 2:0.
Figure 8. Creating a new hard disk log
Setting up an SAP HANA virtual machine in vSphere
23 VMware Virtualized SAP HANA with Dell EMC Storage Storage configuration and connectivity; Scale-up and scale-out deployments
Deployment Guide
RDM—Add the data and log RDM disks and assign them to SCSI controller 1
(data) and SCSI controller 2 (log), as shown in Figure 9.
Figure 9. Assigning an RDM to a virtual machine
Conclusion
Virtualizing SAP HANA scale-up and scale-out systems enables an organization to benefit
from all supported VMware virtualization solutions and options, such as HA and live
migration using vSphere vMotion, to increase service level agreements and lower the
TCO. We verified that virtualized SAP HANA installations on Dell EMC storage arrays
meet the SAP performance KPIs for SAP HANA production deployments when we
followed the storage configuration best practices and supplemental configuration
guidelines described in this document.
Summary
References
24 VMware Virtualized SAP HANA with Dell EMC Storage Storage configuration and connectivity; Scale-up and scale-out deployments Deployment Guide
References
The following documentation on EMC.com or EMC Online Support provides additional
relevant information. Access to these documents depends on your login credentials. If you
do not have access to a document, contact your Dell EMC representative.
Storage Configuration Best Practices for SAP HANA Tailored DatacenterIntegration on EMC VMAX and VMAX3 Storage Systems
Storage Configuration Best Practices for SAP HANA Tailored DatacenterIntegration on Dell EMC PowerMax Arrays
Storage Configuration Best Practices for SAP HANA Tailored DatacenterIntegration on EMC VNX Series Unified Storage Systems
Storage Configuration Best Practices for SAP HANA Tailored Data CenterIntegration on EMC Unity Storage Systems
Storage Configuration Best Practices for SAP HANA Tailored DatacenterIntegration on EMC XtremIO Storage
EMC Host Connectivity Guide for VMware ESX Server
Dell EMC SC Series Best Practices with VMware vSphere 5.x-6.x
Storage Configuration Best Practices for SAP HANA TDI on Dell EMC VPLEXSystems
Dell EMC SC Series Storage Configuration Best Practices for SAP HANA TDIValidation Guide
For SAP Ready Solution documentation, see Solutions for SAP Server Info Hub:
Technical Documentation in the Everything SAP at Dell EMC Community forum.
The following documentation on the VMware website provides additional relevant
information:
Architecture Guidelines and Best Practices for Deployments of SAP HANA onVMware vSphere
The following documentation on the SAP website provides additional relevant information:
SAP HANA Installation Guide
SAP HANA Server Installation and Update Guide
SAP HANA Administration Guide
SAP HANA Storage Requirements
SAP HANA Certified Hardware Directory – Enterprise Storage
SAP Note 1788665: SAP HANA Support for virtualized/partitioned (multitenant)environments
SAP Note 800326: Certified EMC solutions for Linux environments
SAP Note 1943937: Hardware Configuration Check Tool - Central Note
SAP Community WIKI: SAP HANA on VMware vSphere
Dell EMC
documentation
VMware
documentation
SAP
documentation