Technical white paper

Using HP Serviceguard for Linux with Red Hat KVM Guests

Table of contents Introduction .................................................................................................................................................................................... 3

KVM terminology used in this document ................................................................................................................................. 3

Introduction to KVM ...................................................................................................................................................................... 3

Red Hat KVM System requirements, limitations ................................................................................................................ 4

Creating and launching KVM host and guests .................................................................................................................... 4

Supported cluster configuration with KVM guests ................................................................................................................ 5

Cluster with KVM guests from a single host as cluster nodes ........................................................................................ 5

Cluster with KVM guests from multiple hosts as cluster nodes ..................................................................................... 5

Cluster with KVM guests and physical machines as cluster nodes ................................................................................ 6

Network topology for Serviceguard cluster using KVM guests ........................................................................................... 6

Host Network Configuration ................................................................................................................................................... 8

Guest Network Configuration ............................................................................................................................................... 10

Storage requirements for Serviceguard cluster on KVM guests ....................................................................................... 12

Restrictions and exclusion ......................................................................................................................................................... 12

Supported host and guest operating systems ..................................................................................................................... 12

Summary ....................................................................................................................................................................................... 13

References .................................................................................................................................................................................... 13

Technical white paper | Using HP Serviceguard for Linux with Red Hat KVM Guests

2

About this technical white paper Virtual machine technology is a powerful capability that can reduce costs and power usage while improving utilization of resources. HP is also applying virtualization to other aspects of the data center and uniting virtual and physical resources to create an environment suitable for deploying mission-critical applications.

HP Serviceguard for Linux is certified for deployment on Red Hat kernel-based virtual machine (KVM) guests. This white paper discusses the different ways in which a KVM guest can be integrated in a Serviceguard for Linux cluster. It describes how a Serviceguard for Linux cluster can be configured using KVM guests from a single host, KVM guests from multiple hosts and a combination of KVM guests and physical machines, so as to provide high availability for applications.

The white paper provides details on supported Red Hat hypervisor, guest OS, network topology, network device models and storage configurations. It also makes recommendations for eliminating single points of failure and provides pointers to other useful documents.

Note HP Serviceguard for Linux does not support KVM guests carved out of Red Hat Enterprise Virtualization Hypervisor (RHEV) as cluster nodes.

Technical white paper | Using HP Serviceguard for Linux with Red Hat KVM Guests

3

Introduction

Virtual machines are increasingly being deployed for server consolidation and flexibility. Virtual machine technology allows one physical server to simulate multiple servers, each concurrently running its own OS. In virtual machine technology, the virtualization layer also known as hypervisor1 abstracts the physical resources so that each instance of an OS appears to have its own NIC, processor, memory etc when in fact they are virtual instances. This allows you to replace a number of existing physical servers with virtual machines.

The Linux KVM is an offering from Red Hat Enterprise Linux (RHEL) for a complete virtualization solution. This document explains how you can use HP Serviceguard for Linux in RHEL KVM-based virtualization environments to provide mission-critical clustering and failover capabilities. It also makes recommendations for eliminating single point of failure (SPOF) that you can implement.

KVM terminology used in this document

Table 1. KVM terminology used in this document

Term Definition

KVM Kernel-based virtual machine

KVM Host, Host Physical server on which KVM Hypervisor is installed

KVM Guest, Guest, VM KVM virtual machine carved out of the hypervisor

Physical Machine Physical server configured as a Serviceguard cluster node

Bridge A device bound to a physical network interface on the host which enables any number of guests to connect to the local network on the host. It is mapped to a physical NIC which acts as a switch to KVM Guests

Cluster, Serviceguard Cluster HP Serviceguard for Linux cluster

Introduction to KVM

HP Serviceguard for Linux solutions can be used in KVM based virtualization environments to provide mission-critical clustering and failover capabilities. Linux KVM is provided by RHEL as a full virtualization solution. KVM differs from other popular alternatives like Xen and VMware in terms of operation, performance and flexibility. KVM comes as a kernel module, with a set of user space utilities to create and manage the virtual machines.

Kernel-based Virtual Machine technology includes the following:

• A full virtualization solution for Linux on AMD64 and Intel® 64 hardware.

• Each KVM virtualized guest or “VM guest” is run as a single Linux process.

• A hypervisor-independent virtualization API, “libvirt,” which provides common generic and stable layer to securely manage VM guests on a host.

• A command line tool “virsh” used to manage the VM guests.

• A graphical user interface (GUI) “virt-manager” for managing the VM guests.

1 Hypervisor often refers to a layer that resides directly on server hardware, but terms are not used consistently across the industry.

Technical white paper | Using HP Serviceguard for Linux with Red Hat KVM Guests

4

Red Hat KVM System requirements, limitations Supported architecture

• Intel 64—An Intel processor with the Intel VT-x and Intel 64 extensions for x86-based systems

• AMD64—An AMD processor with the AMD-V and the AMD64 extensions

Minimum system requirements

• 6 GB free disk space

• 2 GB of RAM

Recommended system requirements

• 6 GB plus the required disk space recommended by the guest operating system per guest. For most operating systems more than 6 GB of disk space is recommended

• One processor core or hyper-thread for each virtualized CPU and one for the host

• 2 GB of RAM plus additional RAM for virtualized guests

For more information on limitation, support limits and restrictions refer to Red Hat documentation:

https://access.redhat.com/site/documentation/Red_Hat_Enterprise_Linux/ -> Red Hat Enterprise Linux 6 Virtualization Host Configuration and Guest Installation Guide.

Creating and launching KVM host and guests KVM is available as part of Red Hat Enterprise Linux. The “virt-manager” tool provides a very simple, easy-to-use and intuitive GUI interface for all virtual machine operations. A command line alternative, “virsh,” also provides a shell that can be used to create and manage virtual machines using a rich set of commands. The features provided by these tools include creating virtual machines, taking snapshots of virtual machines, creating virtual networks etc.

Following is a high-level overview of the steps required for setting up KVM guests.

For detailed instructions on the steps mentioned below refer to the following document: https://access.redhat.com/site/documentation/Red_Hat_Enterprise_Linux/-> Virtualization Host Configuration and Guest Installation Guide.

1. Before creating KVM guests, ensure that the CPU, memory resources required by the KVM guests are available and if the virtualization support is enabled at the BIOS on all KVM hosts to be used in the cluster.

2. Make sure that the required KVM packages are installed on the hosts.

3. Make sure that the service libvirtd is running on the hosts where KVM guests are to be created.

4. Create the KVM guests and install the required operating system in the KVM guests. HP Serviceguard for Linux does not have any restriction on where the boot image will be stored. You can use "virt-manager" for creating and configuring the KVM guests. For list of supported guest OS refer too Supported host and guest operating systems.

5. Complete the network configuration as mentioned in the section Network Topology for Serviceguard cluster using KVM guests.

6. Complete the storage configuration as mentioned in the section Storage requirements for Serviceguard cluster on KVM guests.

7. Repeat the steps 4 to 6 for all KVM guests that need to be a part of cluster.

8. Install HP Serviceguard for Linux and its prerequisites on all the KVM guests that need to be in the cluster. For more information on installation of HP Serviceguard for Linux, refer HP Serviceguard for Linux Deployment Guide at hp.com/go/linux-serviceguard-docs.

9. Configure HP Serviceguard for Linux cluster and other required resources like packages, service etc. For more information on how to configure refer to Managing HP Serviceguard for Linux at hp.com/go/linux-serviceguard-docs.

Technical white paper | Using HP Serviceguard for Linux with Red Hat KVM Guests

5

Supported cluster configuration with KVM guests

The supported HP Serviceguard for Linux cluster configuration when using KVM guests as cluster nodes are as follows:

Cluster with KVM guests from a single host as cluster nodes In this configuration a cluster is formed with KVM guests all of which are carved out of a single host (cluster in a Box). Though this configuration provides consolidation of resources this is not ideal. A failure of the host will bring down all the nodes of the cluster which is a single point of failure. Hence this configuration is not recommended.

Figure 1. Cluster in a Box

Cluster with KVM guests from multiple hosts as cluster nodes In this configuration a cluster can be formed with KVM guests that are carved out of two or more hosts. Serviceguard is installed on the KVM guests and a cluster is formed with these VMs. Serviceguard provides high availably to only the applications running as packages in the VMs and fails over the application in case of failures. Multiple VMs from single host can be configured in the same cluster as nodes; however, please note that the failure of any one of the hosts should not result in more than half the cluster nodes going down. Hence we need to ensure that the cluster nodes are equitably distributed across the hosts.

Let us consider a four-node cluster. In this case the correct distribution would be to have two VMs each on two hosts configured as Serviceguard cluster nodes, rather than having three VMs from one host and one from the other. In the second case the failure of the host with three VM as nodes will bring down the entire cluster.

Figure 2. Cluster with KVM guest nodes across multiple hosts

Technical white paper | Using HP Serviceguard for Linux with Red Hat KVM Guests

6

Cluster with KVM guests and physical machines as cluster nodes In this configuration a cluster can be formed with KVM guests and physical machines. Serviceguard is installed on the KVM guests, physical machines and a cluster is formed among them. Serviceguard provides high availably to the applications running as packages in the VMs, physical machines and fails over the application to stand by nodes in case of failures. As mentioned in the above configuration, in this case also the cluster nodes must be equitably distributed.

The below figure shows an example where KVM guests are used to consolidate nodes within the clusters. A single standby host is used to run two separate VM guests that are each part of two different Serviceguard clusters. The packages that are normally running on primary physical nodes can failover to their corresponding KVM guest failover nodes running on a single standby KVM host.

Figure 3. Cluster with KVM guests and physical machine

Network topology for Serviceguard cluster using KVM guests

HP Serviceguard for Linux recommends having a highly available network configuration with redundant heartbeats and redundant data networks to avoid single point of failures. The following section describes how to achieve network redundancy using bridged network configuration (also known as physical device sharing).

Figure 4 shows a typical network configuration where four KVM guest across two physical hosts are bridged to form an HP Serviceguard for Linux cluster with redundant heartbeat networks.

Technical white paper | Using HP Serviceguard for Linux with Red Hat KVM Guests

7

Figure 4. Typical network configurations for KVM host and guest

Table 2. Network configuration details

Node name Interface name Remarks

KVM Guest SG_NODE1 bond0 (eth0, eth1) Assign Heartbeat IP

KVM Guest SG_NODE2 bond0 (eth0, eth1) Assign Heartbeat IP

KVM Guest SG_NODE3 bond0 (eth0, eth1) Assign Heartbeat IP

KVM Guest SG_NODE4 bond0 (eth0, eth1) Assign Heartbeat IP

KVM Guest SG_NODE1 eth2 Assign Public IP

KVM Guest SG_NODE2 eth2 Assign Public IP

KVM Guest SG_NODE3 eth2 Assign Public IP

KVM Guest SG_NODE4 eth2 Assign Public IP

KVM Host1 & KVM Host2 bridge0 (uses eth0 as slave) Used as “Source device” for eth0 in all KVM Guests

KVM Host1 & KVM Host2 bridge1 (uses eth1 as slave) Used as “Source device” for eth1 in all KVM Guests

KVM Host1 & KVM Host bridge2 (uses eth2 as slave) Used as “Source device” for eth2 in all KVM Guests

KVM Host1 & KVM Host eth0 Connected to heartbeat network switch

VM Host1 & KVM Host2 eth1 Connected to heartbeat network switch

KVM Host1 & KVM Host2 eth2 Connected to public network

KVM Host1 & KVM Host2 eth3 Can be used for Host Management (Not mandatory)

Technical white paper | Using HP Serviceguard for Linux with Red Hat KVM Guests

8

The bridge network configuration can be performed in two parts:

1. Configuring the host network

2. Configuring the guest network

Note The below listed steps are based on Red Hat documentation, for any updates or alternatives for these steps refer to the latest versions of “Red Hat Enterprise Linux 6 Virtualization Host Configuration and Guest Installation Guide.”

Host Network Configuration This bridge can be created using the following steps:

1. Disable NetworkManager—NetworkManager does not support bridging. NetworkManager must be disabled to use networking with the network scripts (located in the /etc/sysconfig/network-scripts/ directory). # chkconfig NetworkManager off

# chkconfig network on

# service NetworkManager stop

# service network start

Note: If you do not want to disable NetworkManager entirely, add “NM_CONTROLLED=no” to the ifcfg-* network script being used for the bridge

2. Create a new network interface script file in “/etc/sysconfig/network-scripts/” directory with the name “ifcfg-<device_name>” where the <device_name> must match the value of the DEVICE parameter inside the file. Its contents are as follows: DEVICE=bridge0

TYPE=Bridge

ONBOOT=yes

DELAY=0

NM_CONTROLLED=no

Note: The Line TYPE=Bridge is case sensitive. It must have uppercase ‘B’ and lowercase “ridge”

Technical white paper | Using HP Serviceguard for Linux with Red Hat KVM Guests

9

3. Add the physical interface to the bridge by modifying the network interface script of the given physical interface. Edit the file and add a line BRIDGE=bridge0, so that the contents of the configuration file look like the following example:

DEVICE=eth0

BRIDGE=bridge0

BOOTPROTO=none

HWADDR=00:19:b9:7e:c8:63

ONBOOT=yes

TYPE=Ethernet

IPV6INIT=no

NM_CONTROLLED=no

Note: Repeat the steps 2 and 3 for every interface that requires to be bridged.

4. Configure IP Tables to allow all traffic to be forwarded across the bridge # iptables -I FORWARD -m physdev --physdev-is-bridged -j ACCEPT

# service iptables save

# service iptables restart

Alternatively, prevent bridged traffic from being processed by iptables rules by appending the following lines in the file /etc/sysctl.conf

net.bridge.bridge-nf-call-ip6tables = 0

net.bridge.bridge-nf-call-iptables = 0

net.bridge.bridge-nf-call-arptables = 0

Reload the kernel parameter configured with sysctl by executing the below command

# sysctl –p /etc/sysctl.conf

5. Restart the network services to bring all the network configuration changes into effect.

# service network restart

6. Verify that your eth0 was added to the bridge0 using the brctl show command. # brctl show

The output must look similar to the following:

bridge name bridge Id STP enabled interfaces

virbr0 8000.000000000000 yes

bridge0 8000.0019b97ec863 yes eth0

“bridge0” is now available through virt-manager and libvirt. Guests can now connect to this device for full network access. This can be also viewed through virt-manager as shown below.

Click on Edit Connection Details Network Interfaces.

You can see the list of bridge interface created.

Technical white paper | Using HP Serviceguard for Linux with Red Hat KVM Guests

10

Guest Network Configuration Network access is provided to the guest VMs using the underlying bridged network interfaces configured in the host as shown in the previous steps. To achieve this, add the required virtual interfaces to the guest and configure the appropriate “Source device” and “Device model” for each of them.

Step 1: Open virt-manager by executing the command “virt-manager.”

Step 2: From the list of VM select the VM to which interface are to be added and open the same.

Step 3: Go to details view, Click on “Add Hardware.”

Technical white paper | Using HP Serviceguard for Linux with Red Hat KVM Guests

11

Step 4: Click on the “Network” Tab, choose the “Host device” and “Device model” and click finish.

“Host device” should be the appropriate bridge interface created on the host.

The “Device model” must be one of the Serviceguard supported device models as listed below. Ensure that all interfaces across the KVM guests cluster nodes have the same “Device model.”

Table 3. Supported Guest Network Interface Device Model

Guest OS Supported Guest Network Interface “Device model”

RHEL 5.7, 5.8, 5.9 and RHEL 6.1, 6.2, 6.3, 6.4 Hypervisor default, e1000, rtl8139, virtio

SUSE 11 SP1, SP2 Hypervisor default, e1000, rtl8139, virtio

Step 5: Repeat the above steps for all the required network interfaces on the guest.

Technical white paper | Using HP Serviceguard for Linux with Red Hat KVM Guests

12

Storage requirements for Serviceguard cluster on KVM guests

The applications managed by Serviceguard require shared storage across all the cluster nodes for their data. As of writing this white paper, only iSCSI as shared storage is supported in a Serviceguard cluster with KVM guests as cluster nodes.

For details on how to discover and login to iSCSI targets from the KVM guests refer to the below documents.

For RHEL refer to

https://access.redhat.com/site/documentation/Red_Hat_Enterprise_Linux/ -> Storage Administration Guide

For SUSE refer to

https://www.suse.com/documentation/sles11/pdfdoc/stor_admin/stor_admin.pdf

HP Serviceguard for Linux ensures that data integrity during application failover by using persistent reservation utilities provided by SG3_UTILS. SG3_UTILS should be installed on all the KVM guest nodes that are a part of the cluster.

Restrictions and exclusion

1. ISCSI is the only supported storage mechanism for shared storage in a KVM environment. FC storage is currently not supported.

2. Lock LUN as arbitration is not supported with iSCSI disk. Hence Quorum server is the only supported quorum mechanism that can be used for arbitration in HP Serviceguard for Linux cluster with KVM guests as cluster nodes.

3. Live migration of KVM guests that are configured as Serviceguard cluster nodes is not supported.

4. HP Serviceguard for Linux does not support KVM guests carved out of Red Hat Enterprise Virtualization Hypervisor (RHEV) as cluster nodes.

Supported host and guest operating systems

Table 4. Supported host and guest OS

Host OS Guest OS

RHEL 6.3, RHEL 6.4 RHEL 5.7, 5.8, 5.9, RHEL 6.1, 6.2, 6.3, 6.4. SUSE 11 SP1, SP2

Technical white paper | Using HP Serviceguard for Linux with Red Hat KVM Guests

Sign up for updates hp.com/go/getupdated

Share with colleagues

Rate this document

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein.

AMD is a trademark of Advanced Micro Devices Inc. Intel is a trademark of Intel Corporation in the U.S. and other countries.

4AA4-6752ENW, May 2013

Summary

This guide describes best practices for deploying HP Serviceguard for Linux cluster in a KVM environment. It is not the intent of this guide to duplicate the strategies and best practices of other HP or Red Hat technical white papers. The strategies and best practices offered here are presented at a very high level to provide general knowledge. Where appropriate, you are referred to specific documentation that provides more detailed information.

References

1. Red Hat Enterprise Linux 6 Virtualization Getting Started Guide https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/pdf/Virtualization_ Getting_Started_Guide/Red_Hat_Enterprise_Linux-6-Virtualization_Getting_Started_Guide-en-US.pdf

2. Red Hat Enterprise Linux 6 Virtualization Host Configuration and Guest Installation Guide https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_ Linux/6/pdf/Virtualization_Host_Configuration_and_Guest_Installation_ Guide/Red_Hat_Enterprise_Linux-6-Virtualization_Host_Configuration_and_Guest_Installation_Guide-en-US.pdf

3. Red Hat Enterprise Linux 6 Virtualization Administration Guide https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/pdf/Virtualization_ Administration_Guide/Red_Hat_Enterprise_Linux-6-Virtualization_Administration_Guide-en-US.pdf

4. Linux Enterprise Server 11 SP2 Storage Administration Guide https://www.suse.com/documentation/sles11/pdfdoc/stor_admin/stor_admin.pdf

Learn more at hp.com/go/linux-serviceguard-docs