NexentaStor Solutions Guide for OpenStack 1.0 v3content.mirantis.com/rs/451-RBY-185/images/NexentaS… · · 2018-03-28NexentaStor!Solutions!Guide!for!! Mirantis!OpenStack ... NexentaStor!supportMirantis!OpenStack!running!in!either!High!Availability!mode

White Paper NexentaStor Solutions Guide for Mirantis OpenStack

Version 1.1

May 2015

2 Copyright © 2014 Nexenta All Rights Reserved

Table of Contents

Preface ............................................................................................................................................................................... 3 About this document .................................................................................................................................................................. 3 Audience ......................................................................................................................................................................................... 3 External References .................................................................................................................................................................... 4 Document History ....................................................................................................................................................................... 4

Overview ........................................................................................................................................................................... 5 Reference Architecture ................................................................................................................................................ 6 Node configuration ..................................................................................................................................................................... 7 High Availability configurations ............................................................................................................................................ 7 Network topologies ..................................................................................................................................................................... 7

Installation and Configuration .................................................................................................................................. 8 Software requirements ............................................................................................................................................................. 8 Prerequisites ................................................................................................................................................................................. 8 Configure OpenStack Glance to store Images on NexentaStor ..................................................................................... 9 Export the newly created folder through NFS ................................................................................................................ 10 Configure OS to mount NexentaStor file system ............................................................................................................ 11 Updating the Glance configuration ..................................................................................................................................... 11 Configure OpenStack Cinder to use ISCSI from NexentaStor for block device storage ..................................... 12 Configure Cinder to use the NexentaStor ISCSI backend driver ............................................................................... 12 Configure OpenStack Cinder to use NFS from NexentaStor for block device storage ....................................... 13 Export the newly created folder through NFS ................................................................................................................ 14 Update the Cinder driver for NexentaStor 4.0.x ............................................................................................................. 15 Configure Cinder to use NexentaStor NFS Driver .......................................................................................................... 15

Validating the installation ........................................................................................................................................ 17


Preface

About this document This reference architecture describes how to integrate Mirantis OpenStack 5.1 (using OpenStack Icehouse) with NexentaStor 4.0.x, utilizing NexentaStor as a backend storage for both golden images as well as virtual machine storage.

Audience This document is intended for system administrators that want to integrate NexentaStor with an existing OpenStack installation. It is expected that the reader is experienced with OpenStack and has basic knowledge of NexentaStor

How to contact Nexenta For general questions or sales inquires please use [email protected], or the contact forms at http://nexenta.com/contact-‐us . For support and services inquiries, please use [email protected].


External References For additional information, please see the following documents:

Reference Location

Nexenta website www.nexenta.com

Mirantis website www.mirantis.com

OpenStack website www.openstack.org

NexentaStor Download and Documentation http://nexenta.com/products/downloads/nexentastor

Mirantis OpenStack 5.1 download http://software.mirantis.com

OpenStack Cinder Driver Reference documentation for Nexenta Driver

http://docs.openstack.org/icehouse/config-‐reference/content/nexenta-‐driver.html

Nexenta ISCSI driver configuration options http://docs.openstack.org/icehouse/config-‐reference/content/nexenta-‐iscsi-‐driver-‐options.html

Nexenta NFS driver configuration options http://docs.openstack.org/icehouse/config-‐reference/content/nexenta-‐nfs-‐driver-‐options.html

GitHub repository for Nexenta Cinder Drivers

https://github.com/openstack/cinder/tree/master/cinder/volume/drivers/nexenta

Document History

Version Date Changes

1.0 October, 2014 Initial version

1.1 May, 2015 Updated contact details


Overview NexentaStor is a Software-‐Defined-‐Storage platform that delivers unified file and block storage services scalable into the petabyte range using industry standard hardware, and includes advanced data management capabilities. When deployed in OpenStack environments NexentaStor provide a cost effective and scalable backend for storing both virtual machine images and application data.

The integration drivers between NexentaStor and OpenStack have been a part of the upstream releases of OpenStack since the Essex release in 2012. With the integration in the upstream Cinder drivers NexentaStor works ‘out of the box’ with all major distributions of OpenStack, including Mirantis OpenStack 5.1 NexentaStor can provide storage for OpenStack in a number of ways:

• Storage for the Image Repository service (Glance) through NFS • Block Storage for the Cinder Volume service through both ISCSI and/or NFS • Application data storage through both ISCSI and NFS

Below is a conceptual architecture drawing of how NexentaStor provide block and image services in an OpenStack environment. This document provide detailed steps of how an existing NexentaStor configuration can be integrated into Mirantis OpenStack.


Reference Architecture Both Mirantis OpenStack and NexentaStor can be configured to provide services in a variety of ways. To ensure that the best possible end result is achieved the guidelines and best practices for Mirantis OpenStack should be followed to configure OpenStack. For Best Practices for NexentaStor it is suggested that the administrator follow the guidelines outlined in the NexentaStor User Guide, and configure a dual/redundant controller setup as described in the NexentaStor High Availability Guide. During the certification and functional verification of NexentaStor and Mirantis OpenStack 5.1 the following configuration was used:

• A Mirantis Fuel Master node • One OpenStack Controller node • One OpenStack Cinder node • 5 OpenStack Compute nodes

Figure 1 Verification setup


Node configuration When NexentaStor is used in an environment that is managed by Mirantis OpenStack 5.1 all node deployments and OS configuration must be performed through Mirantis Fuel. NexentaStor support deployment using either Centos 6.5 or Ubuntu 12.04 LTS when using OpenStack Icehouse based releases. Note: This document assumes CentOS is being used when referencing command-‐line utilities and/or OS level configuration files and tools.

High Availability configurations NexentaStor support Mirantis OpenStack running in either High Availability mode, or in ‘non-‐HA’ mode. As best praxis NexentaStor itself should always be configured in high availability mode with two redundant controllers as documented in the NexentaStor HA User Guide.

Network topologies The traditional Nova Networking components where used to verify the solution for this document. However, NexentaStor will work with both Nova Network and Neutron modes as long as the Compute, Glance and Cinder nodes can access the NexentaStor server(s). It is highly recommended that the storage network is broken out as either a physical network, or as a separate VLAN, to keep the traffic segregated from other traffic types.

Figure 2 Network used for functional testing


Installation and Configuration When NexentaStor is used as a backend storage solution for OpenStack the guidelines and best practices published on by Mirantis apply. The deployment of Mirantis OpenStack should be done through FUEL, and the deployment should pass all automated health checks. After installing an OpenStack environment using Mirantis Fuel a number of configuration changes are required to use NexentaStor as backend storage for Cinder and Glance:

• It is ideal that a dedicated Cinder server is used for NexentaStor, but it is possible to configure NexentaStor as a part of a multi-‐backend configuration together with LVM or other backends. Please see the OpenStack Reference Documentation for details on how to configure multi-‐backend solutions.

• NexentaStor must be configured with a separate NFS share for Glance Images • If the NexentaStor Cinder NFS drivers is used a separate NFS share must be configured for Cinder

storage • To use the NexentaStor Cinder ISCSI backend it is suggested that a separate dataset/volume is created

to keep the Cinder volumes separated from other file systems • If the Cinder is configured to use the NexentaStor NFS backend on a NexentaStor 4.0.x appliance a

separate patch must be applied to the Cinder driver from Upstream Github repositories. This is to correct changes in behavior between NexentaStor 3.1.x and 4.0.x that occurred after the release of OpenStack Icehouse

Software requirements • Mirantis OpenStack 5.1 (OpenStack Icehouse). While NexentaStor is supported as a backend store for

Cinder from OpenStack Essex and onwards this guide focus on providing details based on OpenStack Icehouse

• OpenStack can be installed on any Linux distribution supported by Mirantis OpenStack 5.1.

Prerequisites This guide assumes that the following base requirements are satisfied:

1. NexentaStor 4.0.3 is installed and configured on supported hardware 2. Mirantis OpenStack 5.1 is used and Mirantis FUEL is used to deploy/manage servers. Technically all

versions since OpenStack Essex are supported, but this document is specific to Mirantis OpenStack 5.1 and Icehouse

3. The environment is running CentOS 6.5. Again, technically any Linux version that support Mirantis OpenStack 5.1 or OpenStack Icehouse are supported, but the documentation is specific to CentOS .


Configure OpenStack Glance to store Images on NexentaStor The OpenStack image repository service can be configured to store images on NexentaStor by using NFS (Network File System). The Glance Image Cache should remain local on the Glance server for best performance. Creating a new share in NexentaStor is done from either from the console of the appliance through an intelligent command line shell, or by the Nexenta Management Console. Below is a screenshot from the NexentaStor Management Console that detail how to create a suitable share for storing Glance Images. As an example we are creating a new share called “glance_images” in the “p1” volume. Figure 4 Create NFS share for Glance Image Repository

Figure 3 Create folder to store Glance Images


Export the newly created folder through NFS To enable the Glance server to access the new folder over NFS it needs to be shared. This is done through the folder overview in the NexentaStor Management Console (see below). If specific permissions are needed to enable the Glance user to access the Glance Images the advanced permissions tab can be reached by clicking Edit next to the NFS checkbox.

Figure 5 Export Glance folder through NFS


Configure OS to mount NexentaStor file system To configure the Glance server to utilize the new share from the NexentaStor appliance it must be mounted on the Glance server, and the Glance configuration file needs to be updated to use the new location as a storage location. To utilize the new file system create a mount point, for example /var/lib/glance-‐nexentastor and then add the following lines to /etc/fstab on the Glance server to ensure it is mounted at boot:

Replace “<NS>” with the name of the NexentaStor appliance and “<VOL>” with the name of the Volume where you created the glance_image folder above. In our case the entry would look like this:

Finally, confirm that the NexenataStor file system is correctly configured by manually mounting the file system by running “mount /var/lib/glance-‐nexentastor” on the Glance server. It should return without any errors and the file system should now be mounted under /var/lib/glance-‐nexentastor.

Note: It is possible that the CentOS OS images does not have the required packages for mounting an NFS file share pre-‐installed. If the mount command fail it may be required to first install the package nfs-‐utils by running “yum install nfs-‐utils”.

Updating the Glance configuration In the Glance configuration file /etc/glance/glance-‐api.conf the following line needs to be changed: so that it points to the new image repository on NexentaStor If the environment already had Glance images stored under /var/lib/glance/images these can now be copied to /var/lib/glance-‐nexentastor/ . It is suggested that a copy operation is used to allow Glance to continue to function during the operation. Restart Glance when all images have been transferred to NexentaStor.

After the restart has completed Glance will now utilize NexentaStor for image repository and the old images can be removed.

# service openstack-glance-api restart Stopping openstack-glance-api: [ OK ] Starting openstack-glance-api: [ OK ]

# Mount Glance Image Repository from NexentaStor <NS>:/volumes/<VOL>/glance_images /var/lib/glance-nexentastor nfs rw 0 0

# Mount Glance Image Repository from NexentaStor nexentastor:/volumes/p1/glance_images /var/lib/glance-nexentastor nfs rw 0 0

filesystem_store_datadir=/var/lib/glance/images/

filesystem_store_datadir=/var/lib/glance-nexentastor/


Configure OpenStack Cinder to use ISCSI from NexentaStor for block device storage NexentaStor can be used as block device storage for OpenStack Cinder using both ISCSI and NFS. When ISCSI is used the Cinder Volumes are created as ZVOLs, block devices, on the NexentaStor appliance. As best practice the ZVOLs should be stored in a dataset/volume that is unique per Cinder backend to keep them separated and ensure that the backend drivers do not interfere. In the examples below we are using the dataset/volume “p1” to store the ZVOLs created by the Cinder ISCSI driver.

Configure Cinder to use the NexentaStor ISCSI backend driver To enable the Cinder driver for NexentaStor ISCSI driver the following additions should be done to the Cinder configuration file /etc/cinder/cinder.conf:

Replace the HOSTNAME, USER and PASSWORD with the fully qualified hostname or IP address, and administrative credentials. The cinder.conf file should have appropriate file permissions so that only the Cinder driver and root can read the file (If unsure run “chmod 640 /etc/cinder/cinder.conf”). The value for nexenta_volume should be set to the name of the dataset/volume where the volumes should be stored. To enable the NexentaStor backend configured above ensure that it is enabled in the cinder.conf file:

Note: It is possible that the CentOS OS image does not have the required packages for using ISCSI pre-‐installed. Please ensure that the iscsi-‐initiator-‐utils package is installed and if it is not, install it using “yum install iscsi-‐initiator-‐utils”

When the configuration has been completed Cinder Volume driver needs to be restarted for the changes to take effect:

After the restart has completed Cinder will now utilize the new NexentaStor ISCSI backend to create new Cinder Volumes based on the scheduling and weighting rules.

# /etc/init.d/opestack-cinder-volume restart Stopping openstack-cinder-volume: [ OK ] Starting openstack-cinder-volume: [ OK ]

enabled_backends=nexentaiscsi-1

[nexentaiscsi-1] volume_driver=cinder.volume.drivers.nexenta.iscsi.NexentaISCSIDriver volume_backend_name=Nexenta_iSCSI

nexenta_rest_port=8457 nexenta_host=<HOSTNAME> nexenta_user=<USER> nexenta_password=<PASSWORD>

nexenta_volume=<VOLUME>


Configure OpenStack Cinder to use NFS from NexentaStor for block device storage NexentaStor can be used as block device storage for OpenStack Cinder using both ISCSI and NFS. When NFS is used the Cinder Volumes are created as regular or sparse files on the NexentaStor appliance. As best practice these files should be stored in one folder per Cinder backend to keep them separated and ensure that the backend drivers do not interfere. If the configuration utilize multiple Cinder backends these must all have separate NFS shares/folders. Below is a screenshot from the NexentaStor Management Console that detail how to create a suitable share for storing the Cinder Volumes used by the NFS driver. As an example we are creating a new share called “cinder_nfs” in the “p1” volume.

Figure 6 Creating a folder for storing Cinder Volumes over NFS


Export the newly created folder through NFS To enable the Cinder Volume server to access the new folder over NFS it needs to be shared. This is done through the folder overview in the NexentaStor Management Console (see below). If specific permissions are needed to enable the Cinder user to access the Cinder Volumes the advanced permissions tab can be reached by clicking Edit next to the NFS checkbox.

Figure 7 Share folder for Cinder Volume storage through NFS


Update the Cinder driver for NexentaStor 4.0.x OpenStack Icehouse was released before NexentaStor 4.0.x and the Nexenta NFS drivers that ship with Icehouse are not compatible with NexentaStor 4.0.x. The upstream drivers in GitHub were fixed during the release of NexentaStor 4.0.1 and this will not be a problem with the Juno release. However, for Icehouse an updated NFS driver is required. To download and install the updated NFS driver run the following commands on the Cinder Volume server:

This downloads the patched NFS driver from the upstream GitHub repository to enable Icehouse to work with NexentaStor 4.0.x.

Configure Cinder to use NexentaStor NFS Driver To enable the Cinder driver for NexentaStor NFS the following additions should be done to the Cinder configuration file /etc/cinder/cinder.conf:

To enable the NexentaStor backend configured above ensure that it is enabled in the cinder.conf file:

[nexentanfs-1] volume_driver = cinder.volume.drivers.nexenta.nfs.NexentaNfsDriver volume_backend_name=Nexenta_NFS nexenta_shares_config=/etc/cinder/nexenta_shares

enabled_backends=nexentanfs-1

# cd /usr/lib/python2.6/site-packages/cinder/volume/drivers/nexenta # mv nfs.py nfs.py.orig # wget \ http://raw.githubusercontent.com/openstack/cinder/46f3b9912af501aed5a2d914183110303f1c9cc5/cinder/volume/drivers/nexenta/nfs.py


The NexentaStor NFS file system access is configured through the file defined in nexenta_shares_config. This file contains all the information the Cinder driver requires to communicate with the NexentaStor appliance. The share definition file has the following format:

Replace the HOSTNAME, USER and PASSWORD with the fully qualified hostname or IP address, and administrative credentials. The share definition file should have appropriate file permissions so that only the Cinder driver and root can read the file (If unsure run “chmod 640 /etc/cinder/nexenta_shares”). Example nexenta_share definition:

When the NexentaStor driver is configured the Cinder driver will automatically mount the NFS export on a as-‐needed basis.

Note: It is possible that the CentOS OS images does not have the required packages for mounting an NFS file share pre-‐installed. If the mount command fail it may be required to first install the package nfs-‐utils by running “yum install nfs-‐utils”.

When the configuration has been completed Cinder Volume driver needs to be restarted for the changes to take effect:

After the restart has completed Cinder will now utilize the new NexentaStor NFS backend to create new Cinder Volumes based on the scheduling and weighting rules.

# /etc/init.d/opestack-cinder-volume restart Stopping openstack-cinder-volume: [ OK ] Starting openstack-cinder-volume: [ OK ]

nexentastor:/volumes/p1/cinder_nfs http://admin:nexenta@nexentastor:8457

<HOSTNAME>:/volumes/<VOLUME>/<FOLDER> http://<USER>:<PASS>@<HOSTNAME>:8457


Validating the installation After the configuration has been completed it should be validated using the automated health check capabilities of Mirantis Fuel. Doing this will catch most errors before trying to deploy production workloads. All Glance and Cinder related tests should pass with no errors. The Health Check is initiated from the Mirantis Fuel console (within the context of the relevant OpenStack cloud). All of the Sanity Tests should pass, and it is important that the “Create Volume… “ related Functional Tests also pass. If any of these basic tests fail the cause should be determined and corrected before proceeding to deploy a workload on these systems.

Figure 8 Run Mirantis Fuel Health Check

Documents

NexentaStor Solutions Guide for OpenStack 1.0 v3content.mirantis.com/rs/451-RBY-185/images/NexentaS… · · 2018-03-28NexentaStor!Solutions!Guide!for!! Mirantis!OpenStack ... NexentaStor!supportMirantis!OpenStack!running!in!either!High!Availability!mode