Optimizing Storage in VMware Environments - Aptare

Optimizing Storage in VMware Environments

Understanding Storage in VMware Virtualization Environments

© 2009 APTARE, Inc. ALL RIGHTS RESERVED. This document is the exclusive property of APTARE, Inc. Page 2 of 12

Executive Summary

The big IT trend this decade has been the attempt by organizations to gain control of escalating IT costs through consolidation. Leading this consolidation wave is server virtualization – possibly today’s hottest high‐tech buzzword. In this context, server virtualization means turning an x86 server into several “virtual machines” through software that allows this physical server to appear as many x86 servers. “Through 2012, virtualization will be the highest‐impact trend in infrastructure and operations” predicts Gartner analyst Thomas Bittman. 1 Leading this trend is the VMware® ESX™ enterprise‐level virtualization product, promoting server virtualization efficiencies to their fullest.

However, improving the efficiency of one IT resource often comes at the expense of another IT resource. In the case of server virtualization, the efficiency gained through better utilization of the physical server resources can be quickly offset by the inefficiency of storage resources that are now overtaxed. The ease at which virtual servers can be instantiated or in deployment of application appliances leads quickly to uncontrolled virtual machine sprawl. In reality, each of these virtual machines occupies real storage resources whether or not a virtual server is “powered on”.

Effectively managing storage resources in a virtual environment is APTARE® StorageConsole™ Virtualization Manager. By complementing VMware’s vCenter virtual console interface with in‐depth monitoring of the storage resources, administrators can be proactive about their real storage utilization in virtual environments. Developing “best practices” policies and procedures around virtual server storage resources using APTARE StorageConsole Virtualization Manager, organizations can gain back control of their virtual server storage allocations while realizing savings in terms of both physical and human costs.

1. Virtual Server Nirvana The de facto method for deploying applications in data centers at the turn of the century was to allocate one or two applications to a single physical server. This allocation method soon led to chaos as proliferation of physical servers hosting all these applications quickly grew out of control. Proliferation of physical servers in the data center created both operational and economic nightmares. IT organizations needed to increase efficiencies of their server operations. But, how?

Enter server virtualization. Clever people in the IT industry noticed that these physical servers were “loafing along”, typically running at less than 20 percent of their potential capacity. They determined that improvements in server utilization could be achieved by applying a solution developed by IBM in the 1960’s. Abstracting the physical hardware as a set of logical resources allowed multiple operating systems to run on a single physical server, each operating system viewing these logical resources as if it were a real server – virtualization. AMD and Intel formalized virtualization by creating a new mode in the x86 architecture (VMX root in Intel terminology). From there, virtualization of the x86 server

1 Thomas Bittman, “Server Virtualization: Trends, Best Practices, and the Future,” Gartner Data Center Conference, November, 2007.



environments took off with offerings like VMware vSphere™, Microsoft® Hyper‐V™, Citrix® XenServer™, and Oracle® Virtual Iron.

Today, server virtualization is the hottest trend in the industry. The next wave of server consolidation is upon us. As industry pundit Charles Babcock states, “While server consolidation made enormous gains in 2008, these advances are only the leading edge of changes that will come in the coming year through virtualization of the enterprise.”2 Gartner Research says virtualization will be the number one technology initiative in 2009, a year when more cost savings will be sought with tangible urgency because of the down economy.3 While Forrester Research’s Frank Gillett claims server virtualization has now reached the “tipping point” with 45% servers in 2009 being virtualized.4 Three industry analysts all agree on what is obvious to nearly everyone with connections to the IT industry. Of course, one could have determined the same predictions by just looking at the meteoric rise of VMware the past few years. Let’s examine the benefits realized through server virtualization.

2. Sever Virtualization Benefits Running four or five virtual servers on a single physical system offers new efficiencies to data center operations. The benefits of adding virtualization are many:

• Greatly improved efficiencies realized from the physical hardware resources

• Ability to “squeeze” more out of existing resources

• Lower electrical and HVAC bills (the “green” data center)

• Ability to bring up a new “system” in minutes rather than months

• Reallocation of systems and applications without having to send in a team of technicians with toolkits and a prayer

• Ease in providing disaster recovery solutions

Now, we’re even seeing six to eight virtualized operating systems running on a single Intel Xeon server. The ability to instantiate new virtual servers has become just a couple of “mouse clicks”. The result is greatly improved efficiencies of the physical servers in a data center through increased utilization of these resources. Applications of all types – from the mundane to the mission critical – now can have their own operating environment through server virtualization, improving reliability and security. One additional side benefit of deploying each application in its own operating environment is the ease in managing these applications through a single visual interface displaying the status of all virtual servers in the data center. This is highly desirable when mission‐critical applications require migration from one physical server to another due to hardware problems or even scheduled maintenance.

Another big benefit of server virtualization is the ability to instantiate new applications for testing or trial within a matter of minutes. In the past, we deployed new applications by finding an existing server

2 Charles Babcock, “Virtualization Set for Desktop Surge in 2009,” InformationWeek, 2 February 2009. 3 Philip Dawson & Jay Pultz, “Toolkit: Virtualization/Consolidation Is a Key Initiative for Infrastructure and Operations Leaders,” Gartner Research, 26 February 2009. 4 Frank Gillett, “Inquiry Insights: Server Virtualization, Q4 2008,” Forrester Research, 6 March 2009.



with excess capacity or even, heaven forbid, request the purchase of a new server through the IT procurement process. The result was weeks to months of delay. Now, with server virtualization, a new operating system along with a new application can be installed within a matter of hours. The savings in terms of time and money using server virtualization are enormous. Today, the trend is to package and deliver applications for a virtual server environment, complete with their operating system, data base, network stack and other run time requirements. These self‐contained “virtual applications” provide total autonomy, simplicity of installation, as well as giving software vendors an incubated environment that has been certified for their application.

3. Server Virtualization Dynamics The dynamics in the operation of physical server to the operation of virtual server environments are enormous. A virtual server is just not the same as a physical server. While it may be true they share many of the same management requirements (configuration, updates, load balancing, etc.), there are some significant differences. Major differences include:

• Deployment – The ease of deploying a new operating system and associated application(s) in minutes rather than months. In fact, with “virtualized applications” the entire process of loading, configuring and running both the operating system and its associated application can be done from a server administrator’s desk.

• Identity – Identifying server ownership and responsibility has been relatively easy in the past by association with something having an asset tag that is real and accountable from an asset inventory perspective. With server virtualization the identity of a specific virtualized server back to an owner now has become a bit tricky. As one can create literally 20 exact copies of a physical server without leaving one’s desk, this ownership and responsibility identity breaks down. The common solution is to provide some uniquely assigned name that may or may not have something to do with the actual application or function provided by this particular virtual server.

• Mobility – Virtual servers by their nature and design are intended to be mobile. They can be started on one physical server and moved easily to another physical server either manually or through system action (e.g., VMware’s VMotion™). In fact, these virtual servers do not need to be confined to the walls of a single data center.

• Operational Status – If the power‐on indicator and status indicators are “green” on a physical server one can usually assume that it is powered on and operational. These are the default states for physical servers, unless they have been powered down for maintenance. Not so with virtual servers. Virtual servers can move from online to suspended to an offline state with a single mouse click. In fact, if one examined the inventory of virtual machines on a physical server a good bet would be that for every virtual server that is in an online state, there will be more than two virtual servers in an offline state.

• Lifecycle – The life of a physical server is measured in years as it is an asset of an organization and normally deploys those applications justified for its purchase. Virtual servers completely break this notion of server and application lifecycles. The lifecycle of a virtual server can be



literally measured in minutes, not an uncommon occurrence in development and test environments.

The ease of deployment and basic invisibility of virtual servers are its biggest assets. These are also the biggest detriments. Simply, it is very easy to lose control of virtual server environments. What are some of the pressing problems with managing virtual server environments?

4. Virtual Server Nightmares What may be perceived as a panacea in improving the efficiency of physical servers through virtualization, can quickly turn into a virtual server nightmare. The ease with which one can instantiate a new virtual server can actually be the source of the problem. We can create a new VM instance in a matter of minutes. Unlike CPU, memory, or network resources where the demand on these resources is zero if a virtual server is “turned off”, storage resource allocation does not go away when a virtual server is “turned off”. The storage resource allocation for virtual servers is a real resource occupying space on expensive storage devices as a virtual server’s data must persist whether a virtual server is active or inactive. This makes managing storage resources allocated to virtual servers a demanding task. From Lynn Freeman, technical supervisor at the Group Health Cooperative in Seattle comes this assessment, “Deploying virtual machines (VMs) is easy and getting easier, but deploying the associated storage and networking components takes longer and is more complex. I can’t provision storage fast enough. It’s a bit of a frustration for my team.”5

This frustration is compounded by the fact that with virtualized servers there is a lack of management responsibility and ownership. The unfortunate consequence of virtualization is reduction in overall planning and limited interaction between the server administrators, network administrators, and storage administrators in the data center. Procurement of physical systems requires a good deal of planning, coordination and communication between these administration teams. In a virtualized environment, the unfortunate consequences are ad hoc planning and limited communication between administration teams. Managing the proliferation of virtual servers got so bad at one company that the CIO now requires that all new virtual server deployments go through the same procedures as a physical server deployment.6 This may seem like overkill, but at least now there is accountability over virtual server responsibilities and ownership.

What happens after a virtual server is deployed? Unless it is driving a mission‐critical application for the enterprise, it is often forgotten . The reason is that a virtual server can easily become invisible. Sure a virtual server shows up in the inventory of virtual servers and is visible through vendor management tools (e.g., VMware’s vCenter). But its identity often is an IP address and a set of status indicators. While the management tools provided by the virtualization platform vendors are excellent at deployment, they are lacking the required ongoing management and control of the environment. This makes tracking, visibility and reporting of virtual server environments difficult to administer and

5 Colin Steele, “Server Virtualization News: TechEd attendees weigh problems, benefits of server virtualization,” SearchServerVirtualization.com, 14 May 2009. 6 Charles Babcock, “Virtualization’s Promise and Problems,” InformationWeek, 16 May 2008.



manage. This cloak of invisibility can also create compliance issues where management and protection of corporate data assets maintained on virtual server storage is often guesswork.

Another trend taxing both physical and human IT resources is deployment of virtual applications. These are applications that are delivered as truly turnkey applications complete with their own operating environment. This concept has merits as Independent Software Vendors (ISVs) can now deliver an application along with the operating system on which it has been tested and certified. Now, an IT organization can “kick the tires” on a new application simply by turning on a virtual server. The problem is that once evaluations of virtual application are completed, they are often quickly forgotten. However, the “foot prints” of their presence remain.

Whatever the cause of VM server proliferation, the result is VM sprawl – or too much of a good thing leads to bad things.

5. Getting Smart about Virtualized Storage Let’s face it – server, storage and network resources are expensive. If we optimize the IT environment for one resource, we create sub‐optimization in other resources. VMware’s ESX server virtualization is ideal at optimizing x86 physical servers. Running multiple operating systems along with their applications on a VMware ESX server platform is a huge optimization. However, this advantage places additional pressure on storage and network resources as they can easily become oversubscribed, leading to performance problems among the virtual machines. Storage resources in a VMware ESX environment create an additional concern – storage allocated to these virtual machines must persist whether or not the machine is running. Thus, if one is not careful, storage resources can quickly become a pain point in managing a VMware ESX environment. Let’s examine how VMware manages storage to understand this pain point better.

VMware uses a concept called datastores to allocate storage to individual virtual machines within a VMware ESX environment. This virtual machine allocation is the virtual machine disk (VMDK). Datastores also store virtual machine snapshots and log files. Thus, they are essential to managing storage resources within VMware. Several types of storage can comprise a datastore: VMware Virtual Machine File System (VMFS), network attached storage, or a local file system. Contained within a datastore are the virtual machine disks (VMDKs), which a virtual machine sees as a physical storage volume (local SCSI target such as a C: drive), snapshots, log files, and various miscellaneous files (configuration files, NVRAM files, etc.). Thus, a datastore is platform independent and host independent. Further, a datastore does not change even when the virtual machines it contains are moved between physical servers.

VMware’s recommended approach to creating and maintaining datastores is to use their VMFS. VMFS is a cluster file system that allows server virtualization to scale across multiple ESX servers. This cluster file system approach allows multiple virtual machines with shared access to a consolidated pool of storage not only across single ESX servers, but also across multiple ESX servers within the data center. A VMFS is created by assigning a logical unit number (LUN) representing real storage allocation from an attached



storage array. Once a VMFS is created, individual virtual machine disks (VMDKs) can be installed within the VMFS datastore. A virtual machine sees the VMDK as a local SCSI target.

Key features of VMFS, tailoring it for use in a virtual environment, include:

• Automated file system with hierarchical directory structure

• Optimization for virtual machines in a clustered environment

• Lock management and distributed logical volume management

• Dynamic datastore expansion by spanning multiple storage extents

• Journal logging for fast recovery

• Encapsulation of an entire virtual machine in a single directory.

Another key concept of VMware’s VMFS is the ability to integrate raw disk maps (RDMs) into the VMFS structure. RDMs provide the ability to map a raw storage volume to a virtual machine through a “symbolic link” mapping file contained in the VMFS. This mapping makes the raw volume appear as files in a VMFS volume. RDMs are a convenient way to allocate application working storage to virtual machines, but still allow this storage to be a “first party” resource supported by all VMware features and capabilities (VMotion, DRS, HA).

If all this sounds complicated, it is! Making matters more confusing are the ongoing debates over how large a LUN should be for a given VMFS volume, whether to isolate or consolidate LUNs to virtual machines, and when to use raw disk maps (RDMs) in lieu of VMFS VMDK files. It’s no wonder that storage administrators supporting virtualized environments are confused and frustrated. Multiply the number of physical servers by the number of virtual servers deployed on each and the result can quickly escalate out of control.

6. The Answer Required to get a true understanding of what is going on in a VMware environment from a storage perspective is the ability to “see” storage usage at the physical server, virtual machine, and storage array level – simultaneously. The solution ‐ APTARE StorageConsole Virtualization Manager. With this advanced storage resource management tool administrators can quickly view their entire storage resource allocation in VMware environments. Further, APTARE StorageConsole Virtualization Manager can be used to proactively plan for transition to VMware, as well as to forecast storage resource demands in VMware environments. Using APTARE StorageConsole Virtualization Manager in concert with VMware vCenter gives users the right combination of tools to successfully manage their environments.

The key strengths of using APTARE StorageConsole Virtualization Manager in VMware environments are:

• Improvement of storage efficiency and usage

• Comprehensive migration planning

• Optimal performance tuning of the system infrastructure

• Effective monitoring of storage resource usage



• Ability to forecast storage demands as environment changes.

One of the main features of APTARE StorageConsole is the ability to group physical servers by business unit, asset allocation assignment, or by any delineation that makes sense to a business unit. This server grouping is independent of VMware vCenter views, providing administrators a natural view of physical server resources. The benefit using this feature is immediate – it provides an understandable structure to resources in the data center without artificial constraints imposed by the underlying virtualization software.

Using this concept, APTARE StorageConsole Virtualization Manager users can quickly see all the physical servers within a logical server group, multiple groups, or the entire ESX environment as selected by the user. The “VM Server Summary” presents a table showing information about the state of each ESX server – hardware identification and status, hardware health, CPU number and usage, memory total and usage, along with high‐level information about the physical storage server, VMware datastore usage, and virtual machines, active and total, assigned to each ESX server.

Figure 1. VM Server Summary Report

One only needs to “click” on the ESX server name to see more information about a physical ESX server and VM Server Detail is displayed in a new StorageConsole window. This window contains useful information specific to the ESX server selected:

• Detail information about the server – system vendor and model number, CPU type, speed and quantity, network interfaces, storage interfaces, virtual machines allocated and active, physical and logical storage assignments, and state of this x86 server

• CPU and memory usage and forecast information

• The largest virtual machines from a virtual disk (VMDK) perspective



• The datastores assigned to this ESX server along with total, used and free capacity, total used by the virtual machines, VMDK usage, and physical information about each datastore including number of extents, disks, and arrays

• Physical disk information – type (direct attach, SAN, NAS), LUN name, state, total capacity, allocated capacity, number of datastores and extents.

With this information, storage administrators can quickly “drill down” to get a precise view of both the physical server and physical storage allocations within an ESX virtualized environment. Complementing these views is the ability to see all virtual machines in inventory to a physical ESX server group or each individual ESX server. The APTARE StorageConsole Virtualization Manager VM Summary shows details about each selected virtual machine, such as name, state (connected, power on/off, status), and average CPU and memory usage over the last 24 hours. Specific to storage resource assignment, the VM Summary report provides details on the total amount of storage resource usage, virtual disks (VMDKs), datastore assignment and actual storage utilization for each virtual machine.

Figure 2. VM Summary Report

Drilling down on total storage assigned to a virtual machine shows all the files associated with this virtual machine – virtual machine disks (VMDKs), raw disk maps (RDMs), snapshots, and various configuration files, log files and other system files. From this one can quickly see just how storage is being “spent” on a specific virtual machine.

Another way to approach the problem of efficient storage utilization in a VMware environment is from the storage perspective. One can examine the details of physical disks assigned to an ESX server, including capacity and utilization details at the storage array level. However, central to ESX server storage allocations are datastores. Viewing detailed usage statistics about datastores is where APTARE StorageConsole Virtualization Manager excels. One invaluable report is the “Datastore Usage



Breakdown”. In a single view, one can visually see the status of all the datastores across an entire VMware deployment‐‐those associated with particular ESX servers, or allocated to various server groups.

Figure 3. Datastore Usage Breakdown Report

Besides determining the usage of VMDKs within datastores, users can easily determine the available space, non‐VM files, and other VM files such as RDMs. Of interest is the amount of Datastore storage associated with virtual machines no longer in the inventory. In this category, this storage represents allocations that have no home and are reclaimable.

Another useful feature of APTARE StorageConsole Virtualization Manager is performance and forecasting information gleaned from historical information at the ESX server and virtual machine level. Historical performance information is kept for the CPU, memory, datastore capacity, storage usage, I/O counts, and latencies. From this information, trending can be calculated to illuminate potential performance bottlenecks before they occur. Storage I/O counts and latencies can point out over and undersubscribed ESX servers, as well as “hot” virtual machines that are consuming I/O resources and are potentially impacting the performance of other virtual machines.



These reports, and others, showing the effects of server virtualization on storage resources are built into APTARE StorageConsole Virtualization Manager. Delivered with the product are standard report templates covering administration reporting, storage capacity and utilization reporting, and forecast and planning reporting. Users can simply generate a report specific to their VMware environment by selecting the scope to be applied to these standard report templates. These reports can be saved, emailed, or exported, as needed, to report on specific aspects of their VMware environment.

However, the real power of APTARE StorageConsole Virtualization Manager is the ability to construct customized reports using APTARE’s Report Template Designer. This sophisticated feature offers users the ability to generate custom reports from all the information collected for the virtualized infrastructure stored in the APTARE StorageConsole database. With this capability the precise information can be delivered to various stake holders relying on a smooth‐running VMware environment – from CIOs, to department heads, down to the administrators actively managing this environment.

What is the value of all this information? Simply, with APTARE StorageConsole Virtualization Manager users can:

• Determine storage resources allocated to virtual machines no longer in inventory that can be reclaimed (zombie virtual machines)

• Proactively plan for migration of new applications to VMware

• Specify alerting thresholds to notify administrators when datastore capacities get close to being out of space

• Inspect virtual machines and their demand on storage resources to contain virtual machine sprawl

• Quickly find out‐of‐date VMware snapshots using the APTARE filter selection reporting capability

• See virtual machines that are consuming more than their share of resources

• Develop chargeback customized reports using the APTARE advanced Report Template Designer

• Find hot spots in overall storage performance and take proactive measures to balance the I/O loads across the physical ESX servers

• Plan orderly growth of datastores by expertly adding extents as demand increases.

These are just some of the capabilities that can be realized using APTARE StorageConsole Virtualization Manager to effectively monitor and report on VMware environments. This advance tool takes the guesswork out of deploying and managing mission‐critical applications that turn up on VMware virtual machines. APTARE StorageConsole Virtualization Manager removes the complexity and frustration in provisioning and managing storage for VMware environments. Administrators can gain a 360 degree view of how their storage resources are being utilized – at the physical ESX server, virtual machine, and down to storage array perspective – in terms of both resource consumption and performance.



7. Putting it All Together Complementing APTARE StorageConsole Virtualization Manager is a comprehensive suite of products all built on the same StorageConsole platform. APTARE StorageConsole is a Web 2.0 platform that delivers comprehensive reporting and management tools for storage environments. Integrated into this platform is an Oracle 10g database, providing industry‐proven scalability and integrity for keeping track of what’s happening in a storage environment, past and present. Besides APTARE StorageConsole Virtualization Manager, this suite of products includes:

• APTARE StorageConsole Backup Manager

• APTARE StorageConsole Capacity Manager

• APTARE StorageConsole Replication Manager.

Used in combination, this suite of products provides a holistic view across an entire VMware infrastructure. APTARE StorageConsole Backup Manager operates in concert with VMware Consolidated Backup (VCB) and Veritas NetBackup™ user can quickly determine the success of their backups for virtual machines. APTARE StorageConsole Capacity Manager can provide greater insight in virtual environments on storage utilization at the array, virtual machine, and even down to the application level. Finally, as mission‐critical applications are deployed on VMware ESX servers using VMware High Availability (HA), APTARE StorageConsole Replication Manager becomes essential for reporting the state of snapshots of databases and file systems maintained on the storage arrays driving these applications.

The richness in product breadth and depth of the APTARE StorageConsole product suite is only part of the APTARE success story. The real strength behind APTARE is its customers and partners. Users of APTARE’s products participate in the APTARE StorageConsole User Group (http://newsgroup.storageconsole.com/) where they can post questions, share customized reports , and exchange ideas among the members of APTARE user community.

More information on APTARE StorageConsole Virtualization Manager or any of the APTARE StorageConsole products can be found by visiting www.aptare.com. You can also contact us at [email protected] or +1.866.927.8273.

Documents

Optimizing Storage in VMware Environments - Aptare