HPE 3PAR Thin Technologies · PDF fileDeleted volume snapshot space ... With HPE 3PAR Thin Conversion, a technology refresh no longer requires a terabyte-for-terabyte replacement,

HPE 3PAR Thin Technologies

Technical white paper


Contents Introduction ................................................................................................................................................................................................................................................................................................................................................... 5

Overview of HPE 3PAR Thin Technologies for data compaction ............................................................................................................................................................................................................... 5

Product highlights ............................................................................................................................................................................................................................................................................................................................ 5 HPE 3PAR Thin Provisioning software ......................................................................................................................................................................................................................................................................... 6 HPE 3PAR Thin Deduplication software ...................................................................................................................................................................................................................................................................... 6 HPE 3PAR Thin Clones software ........................................................................................................................................................................................................................................................................................ 6 HPE 3PAR Thin Persistence software ............................................................................................................................................................................................................................................................................ 6 HPE 3PAR Thin Conversion software ............................................................................................................................................................................................................................................................................ 6 HPE 3PAR Thin Copy Reclamation software ........................................................................................................................................................................................................................................................... 7

HPE 3PAR ASIC with Thin Built In ........................................................................................................................................................................................................................................................................................... 7

The benefits of Thin Provisioning ............................................................................................................................................................................................................................................................................................ 7

Avoid frequent storage capacity addition to servers ........................................................................................................................................................................................................................................ 7 Accelerate time to market ......................................................................................................................................................................................................................................................................................................... 8 Chargeback model in utility storage ................................................................................................................................................................................................................................................................................ 8

Thin Deduplication ................................................................................................................................................................................................................................................................................................................................. 8

Using fully provisioned volumes ................................................................................................................................................................................................................................................................................................ 9

System requirements ............................................................................................................................................................................................................................................................................................................................ 9

HPE 3PAR Volume Manager .......................................................................................................................................................................................................................................................................................................10

Physical disks ......................................................................................................................................................................................................................................................................................................................................10 Logical disks .........................................................................................................................................................................................................................................................................................................................................10 Common provisioning groups..............................................................................................................................................................................................................................................................................................10 Virtual volumes ..................................................................................................................................................................................................................................................................................................................................10 The anatomy of a thin volume ............................................................................................................................................................................................................................................................................................. 11 Thin volume metadata ............................................................................................................................................................................................................................................................................................................... 12 Thin Deduplication implementation ............................................................................................................................................................................................................................................................................... 12 Express Indexing .............................................................................................................................................................................................................................................................................................................................. 13

Common provisioning groups .................................................................................................................................................................................................................................................................................................... 14

CPGs and workloads ..................................................................................................................................................................................................................................................................................................................... 14 Availability level ................................................................................................................................................................................................................................................................................................................................ 14 Reasons to create multiple CPGs ...................................................................................................................................................................................................................................................................................... 15 CPG automatic growth ............................................................................................................................................................................................................................................................................................................... 15 Changing LD characteristics within a CPG ............................................................................................................................................................................................................................................................... 16 Snapshot space within CPGs ................................................................................................................................................................................................................................................................................................ 16 Allocating CPG space to thin volumes ......................................................................................................................................................................................................................................................................... 17

Sizing guidelines ..................................................................................................................................................................................................................................................................................................................................... 17


Thin volumes administration ....................................................................................................................................................................................................................................................................................................... 17

Allocation warnings and alerts ................................................................................................................................................................................................................................................................................................... 18

Allocation warnings ....................................................................................................................................................................................................................................................................................................................... 18 Allocation limits ................................................................................................................................................................................................................................................................................................................................. 19 Remaining physical capacity alerts ................................................................................................................................................................................................................................................................................ 20 Remaining CPG free space alerts ..................................................................................................................................................................................................................................................................................... 20 View logged warning and limit alerts ........................................................................................................................................................................................................................................................................... 20 User recommendations ............................................................................................................................................................................................................................................................................................................. 20

Capacity efficiency ................................................................................................................................................................................................................................................................................................................................ 21

Manually updating the capacity efficiency statistics ...................................................................................................................................................................................................................................... 26 Understanding capacity efficiency ratios .................................................................................................................................................................................................................................................................. 27

Tracking volume space usage ................................................................................................................................................................................................................................................................................................... 28

TPVVs ....................................................................................................................................................................................................................................................................................................................................................... 28 CPGs ........................................................................................................................................................................................................................................................................................................................................................... 28 System space ...................................................................................................................................................................................................................................................................................................................................... 29 Total used space ............................................................................................................................................................................................................................................................................................................................. 29

Usage reporting and trend analysis ..................................................................................................................................................................................................................................................................................... 30

Running out of space ......................................................................................................................................................................................................................................................................................................................... 31

Running out of space in a TV ............................................................................................................................................................................................................................................................................................... 31 Running out of space in a CPG ........................................................................................................................................................................................................................................................................................... 31 Running out of space on the system ............................................................................................................................................................................................................................................................................. 31

Reclaiming unused space ............................................................................................................................................................................................................................................................................................................... 31

Reclaiming unmapped logical disk space from CPGs ................................................................................................................................................................................................................................... 32 Automatically reclaiming unused space from volumes ............................................................................................................................................................................................................................... 32 Deleted volume snapshot space ....................................................................................................................................................................................................................................................................................... 32 Logical disks and chunklet initialization .................................................................................................................................................................................................................................................................... 32 Free page consolidation ........................................................................................................................................................................................................................................................................................................... 32

HPE 3PAR Thin Persistence software ............................................................................................................................................................................................................................................................................... 33

Thin Persistence reclamation .............................................................................................................................................................................................................................................................................................. 33 Thin Persistence methods ..................................................................................................................................................................................................................................................................................................... 34

Thin Deduplication with snapshots and clones ........................................................................................................................................................................................................................................................ 34

Thin Deduplication with Remote Copy ............................................................................................................................................................................................................................................................................. 35

Dynamic Optimization of virtual volumes ....................................................................................................................................................................................................................................................................... 35

Migrate between full and thin volumes on the same array ...................................................................................................................................................................................................................... 36 Estimating Thin Deduplication space savings ..................................................................................................................................................................................................................................................... 36

Online migration to HPE 3PAR StoreServ ...................................................................................................................................................................................................................................................................... 37

Preparing data for migration ................................................................................................................................................................................................................................................................................................ 38 Conclusion ................................................................................................................................................................................................................................................................................................................................................... 38


Appendix A—Thin volumes and applications ............................................................................................................................................................................................................................................................ 38

Oracle ........................................................................................................................................................................................................................................................................................................................................................ 38 Microsoft SQL Server .................................................................................................................................................................................................................................................................................................................. 39 Microsoft Hyper-V ......................................................................................................................................................................................................................................................................................................................... 39 SAP .............................................................................................................................................................................................................................................................................................................................................................. 39 VMware vSphere ............................................................................................................................................................................................................................................................................................................................ 40

Appendix B—Thin Persistence methods ....................................................................................................................................................................................................................................................................... 40

Thin Reclamation for Microsoft Windows Server 2012 ............................................................................................................................................................................................................................... 40 Thin Reclamation for Microsoft Windows Server 2003 and 2008 ..................................................................................................................................................................................................... 41 Thin Reclamation for VMware vSphere ...................................................................................................................................................................................................................................................................... 41 Thin Reclamation for Linux ................................................................................................................................................................................................................................................................................................... 42 Thin Reclamation for HP-UX ............................................................................................................................................................................................................................................................................................... 43 Thin Reclamation for UNIX .................................................................................................................................................................................................................................................................................................... 43 Thin Reclamation for HPE OpenVMS .......................................................................................................................................................................................................................................................................... 43 Thin Reclamation for Symantec Storage Foundation ................................................................................................................................................................................................................................... 44 Thin Reclamation for Oracle databases ..................................................................................................................................................................................................................................................................... 44

Appendix C—File Systems and Thin Deduplication ............................................................................................................................................................................................................................................. 46

Microsoft Windows ........................................................................................................................................................................................................................................................................................................................ 46 Microsoft Hyper-V ......................................................................................................................................................................................................................................................................................................................... 48 VMware vSphere ............................................................................................................................................................................................................................................................................................................................. 49 Oracle Solaris ...................................................................................................................................................................................................................................................................................................................................... 49 Linux........................................................................................................................................................................................................................................................................................................................................................... 50 Symantec Storage Foundation ............................................................................................................................................................................................................................................................................................ 51 HP-UX ........................................................................................................................................................................................................................................................................................................................................................ 51 IBM AIX .................................................................................................................................................................................................................................................................................................................................................... 52 HPE OpenVMS ................................................................................................................................................................................................................................................................................................................................. 52

Technical white paper Page 5

Introduction Compaction technologies such as thin provisioning, Thin Deduplication, and thin reclamation offer efficiency benefits for primary storage that can significantly reduce both capital and operational costs. Thin provisioning has achieved widespread adoption as it dramatically increases capacity efficiencies. It has become a data center “must have” for its ability to break the connection between logical and physical capacity. Deduplication is also an essential consideration when looking into deploying workloads onto a flash tier or an all flash array. Thin technologies can vary widely in how they are implemented; some are complex to deploy, while others use coarse allocation units and cannot deliver the required space savings. Not only is HPE 3PAR StoreServ Storage viewed as the industry’s thin technology leader, but third-party testing and competitive analysis confirm that HPE 3PAR StoreServ offers the most comprehensive and efficient thin technologies among the major enterprise storage platforms.1 HPE 3PAR Thin Technologies including HPE 3PAR Thin Deduplication, Thin Provisioning, Thin Conversion, Thin Persistence, and Thin Copy Reclamation achieve advanced data compaction through leveraging built-in hardware capabilities and Express Indexing technology.

Thin provisioning allows a volume to be created and made available as a logical unit number (LUN) to a host without the need to dedicate physical storage until it is actually needed. HPE 3PAR Thin Provisioning software has long been considered the gold standard in thin provisioning for its simplicity and efficiency. Unlike other “bolt-on” implementations, HPE 3PAR Thin Provisioning software is simple and efficient, helps your organization start new projects more quickly and on demand, and saves millions of dollars. HPE 3PAR Thin Provisioning leverages the dedicate-on-write approach of HPE 3PAR StoreServ Storage, allowing enterprises like yours to purchase only the disk capacity you actually need. HPE 3PAR Thin Provisioning integrates seamlessly with VMware® vSphere, Windows® Server 2012, Red Hat® Enterprise Linux® (RHEL), and Symantec Storage Foundation—greatly enhancing the operative and administrative efficiency of these platforms.

While HPE 3PAR Thin Technologies software is extremely simple to deploy and use, a certain amount of planning is advantageous to help maximize its benefits. This paper documents best practices on thin provisioning on HPE 3PAR StoreServ Storage and is intended for administrators looking to get the most out of their HPE 3PAR StoreServ deployment. In addition, it describes other HPE 3PAR Thin Technologies that you can use in conjunction with HPE 3PAR Thin Provisioning software to help maximize its effectiveness. Unique to HPE 3PAR StoreServ, HPE 3PAR Thin Conversion software enables you to reduce capacity requirements by 50 percent or more by deploying HPE 3PAR StoreServ in place of legacy storage.2

HPE 3PAR Thin Persistence software and other thin-reclamation solutions enable thin-provisioned storage on HPE 3PAR StoreServ arrays to stay thin over time by helping ensure that unused capacity is reclaimed for use by the array on an ongoing basis.

Now, HPE 3PAR Thin Deduplication and related HPE 3PAR Thin Clones software take thin efficiency to the next level. In addition, HPE 3PAR Thin Technologies protect SSD performance and extend flash-based media lifespan while ensuring resiliency.

Overview of HPE 3PAR Thin Technologies for data compaction Product highlights • HPE 3PAR Thin Technologies are completely automated.

• HPE 3PAR Operating System (OS) software uses a reservationless, dedicate-on-write approach to thin technologies and virtual copies that draws and configures capacity in fine-grained increments from a single free space reservoir without pre-dedication of any kind.

• HPE 3PAR Thin Technologies uses an allocation unit size of just 16 KiB, so you don’t have to worry about small writes consuming megabytes or even gigabytes of capacity.

• HPE 3PAR StoreServ is a storage platform built from the ground up to support thin technologies by reducing the diminished performance and functional limitations that plague bolt-on thin solutions.

1 HPE Thin Technologies: A Competitive Comparison, Edison Group 2012. theedison.com/pdf/Samples_HP_3PAR_Thin_Provisioning_WP.pdf 2 Based on documented client results that are subject to unique business conditions, client IT environment, Hewlett Packard Enterprise products deployed, and other factors. These

results may not be typical; your results may vary.

http://theedison.com/pdf/Samples_HP_3PAR_Thin_Provisioning_WP.pdf


HPE 3PAR Thin Provisioning software HPE 3PAR Thin Provisioning is the most comprehensive thin provisioning software solution available. Since its introduction in 2002, HPE 3PAR Thin Provisioning has been widely considered the gold standard in thin provisioning. It leverages dedicate-on-write capabilities of HPE 3PAR StoreServ to make storage more efficient and greener, allowing customers to purchase only the disk capacity they actually need and only when they actually need it.

With Thin Provisioning software, there is no more up-front capacity allocation. No more dedicating resources for each application. No more paying to house, power, and cool drives that might not be needed for months or years to come, or may seldom be needed at all.

HPE 3PAR Thin Deduplication software HPE 3PAR StoreServ Storage systems employ purpose-built HPE 3PAR ASICs at the heart of each controller node that feature efficient, silicon-based mechanisms to drive inline deduplication. This unique implementation relies on built-in hardware capability to assign a unique hash to any incoming write request and leverages the HPE 3PAR Thin Provisioning metadata lookup table for fast hash comparisons. When new I/O requests come in, the signatures of the incoming data are compared to the signatures of the data already stored in the array. When a match is found, the software marks the data as duplicated. The software also leverages the controller node ASICs to perform a bit-to-bit comparison that reduces the possibility of hash collision. The CPU-intensive job of calculating signatures of incoming data and read verify is offloaded to the hardware assist engines, freeing up processor cycles to deliver advanced data services and service I/O requests.

This inline deduplication process carries multiple benefits, including increasing capacity efficiency, protecting flash performance, and extending flash media lifespan. Other storage architectures lack the processing power to simultaneously drive inline deduplication and the high-performance levels demanded by flash-based media while also offering advanced data services (replication, quality of service [QoS], and federation).

HPE 3PAR Thin Clones software HPE 3PAR Thin Clones software is an extension of Thin Deduplication software for server virtualization environments, providing host-initiated deduplication that produces non-duplicative virtual machine clones for Microsoft® Hyper-V and VMware ESXi. These VM clones are created instantly by leveraging copy offload for VMware vStorage APIs for Array Integration (VAAI) and Microsoft Offloaded Data Transfer (ODX) technology without increasing capacity consumption on the HPE 3PAR StoreServ Storage system. HPE 3PAR Thin Clones software leverages Thin Deduplication to update the metadata table without copying data, using the inline deduplication technology to reduce capacity footprint as new write requests come in.

A thin clone is a replica of a VM that is created through copying only the metadata that associates a virtual volume with the physical data on disk. At initial creation, thin clones point to the same blocks of data as the cloned VM, however, as volumes are updated and the content of data changes, new writes will map to different deduplicated blocks (or create new blocks), so no direct overwrite process occurs. Thin clones continue to “stay thin” if updated data continues to map to existing deduplicated data on the array.

HPE 3PAR Thin Persistence software HPE 3PAR Thin Persistence software is an optional feature that keeps thin volumes (TVs) and read/write snapshots of TVs small by detecting pages of zeros during data transfers and not allocating space for the zeros. This feature works in real time and analyzes the data before it is written to the source TV or read/write snapshot of the TV. Freed blocks of 16 KiB of contiguous space are returned to the source volume, freed blocks of 128 MiB of contiguous space are returned to the common provisioning group (CPG) for use by other volumes. With HPE 3PAR Thin Persistence, HPE 3PAR StoreServ Storage customers can now leverage next-generation space reclamation technology to help minimize storage total cost of ownership (TCO).

HPE 3PAR Thin Conversion software With HPE 3PAR Thin Conversion, a technology refresh no longer requires a terabyte-for-terabyte replacement, but instead offers the opportunity to reduce up to 50 percent of your legacy capacity, simply and rapidly. This savings alone can save you up to 60 percent on the cost of a technology refresh.

With HPE 3PAR Thin Conversion, you can quickly shrink your storage footprint, reduce storage TCO, and meet your green IT targets. HPE 3PAR Thin Conversion software makes this possible by leveraging the zero-detection and inline deduplication capabilities within the HPE 3PAR ASIC and HPE 3PAR Thin Engine (a unique virtualization mapping engine for space reclamation) to power the simple and rapid conversion of inefficient, “fat” volumes on legacy arrays to more efficient, higher-utilization “thin” volumes. Getting thin has never been so easy.


HPE 3PAR Thin Conversion software is an optional feature that converts a fully provisioned volume to a thin provisioned or Thin Deduplication volume during data migration. Virtual volumes (VVs) with large amounts of allocated but unused space are converted to thin volumes that are much smaller than the original volume. During the conversion process, allocated but unused space is discarded and the result is a volume that uses less space than the original volume.

HPE 3PAR Thin Copy Reclamation software An industry first, Thin Copy Reclamation keeps your storage as lean and efficient as possible by reclaiming unused space resulting from deleted virtual copy snapshots and remote copy volumes. Thin Copy Reclamation is built on a virtualization mapping engine for space reclamation called HPE 3PAR Thin Engine, which is included as part of the HPE 3PAR OS. HPE 3PAR Thin Copy Reclamation software is an optional feature that reclaims space when snapshots are deleted from a system.

As snapshots are deleted, the snapshot space is reclaimed from a TV or fully provisioned virtual volume and returned to the CPG for reuse by other volumes. Deleted snapshot space can be reclaimed from virtual copies, physical copies, or remote copies.

HPE 3PAR ASIC with Thin Built In At the heart of every HPE 3PAR StoreServ node there is an HPE 3PAR ASIC with Thin Built In which features an efficient, silicon-based zero-detection and hashing mechanism. This unique hardware capability gives HPE 3PAR StoreServ Storage the power to perform inline deduplication and remove allocated but unused space inline and non-disruptively.

The zero-detect capability can recognize an incoming write request of 16 KiB of zeros and either not allocate space for the zero block or free the space if it was already allocated for that block. All this happens in cache, and therefore no zeroes are written to the backend. When a read request comes in for a block that is unallocated, the HPE 3PAR StoreServ will immediately return zeros back to the host.

Many other storage arrays do not detect blocks of zeroes on write. Instead, the zeros are written to disk and a scrubbing process later detects these zeroed blocks and discards them. With this approach, the zeroed blocks consume space until they’re scrubbed, and therefore they may not be available for use by other volumes when needed. Also, there is increased load placed on the storage as the scrubbing process examines the block contents on the physical storage.

The HPE 3PAR StoreServ built-in zero-detection capability can be controlled per TV, and it is enabled by default.

The HPE 3PAR ASIC is also the only solution in the industry with a built-in, silicon-based hash calculation engine. With HPE 3PAR Thin Deduplication software, the process of calculating the hash signatures for incoming data and verifying reads are offloaded to the HPE 3PAR ASICs, freeing up processor cycles to deliver advanced data services and service I/O requests. This hardware-assisted approach together with a unique Express Indexing feature enables extremely efficient, extremely granular block-level inline deduplication that carries multiple benefits, including increased capacity efficiency, flash performance protection, and flash media lifespan extension. Unlike other approaches, HPE 3PAR Thin Deduplication software performs a full check on all data before marking it as duplicated, which is essential to ensuring data integrity for mission-critical environments.

The benefits of Thin Provisioning Thin Provisioning can be used with nearly all applications available in the market to improve storage utilization dramatically. The following use cases illustrate the superior value from Thin Provisioning on HPE 3PAR StoreServ.

Avoid frequent storage capacity addition to servers Storage administrators often allocate and present a large amount of storage to a server at the start of a project to accommodate its long-term growth requirements. This practice reduces the number of times that LUNs mapped to the server have to be expanded, an operation that can be complex, time consuming, and cause server downtime. With the HPE 3PAR Thin Provisioning software, it is possible to allocate large amounts of storage to a particular server but only consume physical space on the array as used. This fits environments where:

• The addition or expansion of storage provisioned to servers is not desired, for example, in mission-critical environments.

• Relatively slow growth rates or unpredictable growth over time occurs. This can happen on large file systems used for group shares, mailbox databases, or general database space.


Accelerate time to market In an application’s development stage, there is sometimes a requirement for the application storage to be in place and ready before the application goes live. With the HPE 3PAR Thin Provisioning software, it is possible to present the full amount of storage immediately so that it is ready for the developers to work on without the requirement for the full physical capacity being in place. Because thin provisioning gives storage administrators the ability to place limits on TPVVs and CPGs, the administrator can make sure that the developer’s work does not affect other production systems by using the free space within the HPE 3PAR StoreServ. After the application is ready to go live, these limits can be removed.

Chargeback model in utility storage HPE 3PAR Thin Provisioning is ideal for enterprise customers and service providers wishing to deploy a storage offering where usage chargeback is an important component of the service. Thin Provisioning offers these customers the following benefits:

• Deploy quickly

• Decouple the charges for storage from the limitations of actual presented storage

• Remove the exposure to disruption of service during future capacity expansions

• Collect detailed charging data at the individual VV level or at the CPG level

When planning to collect charging data, it is recommended that the names chosen for objects like CPGs, VVs, snapshots, and domains contain a meaningful prefix or suffix referring to the project, application, line of business (LOB), or department the objects belong to. This enables the grouping of objects in the chargeback report. The HPE 3PAR OS allows up to 31 characters for the name of an object.

Thin Deduplication Deduplication has become standard with disk-based backup due to a high degree of data redundancy and less emphasis on high performance, backup, and archive workloads have been an ideal target for deduplication technologies. Traditional primary storage workloads such as OLTP have lower data redundancy and hence lower deduplication ratios and therefore deduplication of primary storage has not been seen as beneficial. However, the landscape around primary storage deduplication is changing. The high data redundancy of server virtual machine (VM) images and client virtualization environments with hosted virtual desktops have tremendous potential for benefiting from deduplication. Home and file directory consolidation is another area where primary storage deduplication can offer significant space savings.

With the increasing use of SSD storage, deduplication for primary storage arrays has become critical. The cost differential between SSDs and hard disk drives (HDDs) requires compaction technologies like thin provisioning and deduplication to make flash-based media more cost-efficient. The widespread deployment of server virtualization is also driving the demand for primary storage deduplication.

The following should be taken into account before implementing Thin Deduplication:

• It is only applicable to virtual volumes residing solely on SSD storage. Any system with an SSD tier can take advantage of Thin Deduplication. Since a Thin Deduplication can only be on SSD storage, they are not compatible with the sub-LUN tiering of Adaptive Optimization (AO). If a thinly deduped volume exists within a common provisioning group (CPG) then the CPG is not available for use in an AO configuration. Conversely, if a CPG is already in an AO configuration then it is not possible to create a thinly deduped volume in the CPG.

• The granularity of deduplication is 16 KiB and therefore the efficiency is greatest when I/Os are aligned to this granularity. For hosts that use file systems with tunable allocation units consider setting the allocation unit to 16 KiB or a multiple of 16 KiB. For more information on Thin Deduplication and file systems, see Appendix C. For applications that have tunable block sizes consider to setting the block size to 16 KiB or a multiple of 16 KiB.

• Deduplication is performed not only on the data contained within the virtual volumes but also between virtual volumes in the same CPG. For maximum deduplication store data with duplicate affinity on virtual volumes within the same CPG.

• Thin Deduplication is ideal for data that has a high level of redundancy. Data with a low level of redundancy such as databases or data that has been previously been deduplicated, compressed, or encrypted are not good candidates for deduplication and should be stored on thin provisioned volumes. Use the Dedup Estimate tool to check the dedup ratio of existing volumes before conversion to thinly deduped volumes.


Using fully provisioned volumes The use of thin provisioning has minimal performance impact and has the significant operational benefit of reducing storage consumption. However, there are certain workloads and applications thin provisioning may not be of benefit such as:

• Systems with a high file system utilization—File systems on TPVVs that are nearly full offer reduced benefits of thin provisioning. This relates to the fact that, for thin volumes, the thin-provisioning license charge is incurred in addition to the cost for physical disk capacity used. In the case of file system utilization rates of 80 percent or higher, it may be more cost-efficient to use fat-provisioned volumes to hold the data.

• Applications that write continuously to new space—An example of this is Oracle redo log file.

• Databases not in “autoextend” mode (or equivalent) —Some databases initialize their assigned storage at creation time by writing markers at regular intervals over the entire volume. This has the same effect as provisioning file systems with a high utilization on thin volumes.

• Small capacity requirements—Thin Provisioning is ideal for large-scale volumes. For small size VVs (256 MB up to a few tens of GB), even the minimum growth increment of the CPG may mean that minimal benefit is realized. Use care in the selection of the CPG growth increment in this case.

• Environments that require host encrypted volumes—Writing blocks of zeros to a host encrypted volume on a newly created HPE 3PAR StoreServ thin-provisioned volume will cause space to be allocated on the TPVV because the encryption alters the content of the blocks. Applying encryption to thin-provisioned volumes that already contain data or rekeying them also inflates the zero blocks, making the volume consume space as if it was fully provisioned. Attempting to re-thin the volume by writing zeros to allocated but unused space is not possible as well. As a result, host encryption and thin provisioning do not cooperate well.

• Environments that require SAN encrypted volumes—Like host-based encryption, encryption by a device in the data path (e.g., SAN switch) will also alter the data stream so that blocks of zeros written by the host are not passed onto the storage. A notable exception is Brocade SAN switches. With the introduction of Fabric OS 7.1.0, the Fabric OS encryption switch can automatically detect if a disk LUN is a thin-provisioned LUN. If a LUN is detected as being thin-provisioned, then first-time encryption and rekey are done on the allocated blocks only. This thin provision LUN support requires no action by the user.

• Copy-on-write file systems—File systems that write to new blocks rather than overwrite existing data are not suitable for thin provisioning, as every write will allocate new storage until the volume is fully allocated. An example of a copy-on-write (CoW) file system is Oracle Solaris ZFS.

However, if a significant part of the array will be utilized for thin volumes, it is advised to use thin provisioning for all volumes, to help minimize management overhead and help maximize space efficiency.

System requirements HPE 3PAR Thin Technologies are included as part of the HPE 3PAR OS Software Suite and are available on all HPE 3PAR StoreServ models. The functionality offered by each system license is summarized in table 1.

Table 1. Summary of the thin storage technology features

LICENSE FUNCTIONALITY

Thin Provisioning The creation of TPVVs

Thin Deduplication Creation of TDVVs for inline deduplication on SSD Tier and Thin Clones for host-initiated deduplication on TDVVs

Thin Persistence Zero-detection to free previously allocated space

Thin Conversion Zero-detection to prevent allocation of space and enables T10 UNMAP support

Thin Copy Reclamation Reclaiming unused space resulting from deleted virtual copy snapshots and remote copy volumes


HPE 3PAR Volume Manager The HPE 3PAR OS has a volume manager that provides the virtual volume abstraction. It is comprised of several layers with each layer being created from elements of the layer below.

Physical disks Every physical disk (PD) that is admitted into the system is divided into 1 GB chunklets. A chunklet is the most basic element of data storage of the HPE 3PAR StoreServ. These chunklets form the basis of the RAID sets; depending on the sparing algorithm and system configuration, some chunklets are allocated as spares.

Logical disks The logical disk (LD) layer is where the RAID functionality occurs. Multiple chunklet RAID sets, typically from different PDs, are striped together to form a LD. All chunklets belonging to a given LD will be from the same drive type. LDs can consist of all Nearline (NL), Fibre Channel (FC), or solid-state drive (SSD) type chunklets.

There are three types of logical disk:

1. User (Usr) logical disks provide user storage space to fully provisioned virtual volumes.

2. Shared data (SD) logical disks provide the storage space for snapshots, virtual copies, thin provisioned virtual volumes (TPVVs) or thin duplicated virtual volumes (TDVVs).

3. Shared administration (SA) logical disks provide the storage space for snapshot and TPVV/TDVV administration.

They contain the bitmaps pointing to which pages of which SD LD are in use. The LDs are divided into “regions,” which are contiguous 128 MiB blocks. The space for the virtual volumes is allocated across these regions.

Common provisioning groups The next layer is the CPG that defines the LD creation characteristics, such as RAID type, set size, disk type for chunklet selection, plus total space warning, and limit points. A CPG is a virtual pool of LDs that allows volumes to share resources and to allocate space on demand. A thin-provisioned volume created from a CPG will automatically allocate space on demand by mapping new regions from the LDs associated with the CPG. CPGs and their importance to thin provisioning will be discussed in more detail in the next section.

Virtual volumes The top layer is the virtual volume (VV). VVs draw their resources from CPGs, and are the only data layer visible to hosts when they are exported as virtual logical unit numbers (VLUNs).

A VV is classified by its type provisioning which can be one of the following:

• Full—Fully Provisioned VV, with space for the base volume allocated from the associated user CPG and no snapshot space.

• CPVV—Commonly Provisioned VV. A fully provisioned VV with space for the base volume allocated from the associated user CPG and snapshot space allocated from the associated snapshot CPG.

• TPVV—Thin Provisioned VV, with space for the base volume allocated from the associated user CPG and snapshot space allocated from the associated snapshot CPG (if any).

• TDVV—Thin Deduplicated VV, with space for the base volume allocated from the associated user CPG and snapshot space allocated from the associated snapshot CPG (if any).

On creation of a thin volume (TPVV or TDVV), the size of the VV is specified, but no storage is allocated. Storage is allocated on demand in the shared data area as required by the host operation being performed. The shared admin area contains the metadata indexes that point to the user data in the SD area. Since the SA metadata needs to be accessed to locate the user data, the indexes are cached in policy memory to help minimize the performance impact of the lookups.


Thin volumes associated with the same CPG share the same LDs and draw space from that pool as needed, allocating space on demand in small increments for each controller node. As the volumes that draw space from the CPG require additional storage, the HPE 3PAR OS automatically creates increase in the logical disk storage until either all available storage is consumed or if specified, the CPG reaches the user-defined growth limit, which restricts the CPG’s maximum size. The size limit for an individual virtual volume is 16 TiB.

The relationship between the HPE 3PAR OS abstraction layers is illustrated in figure 1.

Figure 1. Overview of thin virtual volumes

The anatomy of a thin volume The sparse nature of thin volume requires a mechanism to map the virtual volume address space to physical storage pages, which are 16 KiB in size. The HPE 3PAR OS does this by using page tables, which are similar in concept to the virtual memory mapping page tables of OSs. The logical block address (LBA) of a SCSI read or write command is then used as an index into three different page tables to find a particular 16 KiB block belonging to a TV. New writes will result in a free 16 KiB page being allocated (or multiple 16 KiB pages for larger writes); rewrites of data will simply reuse any previously allocated space and writes of 16 KiB or larger of zeros will result in space reclaim.

The address space of a TV is also assigned across all the nodes in the system in 32 MiB increments in a round-robin manner. The LBA for a new write I/O therefore determines which node the I/O will be directed to and this aids the load balancing of I/Os across all nodes in the system.

The system also needs a mechanism to track TV SD page usage since space reclaim creates unused “holes” in the 128 MiB regions allocated to particular TVs. To do this, there are two levels of bitmaps used to track the TV used space. The SD logical disks are made up of 128 MiB regions that contain 8,192 16 KiB pages. The first level (L1) bitmap indicates whether a given 128 MiB region in SD space is in use or not. If a region is in use then a second level (L2) bitmap indicates whether a particular 16 KiB page in that region is in use or not. The relationship between regions, pages, and bitmaps is shown in figure 2.

As individual 16 KiB pages are freed, the space in the SD LDs can become fragmented and a defragment thread periodically examines the bitmaps to determine whether defragmentation should be initiated. The data from partially allocated regions is then reorganized to create larger contiguous blocks of free space that can be reclaimed.


Figure 2. Overview of the SD space tracking bitmaps

Thin volume metadata The user data from thin volumes are stored in SD LDs that use the RAID type and set size defined by the CPG they have a naming convention of tp-x-sd-y.z. The bitmap and page table metadata indexes that point to the user data in the SD LDs are stored in shared administration (SA) logical disks. Since the SA LDs are critical to the system, they are three-way RAID 1 protected with high availability (HA) cage availability, and they have a naming convention of tp-x-sa-y.z. In addition, the SA metadata indexes are cached in policy memory to help minimize the performance impact of the lookups.

Although there is SA space in addition to the SD space, the SA size is not a simple percentage of the SD size. The first TPVV/TDVV created in a CPG will have 8 GB per node-pair of SA LD space created (due to the three-way RAID 1 protection this will be 24 GB of raw space) for the metadata indexes. As the volume is populated, or more thin volumes added, the SD space usage will increase (more LDs may be added or the existing LDs expanded) but the amount of SA space will not expand until the initial 8 GB allocation is used. The CPG will then grow the SA space based on the growth parameters that have been set.

On a small system with a few thin volumes, the SA space may be as much as 10 percent of the SD space, but on a medium-large system, the SA space used is typically only 1 percent of the SD space.

Thin Deduplication implementation Thin Deduplication is an extension of industry leading HPE 3PAR Thin Provisioning. With primary storage, it is very important to keep to I/O latencies as low as possible. Therefore, the process of transferring host data to the array is the same for both normal thin provisioned volumes and Thin Deduplication volumes. The deduplication is performed inline as part of the process of flushing the acknowledged I/Os from cache.

A new volume type called TDVV is used for thinly deduplicated virtual volumes. TDVVs in the same CPG will share common pages of data. A thin provisioned volume called the Dedup Store (DDS) is automatically created when the first TDVV is created from a CPG and it is used to hold the data (both unique and duplicate) written to the TDVVs. The calculation of the hash, which is normally a CPU intensive task, is offloaded to the ASIC. If a hash match is found, the ASIC is used to do a bit level comparison of the new data with the existing data to ensure no hash collision has occurred.


Figure 3. ASIC-based signature generation for inline deduplication

Express Indexing HPE 3PAR Thin Deduplication uses an advanced metadata lookup mechanism called Express Indexing which is unique as it combines the built-in hash generation capability of the HPE 3PAR ASIC with HPE 3PAR Thin Provisioning metadata lookup table for extremely fast hash comparisons.

Once the hash signature of the incoming data has been generated, there must be a check to see if data with the same signature already exists. This is typically a CPU and memory intensive operation involving search mechanisms on large pools of reserved memory containing the signatures of existing data. The HPE 3PAR OS instead uses a technique called Express Indexing to detect duplicate page data. This process takes advantage of the highly optimized and robust address space to physical storage page indexing mechanism of thin provisioned volumes.

When a new write I/O request comes in, the Logical Block Address (LBA) is used as an index into page tables as per a regular TPVV but instead of allocating a new page the hash signature of the incoming data page is computed by the HPE 3PAR ASIC and compared to the signatures of the data already stored in the CPG. If a match is found then the existing block is compared at a bit level with the new block by the ASIC. A successful comparison is a “dedup hit,” in which case the virtual volume pointers are updated to reference the existing data. In the unlikely event a hash collision is detected, then the data is stored in the virtual volume and not treated as duplicate. If the hash of the new data was not found by the lookup, a new page is allocated in the DDS. The value of the hash is used as the offset of the page in the DDS therefore a simple a page translation will turn the hash into a physical page location.

When a read is performed, the page translation of the TDVV LBA can point either to the local store or to the DDS volume.


Common provisioning groups CPGs are policies for how free chunklets within the HPE 3PAR StoreServ should be used when creating volumes. A CPG policy contains parameters such as disk type, disk speed, RAID type, growth increment, chunklet radial placement, and availability level. CPGs automatically grow the underlying LD storage, according to the stated growth parameters, on demand to store data in a TPVV or TDVV. No administrator intervention is needed with this provisioning operation.

CPGs and workloads HPE 3PAR StoreServ performs more efficiently for any type of workload, and different workloads can be mixed on the same array. These different workloads may need different types of service levels to store their data. For example, for high-performance mission-critical workloads, it may be best to create volumes with RAID 5 protection on SSDs or RAID 1 protection on Fast Class (FC or SAS performance hard disk drives [HDDs]). For less-demanding projects, RAID 5 on FC drives or RAID 6 on NL drives may suffice. For each of these workloads, you can create a CPG to serve as the template for creating VVs for the workload. It is most efficient to keep the number of CPGs low as each CPG reserves its amount of growth space. All virtual volume types can be moved between CPGs with the HPE 3PAR Dynamic Optimization (DO) software command tunevv, thereby changing their underlying physical disk layout and hence their service level. The same Dynamic Optimization technology can be used to convert volume types; this is described in greater detail later in the paper.

The following sections discuss what to consider when planning CPGs for thin provisioning and virtual copy snapshots and recommend a set of best practices.

Availability level To provide HA, chunklets from the same RAID set should be distributed across multiple components.3

There are three levels of availability that can be selected with HPE 3PAR StoreServ.

• HA CAGE means that no two members of the same RAID set can be in the same drive enclosure. For example, to support RAID 5 3+1 (set size four), four drive chassis connected to the same node pair are required. This helps ensure that data is still available in the event that access to an entire drive cage is lost. This applies to drive chassis that are point-to-point connected to the nodes (no daisy chain).

• HA MAG means that no two members of the same RAID set are in the same drive magazine. This allows a wider stripe with fewer drive chassis; for example, a RAID 5 stripe size of 7+1 (set size eight) would be possible with only four drive chassis, provided each chassis had at least two drive magazines.

• HA PORT applies only to daisy-chained drive chassis. When this level of availability is selected, no two members of the same RAID set can be in drive chassis that are dependent on one another for node connectivity. For example, in a system in which there are eight drive chassis with four of the drive chassis connected to another drive chassis for node access, HA PORT would only allow RAID 5 3+1 (set size four) in order to prevent the loss of one drive chassis from causing a loss of data access. On systems that do not have daisy-chained cages, such as the HPE 3PAR StoreServ 10000, setting HA PORT is the same as setting HA CAGE.

When creating CPGs, it is strongly recommended to select HA CAGE availability always.

3 It is important to understand that drive magazines consist of four drives for HPE 3PAR StoreServ 10000. Drive magazines consist of only a single drive in the HPE 3PAR StoreServ

7000 and 7450 Storage systems.


Reasons to create multiple CPGs All TPVVs and TDVVs associated with a CPG allocate space from a shared pool of LDs. This means that VVs associated with a particular CPG have identical LD characteristics. VVs that require different characteristics must use a different CPG.

Reasons to create multiple CPGs include:

• To define different service levels, e.g., RAID 1, RAID 5, or RAID 6 with FC, NL, or SSDs.

• To map VVs belonging to different lines of business, departments, or customers onto particular CPGs for reporting and management purposes; creating CPGs with exactly the same characteristics but a different name is possible. This allows a logical separation of resources and may help with chargeback models, as chargeback could be based on CPG space usage rather than usage at an individual VV level.

• There can be multiple deduplication CPGs in the system, each deduplication CPG providing storage for different sets of volumes. This allows volumes with similar datasets to be grouped together and facilitates deduplication at the virtual domain level in a multi-tenancy environment.

• When virtual domains are used, because a CPG can only belong to one virtual domain.

• When HPE 3PAR Adaptive Optimization (AO) software is used, because a CPG can only belong to one Adaptive Optimization policy.

• When using Thin Deduplication there is a limit of 2048 TDVVs per CPG on HPE 3PAR StoreServ 20000 systems and 1024 TDVVs per CPG on all other HPE 3PAR StoreServ systems.4

While there are several reasons to create multiple CPGs, it is recommended that the number of CPGs be kept to a minimum as each CPG will reserve its own growth space. For deduplication, the recommended maximum number of CPGs is shown in table 2.

Table 2. Recommended maximum dedup CPGs

SYSTEM TYPE MAXIMUM DEDUP CPGs PER NODE PAIR

HPE 3PAR StoreServ 7200/7200c/7400/7400c 2

HPE 3PAR StoreServ 7450/7440c/7450c 4

HPE 3PAR StoreServ 8200/8400 2

HPE 3PAR StoreServ 8440/8450 8

HPE 3PAR StoreServ 20000 series 12

CPG automatic growth By default, CPGs dynamically allocate storage in increments specified by the CPG’s growth increment. This on-demand allocation unit determines the automated response to the growth demands of TPVVs and TDVVs. The growth increment should be large enough to ensure wide striping across most or all physical disks that are part of the CPG. To grow the TPVV/TDVV, the HPE 3PAR OS may expand existing LDs according to the CPGs growth increment or create additional ones. Growth is triggered when the CPG’s available space falls below 85 percent of the growth increment value. The CPG growth increment can be changed at any time, which also changes the threshold for the next growth increment to happen. A mechanism with warnings and limits can be configured on the array to control the growth of a CPG. When the growth increment is set to zero, the CPG does not grow automatically. If the CPG cannot grow then when the free space in the CPG is exhausted I/Os that require space to be allocated will fail.

4 Prior to 3.2.2 MU2 the limit was 256 TDVVs per CPG for all HPE 3PAR StoreServ systems.


The default and the minimum growth increment for a CPG depend on the number of nodes in the system. Table 3 lists the default growth increment and its limits for different numbers of node pairs in a system. The maximum growth increment for a CPG is 2,047.750 GB being 2 TB minus 256 MB.

Table 3. Default and limits for the growth increment per node pair

NUMBER OF NODES DEFAULT (GB) MINIMUM (GB) MAXIMUM (GB)

2 32 8 2,047.75

4 64 16 2,047.75

6 96 24 2,047.75

8 128 32 2,047.75

Considerations when selecting a CPG growth increment for a CPG include:

• The CPG growth increment can be changed at any time.

• The grow size of a TDVV is 25 percent of the CPG grow size.

• To determine the optimal growth increment for your environment, review the following factors influencing the value for it:

– With the default growth increment of 32 GB per node pair and 1 GB chunklets, the new CPG growth will be spread across 16 disks on each node.

– If there are less than 16 drives of the device type associated with the CPG connected to each node type, then consideration should be given to reducing the CPG growth increment to help minimize the amount of reserved.

– Growth space—This can be important for CPGs created on SSDs as they often account for a minority of the space within the array.

– If there are more than 16 drives of the device type associated with the CPG connected to each node type, then it may seem tempting to increase the growth increment to spread the data across more disks. This is not necessary, as the next growth will use chunks from the remaining drives. It is therefore recommended to keep the growth at the default value.

– However, if the environment is write-intensive, the rate of consumption of CPG space might be significant. In this case, it is recommended that the growth increment be set to a value above the default value listed in table 3.

Changing LD characteristics within a CPG LD space within a CPG is automatically allocated to thin volumes as needed. Storage administrators have no direct control over which LDs are used for a particular allocation. Although the capability exists to modify the LD characteristics for new LDs within a CPG, its use is advisable only in rare instances, for example, when different LD characteristics would enable the CPG to fully utilize the remaining available physical capacity. The changes to CPGs characteristics will only be reflected on newly created LDs, the layout of existing LD will not change. For this to occur the administrator must use the tunevv or tunesys commands.

Snapshot space within CPGs A thin volume can be associated at creation time or at a later stage with snapshot space, which is used to store data changed in the original volume. The space for one or more snapshots or virtual copies can originate from the same CPG as for the user space in the TV or from a different one. The CPG for the snapshot space of a TV can be changed anytime later using the tunevv command. The snapshot space grows on-demand with growth increments as defined by the CPG in which it was created. This association mitigates the planning required to estimate and dedicate capacity upfront for copy-on-write snapshots of a volume.


Allocating CPG space to thin volumes Each TPVV or TDVV associated with a CPG allocates space from the following sources in the listed order:

• Unallocated LD space in CPGs—LD space is shared by all the thin volumes associated with the CPG. This space can grow to a considerable amount as the SA and SD space of any VV associated with the CPG is returned to the CPG as unallocated LD space upon VV removal or when the free space command is issued for a volume. The returned, unallocated LDs are re-mapped to the SD space of associated TPVVs or their snapshots as needed over time. Unallocated LD space for a CPG can be displayed using the command showld -cpg <CPG>. Newly created LDs to accommodate thin volume growth will show as unused initially.

• Free chunklets—Free chunklets available to a CPG for creating new LDs may be limited by the LD creation parameters for the CPG. It is important to understand that if different CPGs can draw from the same pool of chunklets, the chunklets allocated to one CPG will reduce the pool of storage available to the other CPGs. It is recommended that storage administrators implement the following strategies to stay abreast of available free space:

– Set the CPG allocation warnings (and limits, if necessary)

– Monitor the rate of free space reduction using the command showspace -cpg <CPG> -hist

– Set the free space capacity warning with the setsys RawSpaceAlertXX command (as detailed in “allocation warnings” section later in this paper)

– Monitor available free space alerts set by the showalert command

Sizing guidelines What is the ideal size for a thin volume? There is no definite answer to this, but you should consider the following when deciding on the size for thin volumes:

• The minimum size for a thin volume is 256 MB; the maximum size is 16 TB for all HPE 3PAR OS versions on all types of HPE 3PAR StoreServ systems.

• Thin Provisioning demonstrates the highest value in situations involving large-scale consolidation. For small virtual volumes (256 MB up to a few tens of GB), the growth increment of the CPG of the TPVV may be many times higher than the TPVV size which means that minimal benefit is realized after one growth increment is applied.

• It is possible to increase the size of a thin volume. This provides a significant amount of flexibility. However, the impact of growing volumes for host-based OS needs to be considered. It is not possible to shrink a TPVV.

• When faced with a choice, it is preferable to make volumes larger than needed over making them too small. If the volumes that are presented are too small for the ongoing requirements, then TPVV growth or additional volumes will be required in the future, which is something that needs to be managed.

Thin volumes administration Over-provisioning capacity to hosts is one of the prime reasons why people choose to create thin volumes. It is therefore essential to be in control of the space use to ensure that the allocated capacity does not reach the physical capacity of the system. Natural growth or a process writing excessive amounts of data to VVs can cause a CPG or the entire array to run out of space potentially causing an interruption of service for applications consuming space from that CPG or array.

While HPE 3PAR Thin Technologies are simple to use, a certain level of management is required to help maximize its benefits. HPE 3PAR Thin Technologies offers a comprehensive set of tools that not only allows storage to start thin, but also to stay thin.

A key component of successful over-provisioning is the implementation of a space reclaim regime to leverage HPE 3PAR Thin Persistence capability so that there is a constant ebb and flow of space use and reclaim. See the HPE 3PAR Thin Persistence software section for more details.

The extensive reporting capabilities enable storage administrators to monitor space usage trends and predict future requirements while the alerting functions allow for timely interventions to remedy unexpected capacity shortages.


Allocation warnings and alerts The HPE 3PAR OS provides multiple categories of alerts that notify storage administrators of important events to help ensure the smooth running of an HPE 3PAR StoreServ. Figure 4 illustrates the categories for which warnings can be set.

Figure 4. HPE 3PAR thin volume alert options

These include:

• Allocation warnings and limits for TPVVs

• Growth warnings and limits for CPGs

• Used physical capacity alerts

• Free physical capacity alerts

Allocation warnings Allocation warnings provide a mechanism for informing storage administrators when a specific capacity threshold is reached. An allocation warning can be specified independently for each TV and each CPG. It is recommended that allocation warnings be used, at least on the CPG level, and acted upon when they are triggered.

The relevant CLI commands for setting allocation and growth warnings are:

setvv -usr_aw <percent> <TV>: sets the allocation warning for the user space of the TV as a percentage of the TV size

setvv -snp_aw <percent> <TV>: sets the allocation warning for the snapshot space of the TV as a percentage of the TV size

setcpg -sdgw <num> <CPG>: sets the growth warning for the CPG in MB (append to the value num “g” or “G” for GB or “t” or “T” for TB)

These warnings can be changed at any time and are effective immediately. The CLI commands showvv -alert and showcpg -alert lists the allocation warnings that were set per TV and CPG.


Allocation limits Applications sometimes get into an abnormal state writing data continuously to the storage device. Allocation limits provide a mechanism to prevent such “runaway” applications from consuming disk capacity beyond a specified threshold. Allocation limits can be specified independently for each TV and each CPG. For a TV, after the allocation limit is reached, the capacity allocated to the TV stops growing and new writes by the application fail. Similarly, for a CPG, after the allocation limit is reached, the automatic creation of new LDs, if configured, is disabled.

The relevant CLI commands related to setting allocation and growth limits are:

setvv -usr_al <percent> <TV>: sets the allocation limit for the user space of the TV as a percentage of the TV size

setvv -snp_al <percent> <TV>: sets the allocation limit for the snapshot space of the TV as a percentage of the TV size

setcpg -sdgl <num> <CPG>: sets the growth limit of the CPG in MB (append to the value num “g” or “G” for GB, or “t” or “T” for TB)

These alerts can be changed at any time and are effective immediately. The CLI commands showvv -alert and showcpg -alert list the allocation limits that were set per TV and CPG.

The VV allocation limits and warnings can be set with the HPE 3PAR Management Console by selecting Show advanced options checkbox when creating or editing a VV as shown in figure 5.

Figure 5. VV allocation limit and warning options

The CPG growth limits and warnings can be set with the HPE 3PAR Management Console by selecting Show advanced options checkbox when creating or editing a CPG as shown in figure 6.

Figure 6. CPG allocation limit and warning options

It is important to note that the growth limit for a CPG is a hard limit and the CPG will not grow beyond it. Once the CPG hard limit has been reached any VVs that require more space will not be able to grow. This will result in write errors to host systems until the CPG allocation limit is


raised. Therefore, it is recommended that TV, CPG, and free space warnings and limits are set to sensible levels and managed when they are triggered. As an example, the CPG warning limit should be set sufficiently below the CPG allocation limit so that it alerts the storage administrator with ample time to react before the CPG allocation limit is reached.

Remaining physical capacity alerts As available, physical capacity across the HPE 3PAR StoreServ Storage gets consumed by VVs, preconfigured alerts are generated at 50 percent, 75 percent, 85 percent, and 95 percent of physical capacity in use per drive type (FC, NL, or SSD). Furthermore, the storage administrator can use the CLI command setsys as follows to set another warning level when the available space within the system falls below a custom-defined capacity point:

setsys RawSpaceAlertFC <value>: where value is the remaining capacity on FC disks in GB

setsys RawSpaceAlertNL <value>: where value is the remaining capacity on NL disks in GB

setsys RawSpaceAlertSSD <value>: where value is the remaining capacity on SSD disks in GB

These serve as array-wide, advance warnings to the storage administrator to plan for and add necessary physical capacity. The alerts generated should be monitored and promptly acted upon to prevent all free space of a particular drive type be consumed.

Remaining CPG free space alerts HPE 3PAR OS samples the space available to CPGs once per day. The history of used and free CPG space is stored in an internal table and can be displayed using the -hist option in the showspace and showcpg commands. An alert is automatically generated if the available free space for a CPG falls below the CPG warning limit or the CPG allocation limit.

View logged warning and limit alerts All warning and limit alerts mentioned above can be viewed in several ways:

• The CLI commands showalert and showeventlog list the alerts in various formats and with various options.

• The HPE 3PAR Management Console shows the alerts in the “Events” section.

• Storage Management Initiative software (SMI-S) integrated in HPE 3PAR OS provides asynchronous notification of events for changes in the elements managed by the Common Information Model (CIM) server. A CIM client can subscribe to selected CIM indications to receive event notifications from the CIM server.

• The SNMP agent within HPE 3PAR OS allows for retrieving the alerts by remote SNMP clients.

Alerts can be forwarded (setsys RemoteSyslogHost) to a log host for viewing them in an enterprise management application.

User recommendations The monitoring of alerts for available capacity by storage administrators and internal business processes are a critical component of a successful HPE 3PAR Thin Provisioning management and administration strategy. You should nominate a primary and if possible a backup storage administrator for each site with HPE 3PAR StoreServ equipment. The storage administrator’s roles include:

• Proactively monitor free space availability per TV and CPG.

• Proactively monitor consumption rates for TVs and CPGs.

• Proactively monitor consumed TV capacity and compare to licensed thin provisioning capacity.

• Proactively monitor physical capacity thresholds for each disk type and for the entire array.

• Ensure adequate purchasing and installation of additional physical disk capacity buffer and thin-provisioning license upgrades in a timely manner.

• Nominate an escalation contact who has proper authority to drive the customer responsibilities outlined in this document if the nominated storage administrators fail to carry out their responsibilities.


If you have a network connection with HPE 3PAR Central via the Service Processor, the health of the HPE 3PAR StoreServ can be proactively monitored for CPG growth problems, and you can request to receive thin provisioning and other alerts by mail or via phone. You retain responsibility for managing the thin-provisioning capacity and CPGs; Hewlett Packard Enterprise is not responsible for any failure when thresholds are met or exceeded.

Capacity efficiency HPE 3PAR OS 3.2.1 MU1 introduces two metrics to measure the capacity efficiency of the HPE 3PAR Thin Technologies, compaction ratio, and dedup ratio. The compaction ratio is how much physical storage space a volume consumes compared to its virtual size and applies to both thin provisioned and thinly deduped volumes. The dedup ratio is how much physical storage space would have been used without deduplication, compared to the actual storage space used by a thinly deduped volume. The ratios are shown as decimals and have an implied: 1 i.e., 4.00 is actually 4:1 (4 to 1). The dedup ratio does not include savings from inline zero-detection.

These capacity efficiencies can be shown per volume, volume family, CPG, virtual domain or system and are in terms of usable storage (i.e., not including RAID overhead). The efficiencies displayed will depend on the scope of the request. For example, it is likely that the dedup ratio of a CPG will be higher than that of the individual TDVVs because the data in the Dedup Store can be shared by multiple TDVVs. Note that the capacity efficiency ratios are not dynamically updated and therefore changes to the data may not be immediately reflected in the capacity efficiency ratios displayed.

Base volumes The capacity efficiencies of a base volume are shown by the showvv -space command and are calculated as follows:

• The compaction ratio of a TPVV is the virtual size of the volume divided by its used data space.

• The compaction ratio of a TDVV is the virtual size of the volume divided by the sum of its used data space and Hewlett Packard Enterprise used Dedup Store space.

• The dedup ratio of a TDVV is the size of the data written to the TDVV divided by the sum of the data stored in the TDVV and the data associated with the TDVV in the Dedup Store.

In this following example, vv1 is a TPVV that has virtual size of 100 GB and has 10 GB of data written to it, vv2 is a TDVV that has virtual size of 100 GB and has 10 GB of non-deduplicable data written to it, and vv3 is a TDVV that has virtual size of 100 GB and has two copies of the vv2 data written to it.

cli% showvv -space vv1 vv2 vv3

---Adm---- --------Snp---------- ----------Usr-----------

---(MB)--- --(MB)-- -(% VSize)-- ---(MB)---- -(% VSize)-- ---------(MB)----------- -Capacity Efficiency-

Id Name Prov Type Rsvd Used Rsvd Used Used Wrn Lim Rsvd Used Used Wrn Lim Tot_Rsvd VSize Compaction Dedup

483 vv1 tpvv base 256 9 0 0 0.0 -- -- 12800 10240 10.0 0 0 13056 102400 10.0 --

485 vv2 tdvv base 5184 4428 0 0 0.0 -- -- 13056 5129 5.0 0 0 18240 102400 10.7 1.0

486 vv3 tdvv base 5184 4433 0 0 0.0 -- -- 13056 5129 5.0 0 0 18240 102400 10.7 2.0

------------------------------------------------------------------------------------------------------------------------------

3 total 10624 8870 0 0 38912 20498 49536 307200 10.5 1.5

Note that the capacity efficiencies in the total row are averages of the volumes displayed.

The Adm Used and Usr Used values displayed for each TDVV are an estimation of contribution of the volume to the Dedup Store. Both vv2 and vv3 share the same data and therefore the Usr Used space is divided between them (10 GB/2 = 5 GB).

The base volume savings can also be seen in the Management Console by selecting the particular vv. Figure 7 shows the space savings displayed by the HPE 3PAR Management Console (IMC) for vv3. Note that the virtual volume Allocation shows the reserved space usage (i.e., the number regions allocated to the VV), whereas the Used total in Capacity Efficiency shows the in use space (i.e., number of pages in use).


Figure 7. Virtual volume space savings in IMC

Figure 8 shows the space savings displayed by the HPE 3PAR StoreServ Management Console (SSMC) for vv3.

Figure 8. Virtual volume space savings in SSMC

Volume families If a volume has snapshots then the showvv command will display the individual snapshots below their parent volumes but the capacity efficiency displayed is that of the entire volume family because all the snaps of a base volume share the same snapshot area. If a block changes on a base volume, the old data is copied to the snapshot area for that volume and all snaps of the volume point to that single block. This allows a volume to have hundreds of snaps without requiring additional space or incurring additional performance impact.

The capacity efficiencies of a volume family are shown by the showvv -space command and are calculated as follows:

• The compaction ratio of a TPVV volume family is the sum of the virtual size of the volume and the virtual volume sizes of all its snapshots divided by the used data space.

• The compaction ratio of a TDVV volume family is the sum of the virtual size of the volume and the virtual volume sizes of all its snapshots divided by the sum of the used data space and the used Dedup Store space of the volume and its snapshots.

• The dedup ratio a TDVV volume family is the sum of the data written to the TDVV and all its snapshots divided by the sum of the data stored in the TDVV and the Dedup Store for the volume and its snapshots.


In this example, snapshots were created from the previous VVs and 1 GB of data was changed on vv1 and 100 MB of data was changed on vv3. The result is the compaction ratio for the vv2 volume family is twice that of the vv2 base volume and the compaction ratios of the other volume families have increased but not by as much due to the change in the data.

cli% showvv -space vv*

---Adm---- ---------Snp---------- ----------Usr-----------

---(MB)--- --(MB)--- -(% VSize)-- ---(MB)---- -(% VSize)-- -----(MB)------ -Capacity Efficiency-


483 vv1 tpvv base 256 10 2560 1000 1.0 0 0 12800 10240 10.0 0 0 15616 102400 18.2 --

489 vv1_snp snp vcopy -- *0 -- *0 *0.0 0 0 -- -- -- -- -- -- 102400 -- --

485 vv2 tdvv base 5184 4465 512 0 0.0 0 0 13056 5170 5.0 0 0 18752 102400 21.3 1.0

491 vv2_snp snp vcopy -- *0 -- *0 *0.0 0 0 -- -- -- -- -- -- 102400 -- --

486 vv3 tdvv base 5208 4492 512 0 0.0 0 0 13087 5196 5.1 0 0 18807 102400 21.1 2.0

493 vv3_snp snp vcopy -- *0 -- *0 *0.0 0 0 -- -- -- -- -- -- 102400 -- --

--------------------------------------------------------------------------------------------------------------------------------

6 total 10648 8967 3584 1000 38943 20606 53175 614400 20.1 1.5

Note that the space usage columns for the snapshots contain “—” as the space usage of the snapshots is maintained in the base volume.

Common provisioning groups A CPG with multiple volumes may have a higher dedup ratio than that of each TDVV in the CPG because the Dedup Store pages may be shared by more than one volume (in addition to sharing within a volume). It is therefore a more accurate reflection of the space savings than shown by the virtual volume dedup ratios.

The capacity efficiencies of a CPG are shown by the showcpg -space command and are calculated as follows:

• The compaction ratio is the sum of the virtual sizes of all the volumes and snapshots in the CPG (CPVV, TPVV, and TDVV) divided by the sum of their in use data space, snapshot space, and used Dedup Store space.

• The dedup ratio is the sum of all the data written to the TDVVs and TDVV snapshots of a CPG divided by the size of the data stored in the Dedup Store of the CPG and the sum of the data stored in the TDVVs of the CPG.

In this example, the TPVV_CPG CPG contains only TPVVs so it only has a compaction ratio, whereas the TDVV_CPG CPG contains TDVVs so it also has a dedup ratio.

cli% showcpg -space

---------------(MB)-------------------------

--- Usr --- -- Snp --- --- Adm --- - Capacity Efficiency –

Id Name Warn% Total Used Total Used Total Used Compaction Dedup

0 TDVV_CPG - 26112 26112 10752 0 24576 10368 10.7 3.0

1 TPVV_CPG - 12800 12800 11776 0 8192 256 10.0 -

-----------------------------------------------------------------------------------------

2 total 38912 38912 22528 0 32768 10624 10.5 3.0

The base volume savings can also be seen in the Management Console by selecting the particular CPG. Figure 9 shows the space savings displayed by the HPE 3PAR Management Console (IMC) for TDVV_CPG.


Figure 9. CPG space savings in IMC

Figure 10 shows the space savings displayed by the HPE 3PAR StoreServ Management Console (SSMC) for TDVV_CPG.

Figure 10. CPG space savings in SSMC

Virtual domains The capacity efficiencies of a virtual domain are shown by the showsys -domainspace command and are calculated as follows:

• The compaction ratio is the sum of the virtual sizes of all the volumes in the virtual domain (CPVV, TPVV, and TDVV) divided by the sum of the used data space and the used snapshot space of all CPGs in the virtual domain.

• The dedup ratio is the sum of all the data written to the TDVVs and TDVV snapshots in the virtual domain divided by the sum of the space used by the TDVVs, the TDVV snapshots and the Dedup Stores in the virtual domain.

In this example, there is an additional dom1 virtual domain which contains just TPVVs.


System The capacity efficiencies of the system are shown by the showsys -space command and are calculated as follows:

• The compaction ratio is the sum of the virtual sizes of all the volumes in the system (CPVV, TPVV, and TDVV) divided by the sum of the used data space and the used snapshot space of all CPGs.

• The dedup ratio is the sum of all the data written to the TDVVs and TDVV snapshots in the system divided by the sum of the space used by the all TDVVs, the TDVV snapshots and the Dedup Stores.

This example shows how the system wide capacity efficiencies are displayed.

cli% showsys -space

------------- System Capacity (MB) -------------

Total Capacity : 20054016

Allocated : 1542144

Volumes : 385024

Non-CPGs : 0

User : 0

Snapshot : 0

Admin : 0

CPGs (TPVVs & TDVVs & CPVVs) : 385024

User : 136192

Used : 136192

Unused : 0

Snapshot : 125952

Used : 1024

Unused : 124928

Admin : 122880

Used : 33024

Unused : 89856

Unmapped : 0

System : 1157120

Internal : 321536

Spare : 835584

Used : 0

Unused : 835584

Free : 18511872

cli% showsys -domainspace

--------------CPG(MB)---------------

-Non-CPG(MB)- -----Usr----- ----Snp---- ----Adm----- -----(MB)------ -Capacity Efficiency-

Domain Usr Snp Adm Total Used Total Used Total Used Unmapped Total Compaction Dedup

- 0 0 0 0 0 0 0 0 0 0 0 0.0 -

dom0 0 0 0 102400 102400 94208 1024 98304 32256 0 294912 10.7 3.0

dom1 0 0 0 33792 33792 31744 0 24576 768 0 90112 10.0 -

---------------------------------------------------------------------------------------------------------------

0 0 0 136192 136192 125952 1024 122880 33024 0 385024 10.3 3.0


Initialized : 18511872

Uninitialized : 0

Unavailable : 0

Failed : 0

------------- Capacity Efficiency --------------

Compaction : 10.3

Dedup : 3.0

Figure 11 shows the system wide space savings displayed by the HPE 3PAR StoreServ Management Console (SSMC).

Figure 11. System space savings

Manually updating the capacity efficiency statistics Since the capacity efficiency ratios are not dynamically updated recent changes to the data may not be immediately reflected in the capacity efficiency ratios displayed. It is possible to manually initiate an update of the statistics using the updatesnapspace -dedup_stats command with the argument of a TDVV name. Even though one TDVV name is specified the capacity efficiency ratios all the volumes in the CPG will be updated as well as capacity efficiency ratios of the CPG.

The following example shows how to initiate the manual statistics update.

cli% showvv -space tdvv*

---Adm---- ---------Snp---------- ----------Usr-----------



130 tdvv_1 tdvv base 256 52 0 0 0.0 -- -- 2560 0 0.0 0 0 2816 102400 1947.6 0.0

136 tdvv_2 tdvv base 256 52 0 0 0.0 -- -- 2560 0 0.0 0 0 2816 102400 1934.4 0.0

--------------------------------------------------------------------------------------------------------------------------------

2 total 512 104 0 0 5120 0 5632 204800 1940.9 0.0

cli% updatesnapspace -dedup_stats tdvv_1

Task “dedup_stats_update TDVV_CPG” has been started.

cli% showvv -space tdvv*

---Adm---- ---------Snp---------- ----------Usr-----------



130 tdvv_1 tdvv base 17125 16650 0 0 0.0 -- -- 15105 5277 5.2 0 0 32230 102400 4.7 3.0

136 tdvv_2 tdvv base 8859 8518 0 0 0.0 -- -- 8959 2691 2.6 0 0 17818 102400 9.1 1.2

--------------------------------------------------------------------------------------------------------------------------------

2 total 25984 25168 0 0 24064 7968 50048 204800 6.2 2.4


The capacity efficiency ratios can also be updated from the HPE 3PAR StoreServ Management Console (SSMC) by selecting the Refresh capacity efficiency action for a CPG as shown in figure 12.

Figure 12. Refreshing capacity efficiency

Understanding capacity efficiency ratios It may not be immediately apparent that even low capacity efficiency ratios indicate significant space savings. As capacity efficiency ratios increase, there are diminishing returns in terms of space savings. The percentage of space reduction obtained is 100 percent less the inverse of the capacity efficiency ratio. Space savings for selected capacity efficiency ratios are shown in table 4.

Table 4. Space savings by capacity efficiency ratio

CAPACITY EFFICIENCY RATIO SPACE REDUCTION PERCENTAGE

1:1 0%

1.5:1 33%

2:1 50%

4:1 75%

10:1 90%


Tracking volume space usage Thin volumes consume user space, admin space, and possibly snapshot space on the disk array. The following sections provide the CLI commands needed to determine how much space of every type is being consumed within the array. The output of these CLI commands shows the reserved and the raw reserved space. The reserved space is what is offered by the array as usable space to the host. This value is also shown in the HPE 3PAR Management Console in the Reserved User Size column for a TPVV and in the pie chart for the Logical option in the Summary tab for the virtual volume details screen. The raw reserved space is calculated from the reserved space by multiplying the latter by its RAID overhead factor. For example, this factor has a value of 2 for RAID 1 and 8/7 for RAID 5 with a set size equal to 8. The HPE 3PAR Management Console shows the Raw Reserved space in the pie chart for the Raw option in the Summary tab for the virtual volume details. In chargeback models, most IT departments bill their customers on the amount of raw reserved space consumed.

TPVVs Use the showvv -s -p -prov tpvv command to see how much admin, user, and snapshot space is used by each TPVV. The reserved totals show how much space has been allocated, whereas the used totals show how much of the space is currently in use by the VV. A significant difference between the space in use and the reserved space would indicate that space reclaim has been initiated on the VV, and the reserved space will decrease over time as the space is reclaimed in the background. This is an example of the showvv -s output:

cli% showvv -s -p -prov tpvv

---Adm---- ---------Snp---------- ----------Usr-----------



370 vv1 tpvv base 256 16 0 0 0.0 -- -- 25088 21582 21.1 0 0 25344 102400 4.7 --

371 vv2 tpvv base 256 40 0 0 0.0 -- -- 66048 61642 60.2 0 0 66304 102400 1.7 --

372 vv3 tpvv base 256 47 0 0 0.0 -- -- 74240 73378 71.7 0 0 74496 102400 1.4 --

373 vv4 tpvv base 256 28 0 0 0.0 -- -- 47616 41062 40.1 0 0 47872 102400 2.5 --

--------------------------------------------------------------------------------------------------------------------------------

4 total 1024 131 0 0 212992 197664 214016 409600 2.1 --

The same information is displayed in the HPE 3PAR Management Console when viewing the VV provisioning as shown in figure 13.

Figure 13. VV allocation sizes

CPGs Space in use on the array can be tracked per CPG. The showcpg -r command shows the user, snapshot, and admin space in Used and Raw Used amounts. You can work out the unallocated space within the CPGs by subtracting the used space from the Totals listed.

In addition to showing the CPG usage, the showspace -cpg command will also show how much LD space may still be created, given the amount of free chunklets in the system and the CPG parameters (e.g., RAID level, HA level, device types, etc.).


cli% showspace -cpg *

------------------------------(MB)----------------------------

CPG -----EstFree------- --------Usr------ -----Snp---- ----Adm---- -Capacity Efficiency-

Name RawFree LDFree Total Used Total Used Total Used Compaction Dedup

TPVV_CPG 18499584 9249792 16896 16896 15872 512 8192 256 10.0 -

TDVV_CPG 18499584 9249792 34304 34304 31232 0 24576 10496 10.7 1.6

FC_r5 176078848 154068992 21951616 21951616 104320 3072 32768 8480 13.1 -

NL_r6 54263808 40697856 27969280 27969280 96512 69632 23552 21248 8.3 -

System space Not all space on the physical disks is used for storing your data. A small portion of the space on the array is dedicated to volumes with an administrative function.

There is a fully provisioned volume named admin that is used to store system administrative data such as the System Event Log. The logging LDs, starting with the name log, are used to store data temporarily during physical disk failures and disk replacement procedures. There are also preserved data space logical disks (PDSLDs) which start with the name pdsld. Preserved data is the data moved from the system’s cache memory to the PDSLD space in the eventuality of multiple disk or cage failures. On HPE 3PAR StoreServ systems running HPE 3PAR OS 3.1.2, the HPE 3PAR System Reporter software is integrated into the OS and executed on the controller nodes, and the database files are stored in a fully provisioned volume called .srdata.

The amount of raw disk space consumed by these system functions is summarized in table 5.

Table 5. System space usage5

NUMBER OF NODES LOGGING LDS ADMIN LDS PRESERVED DATA6 SRDATA

2 240 GB 20 GB 128 GB 60 GB

4 480 GB 20 GB 256 GB 80 GB

6 720 GB 20 GB 384 GB 100 GB

8 960 GB 20 GB 512 GB 100 GB

Total used space The showsys -space command gives the total raw space in use on the system with a breakdown of how the space is allocated. This is an example of showsys output:

cli% showsys -space

------------- System Capacity (MB) -------------

Total Capacity : 20054016

Allocated : 1542144

Volumes : 385024

Non-CPGs : 0

5 These are maximum values. The exact values will vary depending on the HPE 3PAR StoreServ model and disk configuration. 6 The total capacity of the PDSLDs is equal to the sum of all data cache memory located in the controller nodes of the system.


User : 0

Snapshot : 0

Admin : 0

CPGs (TPVVs & TDVVs & CPVVs) : 385024

User : 136192

Used : 136192

Unused : 0

Snapshot : 125952

Used : 1024

Unused : 124928

Admin : 122880

Used : 33024

Unused : 89856

Unmapped : 0

System : 1157120

Internal : 321536

Spare : 835584

Used : 0

Unused : 835584

Free : 18511872

Initialized : 18511872

Uninitialized : 0

Unavailable : 0

Failed : 0

------------- Capacity Efficiency --------------

Compaction : 10.3

Dedup : 1.6

The value in the System section of the output is the raw space in use by the admin and the .srdata volume, the PDSLDs, the logging LDs, and the spare chunklets. The internal entry lists the space associated with the local boot disks of the nodes so this should be treated separately from the space allocated from the regular drives.

Usage reporting and trend analysis The CLI command showcpg -hist <CPG> gives a daily account of CPG usage split into user, snapshot, and admin space. The command showspace -cpg <CPG> -hist also shows this information.

You can also use the srcpgspace and srvvspace commands to query the internal System Reporter database.7 Additionally, the optional HPE 3PAR System Reporter software has the ability to track the CPG and TV usage for comprehensive usage and trend analysis.

7 The System Reporter functionality has been included in HPE 3PAR OS 3.1.2. Prior versions of HPE 3PAR OS will require the optional HPE 3PAR System Reporter software on an

external host.


The following example shows the output of a srvvspace command for a CPG.

cli% srvvspace -hourly -btsecs -1h -usr_cpg TDVV_CPG

------RawRsvd(MB)------ ----User(MB)----- ------Snap(MB)------

Time Secs User Snap Admin Total Used Free Rsvd Used Free Rsvd Vcopy

2014-10-16 23:00:00 CEST 1413493200 68608 0 31488 100096 10244 24060 34304 0 0 0 0

------Admin(MB)------- ---------------Total(MB)--------------- -Capacity Efficiency-

Used Free Rsvd Vcopy Vcopy Used Rsvd VirtualSize Compaction Dedup

8896 1600 10496 0 0 19140 44800 67313664 10.0 1.5

Running out of space The following sections describe the severity of the situation when space limitations are met, from the least critical (for example, running out of space on a single TV) to the more severe (for example, running out of space on a CPG or on the entire system).

Running out of space in a TV If an allocation limit was set for a TV, the array stops all writes to that TV after the limit is reached-returning SCSI error codes if more writes come in. Depending on the OS and application, the effect of this can range from the host application stalling to the host server crashing. Reaching a TV limit purely affects that volume and its resident application(s).

TV limits can be changed at any time with the setvv command. After an appropriate change, writes to the TV continue to be processed.

Running out of space in a CPG Reaching a CPG limit has the same effect as reaching a TV limit; however, many more TVs and other types of VVs may be impacted. CPG limits can be changed at any time with the setcpg -sdgl command. After an appropriate change, writes to the volumes in the affected CPG continue to be processed.

It is also possible for a CPG to run out of space if there are not enough free chunklets of the selected device type to complete a grow. In such a circumstance the CPG will attempt to complete the grow by using a lower availability level (i.e., HA MAG instead of HA CAGE) and if that is not possible it will use chunklets from another tier. The system will report that the CPG was grown with degraded parameters and once the space shortage has been corrected it is important to run the tunesys command to correct the degraded grow.

Running out of space on the system Running out of space on the system will affect TVs and all other types of volumes being written to in all CPGs. In the unlikely scenario that all physical capacity becomes used, the system prevents new writes from occurring until more capacity is added.

Reclaiming unused space The HPE 3PAR OS space consolidation features allow you to change the way that virtual volumes are mapped to logical disks in a CPG. Deleting TVs or moving virtual volume regions from one logical disk to another enables you to compact logical disks, and free up disk space so that it can be reclaimed for use by the system.

Mapping is the correspondence of logical disk regions to the virtual volume. Virtual volumes are made up of multiple logical disks, and each logical disk contains regions that are mapped to the virtual volume. All types of volumes are created by mapping data from one or more logical disks to the virtual volume.

Logical disks can be shared by multiple virtual volumes or snapshots. As volumes are deleted or as volume copy space grow and then shrink, logical disks can use space less efficiently. When logical disks do not efficiently use space, the unused space consumes regions on the LD that are not available for use by the system when creating new logical disks. The space management features enable you to consolidate used space onto fewer fully used logical disks so that unused regions are forced onto one or more logical disks that are then deleted. Deleting these logical disks frees the unused space for general use by the system. You can also truncate LDs to free up space. The LD’s used regions are compacted by moving them to the beginning of the LD, and then the LD is truncated so that unused space can be returned to the system’s free chunklet pool.


Reclaiming unmapped logical disk space from CPGs CPGs provide a shared pool of logical disk capacity for use by all virtual volumes that draw space from that pool if volumes that draw from a CPG are deleted, or if copy space for these volumes grows and then shrinks, the underlying logical disks in the CPG pool can become less efficient in space usage. One or more logical disks in the CPG pool may have only a small portion of their regions mapped to existing virtual volumes. However, the logical disk’s unused regions are only available for use by the volumes mapped to the CPG.

It is possible to compact the logical disk regions mapped to these volumes to recover and free logical disk space that can be returned to the free space pool for reuse by other CPGs. Compacting a CPG allows you to reclaim space from a CPG that has become less efficient in space usage from creating, deleting, and relocating volumes. Compacting consolidates logical disk space in CPGs into as few logical disks as possible. Compacting CPGs can be performed with both the HPE 3PAR CLI and the HPE 3PAR Management Console.

By default, the compactcpg command will defragment the LDs owned by the CPG and then return entire unused LDs back into the free spacepool. There is also a -trimonly option that causes compactcpg to simply check for unused LDs and return them to the free space poolwithout running a defragmentation.

When using thin provisioning, it is recommended to schedule regular CPG compactions during periods of low activity. On larger systems, it is advisable to stagger the compaction of the CPGs to reduce the load on the system.

Note If using Adaptive Optimization on all the CPGs, automatic CPG compaction is a configurable option of Adaptive Optimization. It is recommended to run the startao command with the -compact trimonly option to defer the LD defragmentation until the scheduled CPG compaction runs.

Automatically reclaiming unused space from volumes The HPE 3PAR OS automatically reclaims unused user space from thin volumes and returns the space to the LDs. The system examines the shared space for large areas of unused space. The identified areas are unmapped from the corresponding LD regions, and the space is returned to the LDs.

Deleted volume snapshot space HPE 3PAR Thin Copy Reclamation is an optional feature that reclaims space when a snapshot of a TPVV, virtual copy, full copy, or remote copy volume is deleted from the system. The deleted snapshot space is reclaimed from either a standard or a thin-provisioned volume and is automatically returned to its CPG for reuse by other volumes. The HPE 3PAR Thin Copy Reclamation feature works on all HPE 3PAR StoreServ and requires that an HPE 3PAR Thin Provisioning software license be present on the array.

Logical disks and chunklet initialization After logical disks are deleted or space is reclaimed by the Thin Persistence software, the underlying chunklets are reinitialized before their space is available to be used by other LDs. The initialization process for chunklets happens in the background and at a low priority so as not to impact system performance. To see chunklets that are currently in the process of being initialized, issue the showpd -c command. Chunklets that areuninitialized are listed in the Uninit column.

Free page consolidation As individual 16 KiB pages are freed the space in the 128 MiB regions of the LDs can become fragmented, and therefore the HPE 3PAR OS implements a defragment process similar to what an OS may run on a file system. There is a thread that periodically examines the VVs to determine whether defragmentation should be initiated. The data from partially allocated regions is then reorganized to create larger contiguous blocks of free space that can be reclaimed.


HPE 3PAR Thin Persistence software Traditionally when data is deleted on a host, the OS will report that space has been freed, but the storage is not informed that the data is no longer in use. With fully provisioned volumes, this is not an issue but with thin-provisioned volumes, the unused space will remain allocated on the array causing the volumes to grow over time. This creates a hidden utilization penalty that can significantly reduce the space savings of thin provisioning.

HPE 3PAR Thin Persistence software is able to maintain the benefits of HPE 3PAR Thin Provisioning software and HPE 3PAR Thin Conversion software by enabling “thin reclamation” of allocated but unused capacity so the thin volumes are able to stay as lean and efficient as possible. It leverages the HPE 3PAR OS support for the WRITE_SAME and UNMAP commands of the T10 SCSI Primary Commands SPC-3 standard and the unique zero-detection capabilities of the HPE 3PAR ASIC to give HPE 3PAR StoreServ Storage the power to reclaim unused space associated with deleted data simply, quickly, and non-disruptively.

UNMAP is a SCSI command that a host can issue to tell the storage that blocks are no longer need to be allocated. This is particularly important in thinly provisioned environments as it allows the storage array to recognize that these blocks are not used and to return them to the free capacity pool for reuse by other volumes.

The HPE 3PAR ASIC features an efficient, silicon-based zero-detection mechanism. This unique hardware capability gives HPE 3PAR StoreServ Storage the ability to remove allocated but unused space as small as 16 KiB on the fly without impacting performance.

In addition, the benefits of HPE 3PAR Thin Persistence are available to read/write snapshots of TPVVs. The mechanism for initiating reclamation is the same as for the parent TPVV: writing zeros to the allocated but unused space in a read/write snapshot will trigger the ASIC to initiate reclamation of the deleted space. To benefit from thin reclamation, the zero-detect policy needs to be enabled on each read/write snapshot.

Thin Persistence reclamation Thin Persistence reclamation occurs at several levels. Initially all freed 16 KiB pages are returned to the TPVV. This means that on a file system that supports automatic reclaim the spaced freed by an UNMAP after a file deletion is immediately available for reuse by a file creation or file extension operation on the same file system.

To make space available for reuse by other volumes, there is a reclaim thread that returns freed 128 MiB regions allocated to a TPVV back to the CPG. This thread scans volumes every five minutes for SD space that potentially can be reclaimed. If a TPVV has free 128 MiB regions or there is enough free space to warrant a defragmentation of the volume then space will be reclaimed back to the CPG. Defragmentation occurs if there is more than 1 GB of available space to reclaim and results in free 128 MiB regions. Up to 16 volumes at a time can be queued for reclaim processing.

How quickly the space is reclaimed depends on a number of factors. If there is a large amount of freed space on a volume, then this may not be processed within a single reclaim period and once the reclaim process runs on a TPVV, the reclaim process will not run again on that TPVV again for at least 90 minutes. Therefore, large space reclaims can take several hours to complete.

In addition, the reclamation on a TPVV can be deferred for various reasons. For example, if the SD space of a TPVV is grown, then reclaims on the volume will be deferred for 60 minutes. Also, if reclaim is defragmenting a TPVV and the defragmentation does not complete during the reclaim interval further reclaim will be deferred for 8 hours.

Thin Persistence reclamation may not reclaim all the free space on a volume. There is a 4 GB per node threshold below which the TPVV will not be inspected for available 128 MiB regions that can be reclaimed back to the CPG. The free space will still be available for reuse by the TPVV.

Those new to Thin Provisioning often like to verify Thin Persistence reclamation by creating a test scenario of filling a file system then deleting the files and running a space reclamation tool. It is important to understand that the space will not be returned to the CPG immediately. The showvv -s command will show how much space has been allocated to the TPVV and the difference between the space in use and the reserved space shows the amount of space reclaimed for use within the TPVV. The amount of reserved space will decrease over time as the space is reclaimed back to the CPG in the background by the reclaim thread.


Thin Persistence and deduplication From a host perspective, the methods used to initiate space reclaim on TDVVs are the same as those on TPVVs (i.e., zerofiles or UNMAP) but internally there are differences in the way space reclaim operates. With TDVVs, it is not necessary to have a zero-detection mechanism to scan all incoming I/Os as all zero blocks will be reduced to a single zero block by the deduplication engine. However, the original pages in the Dedup Store cannot be removed as they may be in use by other TDVVs. This is also the case for space reclaimed by the UNMAP command.

HPE 3PAR Thin Deduplication uses an online garbage collection process that periodically checks for data in a Dedup Store that is not referenced by any TDVVs in that CPG.

When reclaim is initiated on a TDVV the metadata for the pages being freed are pointed to the deduped zero block. The garbage collection process then scans all the TDVVs belonging to the same CPG and builds a list of hashes being referenced. This is then compared with the hashes in the Dedup Store and if any pages are no longer referenced then they are marked as free.

Once the garbage collection has completed the SD space associated with the TDVVs and Dedup Store is processed by the normal Thin Persistence reclaim thread and the space is returned to the CPG.

Thin Persistence methods The most optimal Thin Persistence method is for the host OS to issue an UNMAP for the unwanted blocks when the data is deleted. The most optimal Thin Persistence method is for the host operating system to issue SCSI UNMAP commands for unwanted data blocks. Typically, this would be done by the file system when files are deleted. However, if the operating systems offers a suitable UNMAP application programming interface (API), an application can directly communicate to the HPE 3PAR StoreServ system that it no longer needed some data. This method provides continual reclaiming of space to allow the storage volumes to stay thin. The disadvantage is it requires significant changes to the storage stack and only the most modern OSs have implemented native UNMAP support.

Another method is to have an OS utility that will examine the blocks of a volume and issue UNMAPs for those that are not being used. This type of utility also requires host OS UNMAP support but to a lesser extent the continual method, and they are specific to a file system or volume manager type. Most of these utilities can be run when the data is online but as they generate the UNMAP requests for all the unused blocks in one go they are generally run manually during an outage window or scheduled to run during a quiet period so the reclaims do not adversely impact other workloads on the storage.

The final method is to reclaim space using a “zerofile” utility that writes zeros to all allocated but unused space on a file system. On HPE 3PAR StoreServ systems, the zero-detection capability of the HPE 3PAR ASIC intercepts the blocks of zeros being written and automatically triggers the reclamation of the space. The advantage of this method is that it does not require any special OS support, and the utilities to generate zerofiles are often supplied with the base OS distribution. To achieve good reclaim rates, the utilities need to fill the majority of the available free space so they are generally run manually during an outage window or scheduled to run during a quiet period to avoid applications failing due to a lack of space.

For the zerofile utilities to work the zero-detect policy needs to be set for each TPVV. Blocks of 16 KiB of contiguous zeros are freed and returned for reuse by the VV; if 128 MiB of space is freed, it is returned to the CPG for use by other volumes.

For more information on Thin Persistence methods for various operating systems, see Appendix B.

Thin Deduplication with snapshots and clones Volumes in HPE 3PAR OS can have read-only or read-write snapshots also known as virtual copies. The storage for the base volume and the storage for the snapshot volume can potentially come from different CPGs. When the user creates a TDVV, if a snapshot CPG is specified which is of dedup CPG type, then any snapshot data stored in the Dedup Store associated with that CPG will be deduplicated.

If the snapshots can have a different CPG to the TDVV then the deduplicated data will still reside in the CPG associated with the TDVV. Only collision data will reside in the snapshot CPG.

A clone, also known as a Physical Copy, from a source volume of any type can also be created as a TDVV.


Thin Deduplication with Remote Copy HPE 3PAR Thin Deduplication is fully supported as either a source or destination with Remote Copy, however, data transmission is not dedup aware:

• In synchronous replication mode data is immediately sent to the secondary site and since deduplication happens post acknowledging to the host and before flushing data to the backend all data is transmitted over the wire. The synchronous replication process is zero-detect aware, as 16 KiB pages of zeroes are detected before being transmitted and only 64 bytes of metadata are transmitted over the wire. This behavior is also applicable to HPE 3PAR Peer Persistence as it uses synchronous replication.

• In asynchronous replication mode data is stored in snapshots and then read from the secondary volume when the period is due. Snapshots of the primary volume are dedup aware, however, data will be rehydrated before being sent over the wire.

The replication can be between volumes of different types on different tiers. For example, a thinly deduplicated volume on SSD drives on a primary array can be in a remote copy relationship with a thin provisioned volume on NL drives on a secondary array.

While deduplicate data is rehydrated HPE 3PAR Remote Copy offers advanced data transmission techniques to optimize bandwidth utilization, for more information please refer to the HPE 3PAR Remote Copy white paper.8

Dynamic Optimization of virtual volumes Traditionally, data migrations associated with a storage technology refresh have carried forward the poor utilization rates from one system to the other. Administrators had to procure at least the same amount of capacity on a new array as on the legacy systems from which data was being migrated, even if the migration involved a large amount of allocated but unused capacity.

HPE 3PAR StoreServ storage offers several products that can be used for service-level optimization.

HPE 3PAR Dynamic Optimization is a powerful software feature that enables storage administrators to perform several online volumes optimizations.

• Perform online data movement of existing volumes to different RAID levels and different tiers of storage. For example, application that requires high performance only during certain windows can be tuned to RAID 1 on SSDs and during lower activity periods moved to more cost-effective RAID 6 storage on Nearline disks.

• Convert existing volumes to a different volume type. For example, thinly provisioned volumes on a HDD CPG to a thinly deduped volume on an SSD tier. Or the conversion of a full volume to a thinly provisioned volume. This conversion happens within the HPE 3PAR StoreServ Storage system transparently and non-disruptively.

This “thinning” of volumes enables data centers to meet green IT targets, reduce wasted space, and increase utilization rates. Because the fat-to-thin conversion mechanism is built into the array hardware, volume conversions take place inline and at wire speeds, while preserving service levels, and without causing disruption to production workloads.

The conversion requires the HPE 3PAR Dynamic Optimization Software license.

8 h20195.www2.hp.com/V2/GetPDF.aspx%2F4AA3-8318ENW.pdf

http://h20195.www2.hp.com/V2/GetPDF.aspx%2F4AA3-8318ENW.pdf


Migrate between full and thin volumes on the same array All HPE 3PAR StoreServ systems running HPE 3PAR OS 3.1.2 and above have the ability to make a convert from a thin-provisioned volume to a fully provisioned volume (or vice versa) without requiring an offline transition.9 HPE 3PAR OS 3.2.1 MU1 adds the ability to convert to a Thin Deduplication volume.

The tunevv command is used to convert between full provisioned, thin provisioned and Thin Deduplication virtual volumes without requiring an offline transition. The convert virtual volume operation from the HPE 3PAR Management Console or the tunevv command can used to perform the conversion. In the following example, the virtual volume vol01 is moved to the FC_r5 CPG and converted to a TPVV:

cli% tunevv usr_cpg FC_r5 -tpvv vol01

To convert the volume to a Thin Deduplication volume in the SSD_r5 CPG the following command would be used:

cli% tunevv usr_cpg SSD_r5 -tdvv vol01

When the -tpvv, -tdvv or -full options for the usr_cpg subcommand are specified, the tune will automatically rollback on a failure. These options do not support virtual volumes with remote copy. These options will only convert virtual volumes using snapshots if the -keepvv option is used, but the snapshots will reside in the virtual volume specified by the -keepvv option.

During the thin conversion, the HPE 3PAR ASIC will assist in reducing the amount copied by using its zero-detect capability to remove the need to copy blocks of zeros and deduplicate data if converting to a TDVV. To make optimal use of this feature, it is advantageous to write zeros to the allocated but unused space on the fully provisioned volume prior to the conversion.

Estimating Thin Deduplication space savings A Dedup Estimate tool is available to show the space saving benefits of Thin Deduplication on existing data without the need to convert the volumes. The tool attempts to estimate the amount of space savings that can potentially be achieved by finding common data across specified TPVVs, and provides the dedup ratio based on the calculated data.

To launch Dedup Estimate from the CLI use the checkvv command with the -dedup_dryrun option on a group of VVs or a VV set to perform a dry run conversion which will report the space savings Thin Deduplication would achieve if the VVs specified were in the same deduplication CPG. The specified VVs must be of type TPVV but they can reside on any type of drive not just SSDs.

The following example launches a dedup estimation task on VVs vv1 and vv2:

cli% checkvv -dedup_dryrun vv1 vv2

The Dedup Estimate can also be launched from the HPE 3PAR Management Console by selecting a VV set or multiple TPVVs and then Right clicking and choosing the Dedup Estimate menu item as shown in figure 14.

9 For earlier versions of HPE 3PAR OS, this “thinning” operation does not complete online: a brief disruption of service is required to change the host mapping from the full to the

thin-provisioned volume. An HPE 3PAR Full Copy software (or Physical Copy) operation of the source volume is required for HPE 3PAR OS 3.1.1 and prior, the license for which is included in HPE 3PAR OS.


Figure 14. Dedup Estimate

The estimated dedup ratio will be shown under the task’s detailed information, which can be accessed via showtask -d.

cli% showtask -d 11705

Id Type Name Status Phase Step -------StartTime-------- -------FinishTime------- -Priority- -User—

11705 dedup_dryrun checkvv done --- --- 2014-10-12 22:25:45 CEST 2014-10-12 22:26:15 CEST n/a 3parsvc

-----(MB)------ (DedupRatio)

Id Name Usr Estimated Estimated

395 vv1 10240 --

431 vv2 10240 -- --

--------------------------------------

2 total 20480 10239 1.82

In this example, checkvv is estimating that if vv1 and vv2 were converted to TDVVs and put in the same CPG there would be a dedup ratio of 1.82:1 for the CPG. The tool does not calculate the amount of unique data within each TPVV so the estimated values for each TPVV are marked with “—.”

Online migration to HPE 3PAR StoreServ There are several approaches to consider when migrating data from a source array to thin provisioned or Thin Deduplication volumes on an HPE 3PAR StoreServ. If the source array is also an HPE 3PAR StoreServ then Peer Motion can be used to import a full or thin provisioned volume from the source system to a thin provisioned or thinly deduped volume on the destination system.

The HPE 3PAR Online Import Software enables an effortless migration of HPE EVA; EMC VMAX, VNX, CX4; HDS NSC, USP, VSP or IBM XIV Storage to an HPE 3PAR StoreServ Storage array, without the need of host-based resources. The source data can be migrated inline to thin provisioned or thinly deduped volumes on HPE 3PAR StoreServ Storage.

A block-based migration tools such as a host-based volume manager or a SAN-based virtualization appliance can also be used copy a source volume to the thin volume destination.

It is also possible to use file-based migration tools such as rsync in UNIX®/Linux, Quest in Windows, or backup and restore. With these tools the target thin volume will grow only to the capacity of data contained within the file system, therefore making efficient use of thin provisioning on the target HPE 3PAR StoreServ.


Preparing data for migration HPE 3PAR Peer Motion, HPE 3PAR Online Import Software and block-based migration tools will copy any non-zero blocks from the source volume to the TPVV or TDVV, whether they contain valid data or not. This means that the customer may not see all benefits from thin provisioning after using the migration tools.

It is therefore recommended to defragment the file systems (where applicable) and run a zerofile tool prior to the migration so that the allocated but unused space in the volume on the source array is filled with zeros. When the data is migrated the HPE 3PAR ASIC accelerated zero-detection will then allow customers to reap the benefits of thin volumes immediately.

For virtual machines the same process should be followed for all VM guests whether the VM files are copied directly or online migration technique such as Hyper-V Live Migration or VMware vMotion are used.

Conclusion HPE 3PAR StoreServ Storage is the only platform that offers a comprehensive thin strategy that not only allows storage to start thin, but to get thin and stay thin. Compaction technologies such as Thin Deduplication, thin provisioning and thin reclamation offer efficiency benefits for primary storage that can significantly reduce both capital and operational costs with spinning media and SSDs. Not only is HPE 3PAR StoreServ Storage viewed as the industry’s thin technology leader, but third-party testing and competitive analysis confirm that HPE 3PAR StoreServ offers the most comprehensive and efficient thin technologies among the major enterprise storage platforms. In addition, HPE 3PAR Thin Technologies protect SSD performance and extend flash-based media lifespan while ensuring resiliency.

HPE 3PAR Thin Technologies enables simple, yet powerful tools for improving the efficiency of storage. Following the best practices outlined in this paper will allow IT staff to help maximize the benefit of HPE 3PAR Thin Technologies and do more with less. To supplement the dramatic savings of thin provisioning, HPE 3PAR StoreServ features a unique HPE 3PAR ASIC with thin capabilities built in and a range of software offerings that can save enterprises 50 percent or more on the cost of a storage technology refresh.

Appendix A—Thin volumes and applications All applications including mission-critical ones can use thin-provisioned volumes. The use of thin volumes may need some consideration upfront to enable them to utilize the volume space thin-efficiently. The following sections include recommendations for installing a few well-known applications on thin-provisioned volumes.

Oracle In an Oracle environment, HPE 3PAR Thin Provisioning provides the better storage space utilization when used in the following cases:

• Autoextend-enabled tablespaces—Oracle Automatic Storage Management (ASM)-managed datafiles with the autoextend feature grow as the need for tablespace grows. Using such autoextendable datafiles on thin-provisioned volumes allows the system to allocate disk space to a database as it grows.

However, Oracle’s process of extending a tablespace is I/O intensive and can affect the performance of the database during the file extension. Lessen the frequency of the performance impact by increasing the increment of space that Oracle adds (the AUTOEXTEND ON NEXT parameter of the CREATE TABLESPACE command). Set autoextend to a larger value for rapidly growing datafiles, and a smaller value if the data is growing at a slower rate.

• ASM disk groups as archive log destinations—The combination of Oracle ASM and HPE 3PAR Thin Provisioning enables an expanding ASM archive log destination. Oracle databases become inaccessible when the archive log destinations fill up. Put Oracle archive logs on thin-provisioned ASM disk groups, which allows the underlying storage to self-tune, accommodating unexpected increases in log switch activity. After a level 0 backup, remove the archive logs and use the Oracle ASM Storage Reclamation Utility (ASRU, described later) to free up the storage that was allocated to the old logs.

HPE 3PAR Thin Provisioning may not be the best option for the following:

• Datafiles residing on file systems—When an Oracle datafile is first created, Oracle initializes the data blocks with a block headers and other metadata. Due to small block size of most databases, there are usually no contiguous ranges of zero blocks for space reclamation. This causes a TPVV to provision all of its space, nullifying the value of thin provisioning.


• Systems with high file system utilization—If the file systems or ASM disk groups are full, then the benefits of thin provisioning are reduced. Consider any ASM disk group or file systems with utilization rates above 80 percent as inappropriate for use with thin provisioning. In this case, it may be more efficient to use fully provisioned virtual volumes to hold this data.

• Oracle datafiles that are not in “autoextend” mode—Oracle databases write format information to datafiles during tablespace creation. This has the same effect as provisioning file systems with high utilization and may be inefficient depending upon the ratio of provisioned storage to database size.

Microsoft SQL Server Microsoft SQL Server transaction log files are formatted when a database is created. The pages written mostly contain contiguous zeros and therefore the HPE 3PAR Thin Persistence software will keep the allocation of the TPVV to a minimum. Over time, as transactions are written to the log files, the log file TPVV allocation will grow, so the CPG associated with log files should be appropriately sized.

As of Microsoft SQL Server 2005, the database creation phase no longer initializes datafiles if the account under which the SQL Server service is running has the “perform volume maintenance tasks” user rights assignment under the local security policy. The instant file initialization process creates a sparse file, which is thin friendly and does not allocate storage space in advance.

Microsoft SQL Server 2012 is able to take advantage of the SCSI UNMAP support in Windows Server® 2012 to automatically reclaim space after shrink/drop datafile operations. In addition, when datafiles and databases are deleted on Windows Server 2012, the system automatically reclaims the space without administrator intervention. Prior to Windows Server 2012 space reclamation required a zerofile tool to write zeros to the free space in the file system.

Microsoft Hyper-V The virtual hard disk (VHD) and the new virtual hard disk (VHDX) formats of Microsoft Hyper-V are fully compatible with HPE 3PAR Thin Provisioning. Both formats offer a choice of fixed or dynamic sizing. A fixed VHD has an allocated size that does not change whereas a dynamic VHD will expand as data is written to it.

Although it may seem natural to use a fully provisioned VV with a fixed VHD, contiguous zeros are written to the space allocated so they are ideal candidates for thin provisioning, as the zeros will be reclaimed by the HPE 3PAR Thin Persistence software.

In the past, fixed VHDs were recommended for production instead of dynamic VHDs, as their I/O performance was higher. With the new VHDX format introduced in Windows Server 2012, performance of dynamic VHDs has been significantly improved and there is now little difference between the two VHD types.

In addition, VHDX disks report themselves to the guest OSs as being “thin-provision capable.” This means that if the guest OS is UNMAP-capable it will be able to send UNMAPs to the VHDX file, which will then be used to ensure that block allocations within the VHDX file that are freed up for subsequent allocations as well as forwarding the UNMAP requests to the physical storage.

SAP By migrating data from traditional arrays to HPE 3PAR StoreServ via Thin Conversion, legacy SAP® systems can reduce up to 80 percent of the capacity in the storage environment. In an SAP system, data gets moved around or deleted within system storage volumes. HPE 3PAR Thin Persistence enables that the thin volumes used by SAP systems stay efficient as possible by reclaiming unused space associated with deleted data.

Typically, SAP databases store data in the form of a matrix (tables and indexes) consisting of rows and columns. Most of the columns in the tables are of fixed length, and there are often leading or trailing zeroes in these columns. The HPE 3PAR ASIC is able to detect these zeros if they form a contiguous 16 KB block and prevent storage from being allocated. In testing with SAP ERP 6 IDES, a LUN with HPE 3PAR Thin Persistence software enabled consumed 10 percent less storage space than a traditional thin LUNs after the initial database creation. See the HPE 3PAR StoreServ Storage for SAP Systems—technical white paper for further details.

http://h20195.www2.hp.com/v2/GetDocument.aspx?docname=4AA4-4652ENW


VMware vSphere VMware vSphere also has its own thin provisioning capabilities therefore a decision has to be made about which level to implement thin provisioning.

When implementing HPE 3PAR StoreServ TPVVs, administrators often ask whether implementing vSphere Thin Provisioning for Virtual Machine Disk (VMDK) files makes any sense. In general, Thin Provisioning with HPE 3PAR StoreServ and vSphere accomplish the same end-result, albeit at different logical layers. With VMware vSphere Thin Provisioning, administrators realize greater VM density at the Virtual Machine File System (VMFS) layer, at the cost of some CPU and disk I/O overhead as the volume is incrementally grown on the ESXi hosts. By implementing HPE 3PAR StoreServ TPVVs, the same VM density levels are achieved; however, the thin provisioning CPU work is offloaded to the HPE 3PAR StoreServ ASIC. If the goal is to reduce storage costs, help maximize storage utilization, and maintain performance, then use HPE 3PAR Thin Provisioning software to provision Virtual Machine File System (VMFS) volumes. If performance is not a concern but overprovisioning VMs at the VMFS layer is important, then administrators can consider implementing both Thin Provisioning solutions. However, administrators should realize that there is no additional storage savings realized by using vSphere Thin Provisioning on top of HPE 3PAR TPVVs; and in fact, implementing both solutions adds more management complexity to the environment.

When creating VMs, there are several options for the file system layout of the VMDK files. By default, VMDK files are created with the “Thick Provision Lazy Zeroed” option, which means the VMDK is sparsely populated so not all the blocks are immediately allocated. When a guest VM reads from the unallocated areas of the VMDK, the vSphere server detects this and returns zeros rather than reading from disk. This VMware thin provisioning capability enables the oversubscription of the sizes of the VMDK files within the datastore.

For performance-intensive environments, VMware recommends using “Thick Provision Eager Zeroed” (EZT) virtual disks. These EZT disks have lower runtime overhead but require zeros to be written across all of the capacity of the VMDK at the time of creation. On traditional arrays, this VMDK format would negate all the benefits of thinly provisioned LUNs as all of the physical storage is allocated when the volume is zero-filled during creation. However, HPE 3PAR Thin Persistence software allows clients to retain thin provisioning benefits when using EZT VMDKs without sacrificing any of the performance benefits offered by this VMDK option. In this case, Thin Persistence helps ensure that, when a new EZT volume is created, the entire volume is not allocated from physical storage since all zeros that have been written to the VMDK were intercepted and discarded by the HPE 3PAR ASIC.

Appendix B—Thin Persistence methods Thin Reclamation for Microsoft Windows Server 2012 Windows Server 2012 integrates very well with thin-provisioned volumes on HPE 3PAR StoreServ. It identifies thinly provisioned volumes on HPE 3PAR StoreServ systems, writes entries in the Windows event log file when storage thresholds are reached on the CPG, and the TPVV level and supports active reclaim by issuing UNMAPs upon file deletion or file shrinking on thin-provisioned volumes on NT File System (NTFS) formatted volumes. The standard defragmentation scheduled task also automatically reclaims storage.

In addition, Windows Server 2012 extends UNMAP support to the virtual layer. The Hyper-V VHDX disks report themselves to the guest OSs as being “thin-provision capable.” This means that if the guest OS is UNMAP-capable, it can send UNMAPs to the VHDX file, which will then be used to help ensure that block allocations within the VHDX file are freed up for subsequent allocations as well as forwarding the UNMAP requests to the physical storage.

There is also a File TRIM API which is mapped to the TRIM command for ATA devices and the UNMAP command for SCSI devices. TRIM hints allow the application to notify the storage that blocks that, previously were allocated, are no longer needed and can be reclaimed.

In summary, the following operations trigger storage space reclamation in Windows Server 2012:

• Deletion of a file from a file system on a thin-provisioned volume

• Running storage optimization, a new feature of Windows Server 2012 disk management

– You can use manual or automatic scheduling of the Optimize operation utility

– The standard Defrag scheduled task automatically runs Optimize


• UNMAP requests from a Hyper-V guest OS

– Deleting a file from the file system of an UNMAP capable guest OS sends UNMAP requests to the driver stack of the Hyper-V host

• UNMAP requests from applications using the TRIM API

The default behavior of issuing UNMAPs on file deletion can be disabled on a Windows 2012 server by setting the “disabledeletenotify” parameter of the fsutil command. This will prevent reclaim operations from being issued against all volumes on the server.

To disable reclaim operations, run the following PowerShell command:

fsutil behavior set DisableDeleteNotify 1

Thin Reclamation for Microsoft Windows Server 2003 and 2008 Windows Server versions prior to 2012 do not implement UNMAP support and therefore to reclaim thinly provisioned storage you must leverage the zero-detection capabilities of HPE 3PAR StoreServ.

Microsoft provide a Sysinternals advanced system utilities suite that includes the Secure Delete (SDelete) application that can used to overwrite a deleted files on-disk data to make disk data unrecoverable.

As well as overwriting a single file data space, SDelete can indirectly overwrite free space by allocating the largest file it can and then performing a secure overwrite to ensuring that all the disk space that was previously free becomes securely cleansed. You can utilize this feature of SDelete to perform thin reclamation on zero-detect enabled HPE 3PAR StoreServ volumes by specifying the -z flag when running SDelete to writes zeros to the free space.

One disadvantage of SDelete is that it does not support mount points so a volume must have a drive letter associated with it before the utility can be run. Using the subst command one can temporarily attach a drive letter to a mount point before running SDelete.

It is recommended that applications are shut down before running SDelete as it can cause a file_system_full condition due to consuming all “free” space. An alternative solution is to create a PowerShell script that uses fsutil to create a balloon file that is limited to a certain percentage of the “free” space.

This is an example of using fsutil to zero 100 MiB of space:

fsutil file createnew zerotemp.txt 104857600

fsutil file setvaliddata zerotemp.txt 104857600

fsutil file setzerodata offset=0 length=104857600 zerotemp.txt del zerotemp.txt

Thin Reclamation for VMware vSphere vSphere 5.0 introduced automatic space reclamation upon deletion of a VMDK. However, due to poor performance from some storage devices when the UNMAP command was used, VMware recommended this feature was to be disabled on vSphere 5.0 hosts. As an alternative vSphere 5.1 Update 1 included an update to the vmkfstools command that provided a new option (-y) to issue UNMAPs to reclaim unused space on a thin-provisioned volume. This new implementation is not an automated process and will need to be run manually. On vSphere 5.0 Update 1 and 5.2 the command should be run after changing to the root directory of the desired VMFS volume.

# cd /vmfs/volumes/<volume-name>

# vmkfstools -y <percentage of free space to reclaim>

The vmkfstools -y command creates a temporary “balloon” file in the datastore that can be up to the size of the free space available in the datastore, and then it issues UNMAPs for the blocks of the balloon file. However, it is recommended that the reclaim percentage is not more than 60 percent as the resulting balloon file temporarily uses space on the datastore that could cause the deployment of new virtual disks to fail while the vmkfstools command is running. This is an important consideration as there is no way of knowing how long a vmkfstools -y operation will take to complete. It can be anywhere from few minutes to several hours depending on the size of the datastore and the amount of space that needs to be reclaimed.


There is an alternative method that can be faster and takes advantage of the ASIC-based zero-detect capability of HPE 3PAR StoreServ. Run the vmkfstools command with the -d eagerzeroedthick option to create a zerofile that can then be removed.

# cd /vmfs/volumes/<volume-name>

# vmkfstools -c <size of space to reclaim> -d eagerzeroedthick zerofile

# rm zerofile-flat

These methods can be run when the file system is online but if reclaims of very large amounts of data (100’s of GB) are required consideration should be given to running it during a quiet period so the reclaims do not adversely impact other workloads on the storage.

In vSphere 5.5, the vmkfstools -y command is deprecated and in its place a new esxcli command UNMAP namespace has been added that allows deleted blocks to be reclaimed on thin-provisioned LUNs that support the UNMAP primitive. The reclaim mechanism has been enhanced so that the reclaim size can be specified in blocks instead of a percentage value to make it more intuitive to calculate, and the unused space is reclaimed in increments instead of all at once. However, reclaim operation is still manual.

To reclaim unused storage blocks on a vSphere 5.5 VMFS datastore for a thin-provisioned device, run the command:

# esxcli storage vmfs unmap -l volume_label | -u volume_uuid [-n number]

The datastore to operate on is determined by either using the -l flag to specify the volume label or -u to specify the universal unique identifier (UUID). The optional -n flag sets the number of VMFS blocks to UNMAP per iteration. If it is not specified, the command uses a default value of 200.

The VMware hypervisor does not report disks as being thin provisioning capable to the guest OS, when using VMDKs therefore to reclaim thinly provisioned storage you must leverage the zero-detection capabilities of HPE 3PAR StoreServ. This means using standard file system tools (such as SDelete in Microsoft Windows, dd in UNIX/Linux) to write zeros across deleted and unused space in the file system of a VM. The zeros will be autonomically detected by the HPE 3PAR ASIC and the disk space they were consuming will be freed up and returned to the thin-provisioned volume.

If running VMware ESXi 4.1, it is strongly recommended to install the HPE 3PAR VAAI plug-in before creating EZT VMDKs as it will speed up their creation by 10X to 20X.

Thin Reclamation for Linux Linux uses the term discard for the act of informing a storage device that blocks are no longer in use. This is because it uses the same mechanism to support both TRIM commands on ATA SSDs and UNMAP commands on SCSI devices, so discard was chosen as a protocol neutral name. RHEL 6 was the first major distribution to support discards and offers both real-time and batched support.

Real-time support is offered by an ext4 or XFS file system when mounted with the -o discard option; the default is not to issue discards. When data or files are deleted on a discard enabled ext4 file system, UNMAPs are generated to free up space on the thinly provisioned virtual volume on the storage. The LVM and the device-mapper (DM) targets also support discards so space reclaim will also work on file systems created on LVM and/or DM volumes.

For example, to mount the DM device tpvv_lun on /mnt with discards enabled, run:

# mount -t ext4 -o discard /dev/mapper/tpvv_lun /mnt

This will cause the RHEL 6 to issue the UNMAP command, which in turn causes space to be released back to the array from the TPVV volumes for any deletions in that ext4 file system. This is not applicable for fully provisioned virtual volumes.

In addition, the mke2fs, e2fsck, and resize2fs utilities also support discards to help ensure the TPVV volumes are enhanced when administration tasks are performed.

There is also batched discard support available using the fstrim command. This can be used on a mounted file system to discard blocks, which are not in use by the file system. It supports ext3 and ext4 file systems and can also re-thin a file system not mounted with the discard option.


For example, to initiate storage reclaim on the /mnt file system, run:

# fstrim -v /mnt

/mnt: 21567070208 bytes were trimmed

The fstrim can be run when the file system is online but as it generates UNMAP requests for all the unused blocks in one go consideration should be given to running it during a quiet period so the reclaims do not adversely impact other workloads on the storage.

It is also possible to configure the Logical Volume Manager (LVM) to automatically issue discards to a logical volumes’s underlying physical volume(s) when the logical volume is no longer using the physical volumes’ space (e.g., by using the lvremove or lvreduce commands). This can be done by enabling the option issue_discards in the LVM configuration file:

$ cat /etc/lvm/lvm.conf

devices {

issue_discards = 1

}

Then when removing a volume, you would see the following message:

# lvremove vg1/lvol0

Do you really want to remove and DISCARD logical volume lvol0? [y/n]:

The Linux swap code will also automatically issue discard commands for unused blocks on discard-enabled devices and there is no option to control this behavior.

Thin Reclamation for HP-UX HP-UX systems using Veritas Storage Foundation 5.1 or higher can reclaim space associated with file deletions or file shrinking (see the Thin Reclamation for Symantec Storage Foundation section for more details). However, non-VxFS file systems or VxFS file systems on LVM volumes will need to reclaim space using a “zerofile” script that writes zeros to all allocated but unused space on a file system. The zero-detection capability of the HPE 3PAR ASIC will intercept the blocks of zeros being written and automatically trigger the reclamation of the space.

Thin Reclamation for UNIX On UNIX systems or Linux distributions that do not support discard, you will need to reclaim space using a “zerofile” script that writes zeros to all allocated but unused space on a file system. The zero-detection capability of the HPE 3PAR ASIC will intercept the blocks of zeros being written and automatically trigger the reclamation of the space.

The script would use the dd command to copy zero data blocks from the /dev/zero device to a file in the file system. However, it is recommended that the size of the space to zero is not more than 70 percent of the free space as a very large zerofile could cause the creation of new files to fail while the zerofile script is running.

Thin Reclamation for HPE OpenVMS Automatic space reclamation upon deletion is possible in OpenVMS by leveraging the erase-on-delete capability to write erasure patterns of zeros. The zero-detection capability of the HPE 3PAR ASIC will intercept the blocks of zeros being written and automatically trigger the reclamation of the space.

The inclusion of the /ERASE qualifier with the DELETE or the PURGE command causes the system to write an erasure pattern of zeros over the entire file location when you delete or purge that file. Users can use this qualifier voluntarily or this can be made automatic by including the following command definitions in the system login command procedure (usually SYS$MANAGER:SYLOGIN.COM):

DEL*ETE :== “DELETE/ERASE”

PUR*GE :== “PURGE/ERASE”


However, any user can bypass these definitions by adding the /NOERASE qualifier to the DELETE or PURGE commands. To guarantee erase-on-delete, it is possible to turn on the feature for the entire volume by using the DCL command SET VOLUME/ERASE_ON_DELETE. Once this is set, when files are deleted they are overwritten with an erasure pattern of zeros.

To completely erase the volume and enable erase-on-delete for the volume at volume initialization, use the DCL command INITIALIZE/ERASE.

Thin Reclamation for Symantec Storage Foundation Since the introduction of Symantec Storage Foundation 5.1 hosts with VxFS file systems managed by the Veritas Volume Manager (VxVM) software can also reclaim space associated with file deletions or file shrinking on HPE 3PAR StoreServ Storage.

The space reclamation is not automatic, the VxFS file system informs the VxVM about deleted blocks and a VxVM command has to be manually run which send WRITE SAME SCSI commands with the UNMAP bit turned on to the HPE 3PAR StoreServ. No tool to write zeros to the deleted space in the file systems is required for reclaiming the space.

The list of disks whose allocated but unused space can be reclaimed is given by the command executed on the host:

# vxdisk -o thin,fssize -u m list

This will display the VV allocated space and the file system usage. The space reclamation is initiated by the VxVM vxdisk command:

# vxdisk reclaim [<disk>|<dg>|<encl>]

By default, the reclamation does not affect unmarked space, which is the unused space between subdisks. If a LUN has a lot of physical space that was previously allocated, the space between the subdisks might be substantial. Use the -o full option to reclaim the unmarked space.

# /opt/VRTS/bin/fsadm -V vxfs -R /<VxFS_mount_point>

To monitor the reclamation status, run the following command:

# vxtask list

TASKID PTID TYPE/STATE PCT PROGRESS

171 RECLAIM/R 00.00% 0/41875931136/0 RECLAIM vol100 dg100

The vxrelocd daemon tracks the disks that require reclamation. The schedule for reclamation can be controlled using the vxdefault command. The reclaim_on_delete_wait_period parameter specifies the number of days after a volume or plex is deleted when VxVM reclaims the storage space. The default value is 1, which means the volume is deleted the next day. A value of -1 indicates that the storage is reclaimed immediately and a value of 367 indicates that the storage space can only be reclaimed manually using the vxdisk reclaim command. The reclaim_on_delete_start_time parameter specifies the time of day when VxVM starts the reclamation for deleted volumes and this defaults to 22:10.

To completely disable thin-reclaim operations, add the parameter reclaim=off to the /etc/vxdefault/vxdisk file.

Thin Reclamation for Oracle databases During an Oracle Automatic Storage Management (ASM) database lifecycle, the utilization of allocated storage capacity in a thin-provisioned volume can decrease as changes are made to the database through common operations such as:

• Dropping of a tablespace or database upon deletion of transient data

• Resizing of an Oracle datafile upon shrinking a tablespace

• Addition of new disks to an ASM disk group to accommodate growth or load balance performance

These changes result in the creation of unused ASM disk space that can built up over time and although this space is available for reuse within ASM, it remains allocated on the storage array. The net result is that the storage utilization on the array eventually falls below desirable levels.


To solve this problem, HPE and Oracle have partnered to improve storage efficiency for Oracle Database 10g and 11g environments by reclaiming unused (but allocated) ASM disk space in thin-provisioned environments. The Oracle ASRU is a stand-alone utility that works with HPE 3PAR Thin Persistence software to reclaim storage in an ASM disk group that was previously allocated but is no longer in use. Oracle ASRU compacts the ASM disks, writes blocks of zeroes to the free space, and resizes the ASM disks to original size with a single command, online and non-disruptively. The HPE 3PAR StoreServ, using the zero-detect capability of the HPE 3PAR ASIC, will detect these zero blocks and reclaim any corresponding physical storage.

You can issue a SQL query to verify that ASM has free space available that can be reclaimed:

SQL> select name, state, type, total_mb, free_mb from v$asm_diskgroup where name = ‘LDATA’;

NAME STATE TYPE TOTAL_MB FREE_MB

----- ------- ------ ------- -------

LDATA MOUNTED EXTERN 1023984 610948

Run the Oracle ASRU utility as the Oracle user with the name of the disk group for which space should be reclaimed:

# bash ASRU LDATA

Checking the system ...done

Calculating the new sizes of the disks ...done

Writing the data to a file ...done

Resizing the disks...done

/u03/app/oracle/product/11.2.0/grid/perl/bin/perl -I /u03/app/oracle/product/ 11.2.0/grid/perl/lib/5.10.0 /home/ora/zerofill 5 /dev/oracleasm/disks/LDATA2 129081 255996 /dev/oracleasm/disks/LDATA3 129070 255996 /dev/oracleasm/disks/

LDATA4 129081 255996 /dev/oracleasm/disks/LDATA1 129068 255996

126928+0 records in

126928+0 records out

133093654528 bytes (133 GB) copied, 2436.45 seconds, 54.6 MB/s

126915+0 records in



126926+0 records in



126915+0 records in



Calculating the new sizes of the disks ...done

Resizing the disks...done Dropping the file ...done


Appendix C—File Systems and Thin Deduplication File systems with lots of common files such as home directory shares are ideal candidates for deduplication. However, the space savings achieved can vary greatly depending on the file system in use. Thin Deduplication uses a granularity, also called chunk size, of 16 KiB and therefore optimum deduplication is achieved when the file system aligns the files with this granularity.

They are many different file systems choices available across the supported operating systems and they work in very different ways with deduplication. Some file systems work perfectly, some need online tuning, some need reformatting with different options and some cannot be changed. In general, older file system technologies designed to work with discrete drives do not perform as well as modern log structured file systems.

It is often thought that deduplication ratio is dependent on the file system block size alone but it actually depends on block size, extent size and alignment, all of which vary from file system to file system. Block size is the basic unit of storage for the file system and it is usually, but not always, page size of the processor architecture. Extent size is the number of contiguous blocks that can be written. Alignment is the starting offset of a file, most file systems align to block size boundaries.

For normal volumes changing the block size or alignment can increase the space usage for very small files (although it may improve performance for larger files). However, by aligning the files the increase in deduplication should more than offset any space increase for small files so the overall space usage will reduce.

Note that with virtualized systems the file systems inside the virtual machine, which may be under the control of a tenant, should be aligned well as the file system of the hypervisor.

Microsoft Windows New Technology File System (NTFS) NTFS is the default file system for Microsoft Windows and by default has an allocation unit of 4 KiB. For optimal deduplication consider setting the allocation unit to 16 KiB or a multiple of 16 KiB when the file system is created. Do this by selecting the desired the allocation unit in the format dialog box when creating the file system.10

Using a 16 KB allocation unit size not only ensures that all files start on a 16 KiB boundary, relative to the offset of the partition, but also any UNMAP operations will also be aligned to 16 KiB boundaries.

To determine the allocation unit size of an existing NTFS file system run the fsutil fsinfo ntfsinfo command and check the Bytes per Cluster value. The following example shows a file system with a 16 KiB allocation unit.

C:\>fsutil fsinfo ntfsinfo f:

NTFS Volume Serial Number: 0x004eab4c4eab38f4

Version: 3.1

Number Sectors: 0x0000000006f997ff

Total Clusters: 0x00000000001be65f

Free Clusters: 0x00000000000317a5

Total Reserved: 0x0000000000000000

Bytes Per Sector: 512

Bytes Per Physical Sector: 512

10 For Microsoft Windows PowerShell Format-Volume cmdlet syntax see technet.microsoft.com/en-us/library/hh848665.aspx.

http://technet.microsoft.com/en-us/library/hh848665.aspx


Bytes Per Cluster: 16384

Bytes Per FileRecord Segment: 1024

Clusters Per FileRecord Segment: 0

Mft Valid Data Length: 0x0000000000040000

Mft Start Lcn: 0x0000000000018000

Mft2 Start Lcn: 0x0000000000000001

Mft Zone Start: 0x0000000000018000

Mft Zone End: 0x0000000000019920

RM Identifier: D252D40C-592D-11E4-803D-A0E201E4931F

The following example shows the effect of allocation unit on dedup ratios. Five TDVVs were created and then formatted with NTFS file systems using allocation units of 4 KiB–64 KiB. Four copies of the Linux kernel source tree, which contains many small files, were unpacked into each file system.

cli% showvv -space ntfs*

---Adm---- ---------Snp---------- ----------Usr-----------



149 ntfs_4k tdvv base 6781 6233 0 0 0.0 -- -- 11831 8660 1.7 0 0 18612 512000 34.4 1.1

151 ntfs_8k tdvv base 6200 5679 0 0 0.0 -- -- 11006 7889 1.5 0 0 17206 512000 37.7 1.4

152 ntfs_16k tdvv base 3908 3498 0 0 0.0 -- -- 7748 4853 0.9 0 0 11656 512000 61.3 3.1

153 ntfs_32k tdvv base 3907 3491 0 0 0.0 -- -- 7748 4846 0.9 0 0 11655 512000 61.4 3.1

154 ntfs_64k tdvv base 3907 3494 0 0 0.0 -- -- 7748 4847 0.9 0 0 11655 512000 61.4 3.1

It is clear that the file system with allocation units of 16 KiB and above have significantly higher dedup ratios and from the compaction ratios you can see that they also save more space even though the file systems contained only small files.

Setting boot disk allocation units While changing the NTFS allocation unit on a file system requires a reformat, it is relatively easy to do for non-system disks. The Windows operating system unfortunately does not give an option to set the allocation unit during installation so the system disk will have 4 KiB by default. Although there is little dedupable data within a single Windows OS instance, having a 4 KiB allocation unit reduces the deduplication achievable between multiple Windows Servers or virtual machines. Therefore, it is still desirable to set 16 KiB allocation units for boot disks.

The Windows installation does not give an option to change the allocation unit on Windows boot disks and Microsoft does not have a documented procedure. The challenge is that Windows boots from a hidden System Partition which must have a 4 KiB allocation unit. A solution is at the start of the installation process is to create a small (at least 500 MB) System Partition formatted with a 4 KiB allocation unit and then the main Windows partition formatted with a 16 KiB allocation unit.

The following is an example of the procedure:

1. Start Windows installation

2. Press Shift + F10 to open CMD

a. disk part

b. select disk 0

c. create partition primary size=1000

d. format quick

e. create partition primary

f. format quick unit=16K


3. Close CMD

4. Install Windows

This method has been tested on Windows Server 2008 and Windows Server 2012.

If the Windows System Partition is not formatted with a 4 KiB allocation unit the message shown in figure 15 will be seen during installation.

Figure 15. System partition error

Resilient File System (ReFS) ReFS is a new file system introduced with Windows Server 2012. It was designed to have higher data integrity and scalability than NTFS. Internally ReFS allocates data on disk in clusters of 64 KiB and metadata in clusters of 16 KiB. Therefore, no tuning is necessary for use with Thin Deduplication.

Formatting a volume with ReFS can be done from the UI by choosing ReFS as the file system in the drop down of the Format dialog box or from the command line using the following syntax:

Command-Line Format /fs:ReFS /q J:

PowerShell Equivalent Format-Volume -DriveLetter J -FileSystem ReFS

Microsoft Hyper-V Windows Server 2012 brought many new virtualization improvements, including the introduction of the VHDX file format. In Windows Server 2008, Hyper-V used a virtual hard disk (VHD) file format that had a 2 TiB limit. The new VHDX file format has a maximum capacity 64 TiB. The advantages of VHDX aren’t limited to improved capacity, it has a 4 KiB logical sector size that improves performance and can increase deduplication compared with VHD files. It is therefore recommended to convert VHD files to VHDX format when using thin volumes.


Use one of the following methods to convert between formats:

1. Use the Hyper-V UI in Windows Server 2012, select edit the VHD file and choose to convert to VHDX.

2. Use the new Convert-VHD PowerShell cmdlet.11

Note that the VHD conversion must be done with the VM shut down.

Furthermore, the virtual machines using the VHDs may have file systems that have alignments that are different from the host file system. It is therefore recommended to follow the operating system guidelines from this section for each virtual machine.

VMware vSphere The VMFS-5 file system of VMware vSphere 5.x uses a 1 MiB block size, the VMFS versions used in prior releases used block sizes that depended on the file system size but were always a multiple of 1 MiB. Due to these large block sizes there is no tuning necessary for deduplication with VMware.

However, the virtual machines using the VMDK files within the VMFS may have file systems that have alignments that are not optimal for deduplication. It is therefore recommended to follow the operating system guidelines from this section for each virtual machine.

Oracle Solaris ZFS ZFS is a combined file system and logical volume manager for Oracle Solaris systems. It uses variable-sized blocks, with 128 KB as the default size. While this is good for deduplication within a file, it aligns files on physical block boundaries which are 512 bytes by default and this significantly reduces deduplication between files containing like data. The inter-file deduplication rate can be increased by configuring Solaris to use 16 KiB physical block sizes for HPE 3PAR StoreServ volumes.

On SPARC based Solaris platforms the SAN SCSI Disk driver configuration file /kernel/drv/ssd.conf should be edited to include the following line:

ssd-config-list=“3PARdataVV”,“physical-block-size:16384”;

On Intel® based Solaris platforms the SCSI Disk driver configuration file /kernel/drv/sd.conf should be edited to include the following line:

sd-config-list=“3PARdataVV”,“physical-block-size:16384”;

After the system is rebooted any new zpools created should have a 16 KiB block size. You can verify this by running the command zdb -L. The zpools created with 16 KiB block sizes will have an ashift value of 14 and the zpools created with the default block size will have an ashift value of 9 as shown in this example.

# zdb -L | grep ashift

ashift: 14

ashift: 9

11 For Microsoft Windows PowerShell Convert-VHD cmdlet syntax see: technet.microsoft.com/en-us/library/hh848454.aspx.

http://technet.microsoft.com/en-us/library/hh848454.aspx


The following example shows the difference in dedup ratio between a zpool created with 16 KiB block size and the default block size. Ten copies of the same 1 GB file were copied into to each file system.

cli% showvv -space zfs*

---Adm---- ---------Snp---------- ----------Usr-----------



12 zfs_def tdvv base 256 9 0 0 0.0 -- -- 5632 10 0.0 0 0 5888 51200 4.8 1.2

13 zfs_16K tdvv base 256 9 0 0 0.0 -- -- 5632 0 0.0 0 0 5888 51200 39.4 9.9

It is clear that with the default block size there is very little deduplication due to the 512 byte alignment but with the 16 KiB block size there is almost perfect deduplication.

UNIX file system (UFS) UFS is the original Solaris file system and is based on the BSD Fast File System. The maximum file system size is 1 TB on Intel Solaris (x86) and 16 TB on SPARC Solaris. The default logical block size which files are aligned to is 8 KiB and this is also the maximum block size, therefore to obtain the best deduplication rate it may be necessary to migrate to ZFS with the aforementioned block size tuning.

Linux There are many different file systems available on Linux but the most widely used are ext3, ext4, and XFS.

Ext3 is an extension of the ext2 file system with journal support. It does not support extents and therefore files will be fragmented into block sized chucks which have a default, and maximum, size of 4 KiB. Ext3 was introduced to the Linux kernel in 2001 and for a long time was considered to be the standard Linux file system.

Ext4 superseded ext3 and replaced the traditional block mapping scheme by extents. An extent is a range of contiguous physical blocks, improving large file performance and reducing fragmentation. Ext4 uses a delayed allocation mechanism that buffers data and allows it to allocate blocks in groups. Ext3 file systems can be migrated offline to ext4 by using the tune2fs command, however, exiting files will retain their ext3 layout therefore backup and restore operations are required to take full advantage of the ext4 improvements. Ext4 was made the default file system in Red Hat Enterprise Linux 6.

Extend file system (XFS) is a highly scalable, high-performance file system with metadata journaling and online defragmentation. Compared to ext4 it excels at multi-threaded, parallel I/O workloads and scales higher with support for file systems up to 16 EB and files of 8 EB. There is no direct conversion available from ext3/ext4 to XFS so migration involves backup and restore operations. XFS was the default file system in SUSE Linux Enterprise Server from SLES8-SLES11 and is the default file system in Red Hat Enterprise Linux 7. With SLES12, the B-tree file system (Btrfs) is the default file system for the operating system but XFS is the default for all other use cases.

The following example shows three TDVVs with ext3, ext4, and XFS file systems created on them. Four copies of a Linux system backup were restored to each file system.

cli% showvv -space ext3 ext4 xfs

---Adm---- ---------Snp---------- ----------Usr-----------



19 ext3 tdvv base 13306 12037 0 0 0.0 -- -- 17401 12831 12.5 0 0 30706 102400 4.1 1.6

20 ext4 tdvv base 5552 4887 0 0 0.0 -- -- 8583 5208 5.1 0 0 14136 102400 10.1 2.3

21 xfs tdvv base 2989 2526 0 0 0.0 -- -- 5668 2688 2.6 0 0 8658 102400 19.6 3.5

--------------------------------------------------------------------------------------------------------------------------------

3 total 21847 19450 0 0 31652 20727 53500 307200 7.6 2.2


It is clear from this example how the use extents allows ext4 to give more than double the space savings of ext3. However, the more sophisticated XFS provides the greatest space savings and therefore consideration should be given to migrating existing ext3 or ext4 file systems to XFS when using thinly deduped volumes.

Symantec Storage Foundation The Veritas File System (VxFS) of Symantec Storage Foundation is available on many platforms including AIX, HP-UX, Linux, Solaris, and Windows. It is an extent based file system that allocates storage in groups of extents rather than a block at a time.

VxFS file systems are formatted with a fixed block size ranging from 1 KiB to 8 KiB which represents the smallest amount of disk space allocated to a file. The block size largely used to define the maximum size of the file system, a block size of 1 KiB allows a maximum file system size of up to 32 TB and a block size of 8 KiB allows for a file system up to 256 TB.

The extent based nature of VxFS means that no tuning is necessary for use with Thin Deduplication as the alignment is not based on the block size. This is demonstrated by the following example which shows two TDVVs, one with a 1 KiB block size file system and the other with an 8 KiB file system. Four copies of a backup were restored to each file system.

cli% showvv -space vxfs*

---Adm---- ---------Snp---------- ----------Usr-----------



156 vxfs_1k tdvv base 8393 7889 0 0 0.0 -- -- 8627 4657 2.3 0 0 17021 204800 16.3 3.9

157 vxfs_8k tdvv base 9271 8743 0 0 0.0 -- -- 9292 5167 2.5 0 0 18563 204800 14.7 3.6

--------------------------------------------------------------------------------------------------------------------------------

2 total 17664 16632 0 0 17919 9824 35584 409600 15.5 3.8

Thin Deduplication achieves good results with both file systems with the 1 KiB block size having slightly a lower space usage (~10 percent) and a higher dedup ratio than the one with the 8 KiB block size.

HP-UX Journaled File System (JFS) JFS is an implementation of the Veritas File System (VxFS) and due to the extent based nature of VxFS means that no tuning is necessary for use with Thin Deduplication.

Hierarchical File System (HFS) HFS is based on the BSD Fast File System. HFS file systems are not commonly used as their maximum size is limited to 128 GB. However, HFS does support a wider range of block sizes (4 KiB-64 KiB) than other FFS implementations and therefore is suitable for use with Thin Deduplication.

The default primary block size of HFS is 8192 bytes and can be checked by looking at the bsize field of the dumpfs command output.

$ dumpfs /dev/rdisk/disk63

**/dev/rdisk/disk63:

magic 5231994 time Fri Mar 20 09:30:54 2015

sblkno 16 cblkno 24 iblkno 32 dblkno 192

sbsize 2048 cgsize 2048 cgoffset 32 cgmask 0xfffffff0

ncg 12800 size 104857600 blocks 102604584

bsize 8192 shift 13 mask 0xffffe000

fsize 1024 shift 10 mask 0xfffffc00

frag 8 shift 3 fsbtodb 0

minfree 10% maxbpg 256


maxcontig 1 rotdelay 0ms rps 60

csaddr 192 cssize 204800 shift 9 mask 0xfffffe00

ntrak 16 nsect 32 spc 512 ncyl 204800

cpg 16 bpg 1024 fpg 8192 ipg 1280

nindir 2048 inopb 64 nspf 1

To create a new HFS file system suitable for use with Thin Deduplication use the -b flag of the newfs command to set a 16 KiB block size.

$ newfs -F hfs -b 16384 -f 4096 /dev/rdisk/disk63

mkfs (hfs): Warning - 608 sector(s) in the last cylinder are not allocated.

mkfs (hfs): /dev/rdisk/disk63 - 104857600 sectors in 168042 cylinders of 16 tracks, 39 sectors 107374.2Mb in 10503 cyl groups

(16 c/g, 10.22Mb/g, 1536 i/g)

IBM AIX Journaled File System (JFS) is a 64-bit extent based file system. The initial version of JFS (also called JFS1) can support a file system of up to 1 TB and a file size of 64 GB. JFS2 (the Enhanced Journaled File System) is similar to JFS, but supports a file system of up to 32 TB and a file size of 16 TB.

Both JFS and JFS2 use a block size of 4 KiB and use extents of up to 64 GiB to allocate blocks to files. JFS2 also divides the space into allocation groups which are ranging from 8 MiB to 64 MiB which allow the resource allocation policies to cluster disk blocks for related data. Due to the extent based nature of JFS no tuning is necessary for use with Thin Deduplication.

HPE OpenVMS OpenVMS uses the Files-11 file system with On-Disk Structure Levels 2 (ODS-2) or 5 (ODS-5). The minimum unit of storage allocation is known as a cluster and is sized in terms of 512 byte blocks.

For ODS-5 disks, the default cluster size is 16 (8 KiB) and the minimum value allowed is defined by the following formula:

For ODS-2 disks, the default cluster size depends on the disk capacity; disks with less than 50,000 blocks have a default of 1. Disks that are larger than 50,000 blocks have a default of either 16 (8 KiB) or the result of the following formula, whichever is greater:

The cluster size is set when the file systems are created using INITIALIZE command to make the disk volume structures. The syntax of the INITIALIZE command is:

INITIALIZE /CLUSTER_SIZE=number-of-blocks device-name[:] volume-label

For optimal deduplication initialize the volumes with a cluster size which is a multiple of 32.


Sign up for updates

Rate this document

© Copyright 2012–2016 Hewlett Packard Enterprise Development LP. The information contained herein is subject to change without notice. The only warranties for Hewlett Packard Enterprise products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. Hewlett Packard Enterprise shall not be liable for technical or editorial errors or omissions contained herein.

Intel is a trademark of Intel Corporation in the U.S. and other countries. Linux is the registered trademark of Linus Torvalds in the U.S. and other countries. Microsoft, Windows, and Windows Server are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. Oracle is a registered trademark of Oracle and/or its affiliates. Red Hat is a registered trademark of Red Hat, Inc. in the United States and other countries. SAP is the trademark or registered trademark of SAP SE in Germany and in several other countries. UNIX is a registered trademark of The Open Group. VMware is a registered trademark or trademark of VMware, Inc. in the United States and/or other jurisdictions.

4AA3-8987ENW, March 2016, Rev. 9

Learn more at hp.com/go/3PARStoreServ

http://www.hpe.com/info/getupdated

http://www.facebook.com/sharer.php?u=http://h20195.www2.hpe.com/V2/GetDocument.aspx?docname=4AA3-8987ENW

http://twitter.com/home/?status=HPE 3PAR Thin Technologies+@+http://h20195.www2.hpe.com/V2/GetDocument.aspx?docname=4AA3-8987ENW

http://www.linkedin.com/shareArticle?mini=true&ro=true&url=http://h20195.www2.hpe.com/V2/GetDocument.aspx?docname=4AA3-8987ENW&title=HPE 3PAR Thin Technologies+&armin=armin

https://hpresearch.az1.qualtrics.com/SE/?SID=SV_6EuQKOr2Ku1CUzH&Pubnumber=4AA3-8987ENW






















http://twitter.com/home/?status=HPE%203PAR%20Thin%20Technologies+@+http://h20195.www2.hpe.com/V2/GetDocument.aspx?docname=4AA3-8987ENW



http://www.hp.com/go/3PARStoreServ

Documents

HPE 3PAR Thin Technologies · PDF fileDeleted volume snapshot space ... With HPE 3PAR Thin Conversion, a technology refresh no longer requires a terabyte-for-terabyte replacement,