January 2005

PowerEdge 1855: Best Practice Recommendations

to Aid in Data Center Deployment Enterprise Product Group (EPG)

Dell Whitepaper

January 2005 Page 2 Dell Enterprise Product Group

Contents Executive Summary.................................................................................................................... 3

Introduction................................................................................................................................. 4

Data center Cooling Considerations and Best Practices .................................................... 5

PowerEdge 1855 Single System Data.................................................................................... 10

PowerEdge 1855 Additive System Data ............................................................................... 12

PowerEdge 1855 Cabling Best Practices ............................................................................... 13

Glossary...................................................................................................................................... 16

References .................................................................................................................................. 16

January 2005 Page 3 Dell Enterprise Product Group

Section 1 Executive Summary

Technologies deployed in today’s data centers are providing benefits that few would have considered possible as little as five years ago. Processing power, density, availability, and affordability are drastically better and as a result businesses have taken advantage of greater choice and flexibility by deploying even more servers without drastic changes in real estate. Most of the new server technologies provide immediate bottom line benefits to businesses around the world, yet these same server technologies can also create challenges when proper deployment considerations are not made. The most pressing data center challenge for many businesses is the data center design or layout, including appropriate power and cooling architectures. Is there enough power running to each rack to ensure maximum availability of all systems? Is the cooling system design sufficient to handle new servers that were just purchased? How many servers can fit into a single rack and will my floor support the weight? The evolution of server technologies from traditional tower servers to high density blade servers has increased the importance placed on each answer to each question.

Most businesses consider deploying blade servers in an effort to harness increased processing power in the same amount, or even a lesser amount, of rack space, ease management, and reduce cable sprawl. Not only can Dell’s PowerEdge 1855 blade server deliver all of these benefits, but it also can deliver additional benefits, such as reduced power consumption and thermal dissipation compared to the Dell PowerEdge 1850 1U server, when deployed properly in the data center. The remainder of this whitepaper is dedicated to covering many Dell recommended deployment strategies for high density computing environments.

January 2005 Page 4 Dell Enterprise Product Group

Section 2

Introduction

The purpose of this whitepaper is to provide critical deployment data as well as system independent data center cooling best practices that help end-users effectively plan for the deployment of the PowerEdge 1855. Customers should refer to the PowerEdge 1855 system and rack user guides for specific system and rack requirements.

Measured data taken from both minimal and more extensive configurations are shown in the single system data sheet. These configurations are for reference only. It is highly recommended that the end-user take actual measurements using their respective configuration and application loads in order to achieve precise power consumption measurements.

Disclaimer THIS WHITE PAPER IS FOR INFORMATIONAL PURPOSES ONLY, AND MAY CONTAIN TYPOGRAPHICAL ERRORS AND TECHNICAL INACCURACIES. THE CONTENT IS PROVIDED AS IS, WITHOUT EXPRESS OR IMPLIED WARRANTIES OF ANY KIND.

January 2005 Page 5 Dell Enterprise Product Group

Section 3 Data Center Cooling Considerations and Best Practices

1) Obtain the most accurate power estimate in order to size an appropriate HVAC.

Acceptable: Use maximum measured power as provided by Dell.

Better: Use information provided in this whitepaper or a Dell configuration calculator that allows configuration-based power estimates.

Best: Measure similarly configured system under actual configuration and application load.

2) Maximize the amount of chilled air delivered to each rack.

As with any high density computing deployment, it is not only important to match the HVAC capacity to the equipment load, but it is also, and in some cases more critical, to ensure there is an adequate delivery of chilled air mass to the rack. One of today’s greatest challenges in high density computing deployments is the delivery of chilled air in sufficient quantities to satisfy equipment flow rate requirements. Efforts to maximize chilled air delivery to the equipment intake include, but are not limited to, implementing hot aisle / cold aisle orientation, minimizing cable and perimeter leaks, installing close out panels in empty rack spaces, eliminating hot aisle vent tiles, and reprovisioning chilled air under the raised floor. Reprovisioning through the use of additional vent tiles or by using a higher percentage open grate can add more chilled air in the vicinity where it is needed. Supplemental overhead cooling may also be a consideration in delivering an increased delivery of chilled air in specific locations.

3) Understand cold air depletion and subsequent hot air recirculation.

When there is not an adequate supply of chilled air, the chilled air supply depletes before it reaches all of the equipment. In a raised floor environment, when chilled air depletion occurs, systems mounted above the point of depletion may be exposed to ambient temperatures above the equipment’s specification.

January 2005 Page 6 Dell Enterprise Product Group

Figure 1 - CFD example of data center with six rows

Figure 1 is taken from a detailed airflow dynamics analysis simulation of a data center. Six rows of racks are shown with airflow paths and background colors showing air temperatures. The chilled air supply is inadequate for the demands of the two middle rack rows. Equipment in the upper half of the rack draws air in from above which is recirculating from the adjacent (hot) exhaust aisle. In this particular example, the rack cannot sustain systems above the 25th or 27th rack unit location. One way to alleviate this problem is to remove equipment from these racks and/or increase the amount of chilled air delivered to these racks. As shown by the cool background temperatures, the other two aisles have an adequate supply of chilled air that matches or exceeds the equipment demand in those rows.

The data center is an extremely dynamic system. It is extremely difficult to predict capacity at any point in the data center simply by knowing the chilled air flow rate. Generally, the more chilled air delivered, the greater the potential for deployment. It is critical to understand flow rates through precise measurements. It is also highly advised to have a data center analyzed. Computational Fluid Dynamics (CFD) tools are available that allow predictive analysis of airflow and air temperatures. CFD is an extremely valuable tool that can minimize most of the trial and error associated with optimizing a dense deployment.

Best practices summary:

• Orient equipment racks using the practice of hot aisle / cold aisle. The cold aisle (equipment front intake) contains vented tiles. The hot aisle (equipment exhaust) should have no vented tiles.

• Minimize leakage around cable openings in the floor and install floor tile cable grommets if possible (Figure 2).

• Measure vent tile flow rates and understand areas of stronger or weaker airflow delivery (Figure 3).

January 2005 Page 7 Dell Enterprise Product Group

• Obtain equipment flow rates. A good first order approximation to maximum deployment potential is to compare the average chilled air flow rate in an aisle to the equipment flow rate demand.

• Reprovision air where a higher demand exists. There are numerous types of vent tiles with varying resistance to flow. By putting more restrictive tiles or a lesser number of vent tiles in areas where the equipment requirements are lower, greater volumes of chilled air can be rerouted to areas with greater chilled air requirements. Floor tile examples (Figure 4) show different types of vent tiles with approximate relative delivery potential.

• Consider the use of supplemental cooling products like Liebert XD1 systems or APC InfrastruXure2 to increase chilled air delivery.

• Consider the use of CFD analysis (Figure 1) to aid with the understanding of airflow and temperatures throughout the data center.

Figure 2 - Floor tile cable grommet

January 2005 Page 8 Dell Enterprise Product Group

Figure 3 - Measuring airflow and air temperature

Figure 4 - Floor tile options: 1X provides a baseline air flow rate that is dependent on the efficiency of a data center. 14X provides the greatest air flow rate of all tile options displayed in this figure. Assuming that a data center is perfectly efficient, the tiles labeled 14X should provide 1400% greater air flow than the tiles labeled 1X.

January 2005 Page 9 Dell Enterprise Product Group

Figure 5 – The NCSA Tungsten Cluster using Dell PowerEdge 1750s is cooled with a combination of 20% open perforated tiles and 60% open grates. The grates deliver 3X the flow rate of the adjacent perforated tiles.

January 2005 Page 10 Dell Enterprise Product Group

Section 4 PowerEdge 1855 Single System Data

Figure 6 - PowerEdge 1855 single system data sheet

January 2005 Page 11 Dell Enterprise Product Group

Depending on the application, a fully loaded PowerEdge 1855 chassis with ten fully loaded blade servers is expected to have a maximum power dissipation of 4,170W (Figure 6, configuration B). Each 1200W power supply, in an optional 3+1 redundant configuration, requires 200-240VAC single phase input voltage. The system will not operate with 120V power. The required amperage will vary from an expected 19A (at 220V) to 21A (at 200V) depending on the source voltage, yet the expected power draw remains the same independent of a redundant or non-redundant power supply configuration. Each PowerEdge 1855 1200W power supply comes with one jumper cord (figure 7) and each jumper cord contains one C13 female plug and one C20 male plug. The C13 plug connects to the power supplies while the C20 plug connects to a Power Distribution Unit (PDU). The PowerEdge 1855 power supplies cannot be plugged directly into a power source located in the wall. PDUs support a variety of voltages and amperages, but only a subset will support the PowerEdge 1855. The following two PDUs are supported in North America: • The 16A Hybrid PDU with L6-20P plug - requires an L6-20R receptacle (20A

circuit, in North America). • The 24A PDU with L6-30P plug - requires an L6-30R receptacle (30A circuit,

in North America). Please note that it is difficult to visually identify receptacle and plug types, so Dell always recommends checking the embosses or labels on the plug and receptacle prior to making a connection.

Figure 7 - Example of how a fully loaded PowerEdge 1855 chassis with 1200W power supplies can be powered up using either 16A or 24A Dell supported PDUs in North America.

January 2005 Page 12 Dell Enterprise Product Group

Section 5 PowerEdge 1855 Additive System Data

Figure 8 - Data for multiple systems in rack (Note: Weight does not include rack or external cabling; Input power ranges are from minimum configuration idle to maximum configuration stressed).

January 2005 Page 13 Dell Enterprise Product Group

Section 6 PowerEdge 1855 Cabling Best Practices

Depending on the configuration, the PowerEdge 1855 can have up to 51 cables attached to the rear of the system. It is critical that these cables be dressed and routed properly in order to avoid impeding exhaust airflow and maximizing access to the rear modules. In addition to the Rack Installation Guide provided with the rail kit, the following is a list of best practices and pictures to aid in proper cable routing:

• It is recommended that labels be applied to both ends of all data cables prior to installation in order to ensure that cables are connected to the correct ports.

• The releasable cable ties provided with the rail kit should be threaded through the available holes in the rear rack flange (Figure 9).

• Cables and power cords should be routed through the cable ties and directed vertically along the rack flanges as indicated (Figure 10).

• Be sure to provide a large bend radius in the cable bundles for strain relief while avoiding the blocking of any fan outlet areas with the cable bundle (Figure 11). Infiniband cables should have a minimum bend radius of no less than 2.

• PowerEdge 1855 cables are intended for static rack-mount applications only and therefore should not be bundled in cable management arms (CMAs).

• For reference in cable cutout requirements, a rack containing six maximum configured PowerEdge 1855s could contain left and right side cable bundles of 31 cm (12 inches) in diameter at the base of the rack.

• Relieve stress on KVM connector by securing cable dongle to rack (Figure 12). KVM dongle should be secured independently from other cable bundles to ensure minimum weight load is transferred to the KVM connector and housing. KVM connector should be oriented horizontally to ensure proper contact between connector plug and receptacle is maintained.

January 2005 Page 14 Dell Enterprise Product Group

Figure 9 - Use of cable ties

Figure 10 - Example cable routing

January 2005 Page 15 Dell Enterprise Product Group

Figure 11 - Provide large bend radius

Figure 12 - Secure KVM dongle to rack

January 2005 Page 16 Dell Enterprise Product Group

Section 7 Glossary

• Inlet Ambient – Temperature of air entering system (°C or °F). • U – Unit of measurement equal to 1.75 inches. Rack system height is typically referred to

by this unit (e.g. system is 7U meaning it fits into a 7U envelope). • CFD – Computational Fluid Dynamics. For context in this white paper it is the finite

element analysis that is performed by which a virtual environment is created in which user defined inputs for power, heat, airflow and resistance are used to simulate an environment.

• CFM – Cubic feet per minute. Unit for volumetric flow rate (Metric – m3/Hr). • HVAC – Heating, Ventilation, and Air Conditioning • PDU – Power Distribution Unit • Variable speed – Fan blade RPM varies from low speed to high speed depending on inlet

ambient temperature or the state of fan operation. If a single fan fails in a system supporting redundant cooling, then all fans run at high speed until the failed fan is replaced.

• 2+2 Redundant – Refers to redundant power configuration by which 2 power supplies can fail or the loss of AC power from one branch and the system can still operate.

• Rated Input Voltage – The supply voltage as declared by the manufacturer. • Rated Input Current – The input current of the equipment as declared by the

manufacturer. • Rated Input Frequency – The supply frequency as declared by the manufacturer. • Rated Input Power – The supply power as declared by the manufacturer.

References 1) Liebert Web Site: www.liebert.com 2) APC (American Power Conversion): www.apc.com

THIS WHITE PAPER IS FOR INFORMATIONAL PURPOSES ONLY, AND MAY CONTAIN TYPOGRAPHICAL ERRORS AND TECHNICAL INACCURACIES. THE CONTENT IS PROVIDED AS IS, WITHOUT EXPRESS OR IMPLIED WARRANTIES OF ANY KIND.

Dell, PowerEdge, and PowerConnect are trademarks of Dell Inc. Intel and Xeon are registered trademarks of Intel Corp. Other trademarks and trade names may be used in this document to refer to either the entities claiming the marks and names or their products. Dell disclaims proprietary interest in the marks and names of others.

©Copyright 2005 Dell Inc. All rights reserved. Reproduction in any manner whatsoever without the express written permission of Dell Inc. is strictly forbidden. For more information, contact Dell. Information in this document is subject to change without notice.