Load Capacity and Thermal Efficiency Optimization of a Research

Data Center Using Computational Modeling

Joseph SchaadtUndergraduate, SeniorVillanova University

• A data center is a large group of networked computer servers typically used by organizations for the remote storage, processing, or distribution of large amounts of data

• In 2007, 1.5% of the electricity consumption in the United States was used for data centers

• Of the 1.5%, one-third to one-half of this energy was used for cooling

• Data centers are most commonly cooled by air delivered to electronic equipment from centralized cooling systems.

Introduction

2

• Data centers are one of the largest and fastest growing consumers of electricity in the world

• Data centers, which power the internet, are a key part of the infrastructure in the United States

• There has been a push in recent years to improve their energy efficiency

• The Villanova Steel Orca Research Center is a research data center that will be built in Princeton, New Jersey that will allow for testing of improved energy efficient cooling strategies

Introduction (cont.)

3

Data Centers

4

• In 2013, US data centers– Consumed 91 billion kWh of electricity (2.5% of total consumption)– Emitted nearly 100 million metric tons of CO2 annually – Projected to increase to 140-200 billion kWh annually by 2020

• Data centers are commonly cooled by air from centralized cooling systems– 40% - 50% of energy consumed in data centers is for cooling

Motivation

5

Optimize cooling system design of a research data center in both load capacity and thermal efficiency using CFD

Villanova Steel Orca Research Center (in design stages) will allow for investigation of:• Cooling strategies- Perimeter, In-row, Overhead, or Hybrid cooling• Containment- Hot & cold containment• Layout & load distribution

Objective and Research Goals

6

Determine optimal operating conditions & design layout while meeting thermal constraints

• Minimization function: Total energy consumed for cooling

• Design variables: • Air conditioning units (ACU) air flow rate• Chiller supply temperature setpoint (CHWST)

• Constraint: Racks’ maximum inlet temperature < 85˚F

• Investigate effects of containment

• This research is motivated by a push for improved energy efficiency for data centers

• State of the art data center cooling strategies will be investigated

• CFD software 6SigmaRoom provides ability to easily test various strategies such as high density zones and hot and cold aisle containment

• Efficient strategies will be validated at VSORC facility

Objective and Research Goals (cont.)

7

Containment

8

Enclosures to control travel path of airflow

CRAH

Containments:• Enclose hot or cold aisles• Prevent premature mixing of hot & cold streams – reduce cooling load• Requires racks layout & height uniformity • Are difficult to install and remove

Both hot & cold containment strategies were investigated

• Containment enclosures are installed in data centers to control the path of airflow travel

• Containment helps in reducing the air inlet temperatures to the IT equipment

• Containment does not change air movement, it can be difficult to install and remove, and it requires uniformity in the server cabinet layout and height

Containment (cont.)

9

• Addition of containment (hot or cold) does not directly alter power usage effectiveness (PUE), a popular measure of the energy efficiency of a data center:

• Containment reduces premature mixing of hot and cold air streams, resulting in reduced cooling load

Containment (cont.)

10

PUE=𝑇𝑜𝑡𝑎𝑙 𝐹𝑎𝑐𝑖𝑙𝑖𝑡𝑦 𝑃𝑜𝑤𝑒𝑟𝐼𝑇 𝐸𝑞𝑢𝑖𝑝𝑚𝑒𝑛𝑡 𝑃𝑜𝑤𝑒𝑟

The Center for Energy-Smart Electronic Systems

• Containment prevents the mixing of different airflow paths and damage to the IT equipment

• Containment can help reduce the inlet temperatures of the IT equipment and the needed cooling power

Containment (cont.)

Without Hot Aisle Containment Full Hot Aisle Containment

High Density Zones

12

High power server/racks (high kW) are clustered together in one zone

– Allows for higher computing power per unit area

High density zones cause cooling issues requiring additional cooling

Additional cooling accomplished through In-row coolers

high density zones in-row

Research Methods

13

1. Two distinct VSORC design configurations have been modeled

Each model required extensive clean up of initial layout by removing numerous collision and cooling errors

2. Optimization modelDesign variables: total supply flow rate and chiller supply temperatureDesign constraint: maximum rack inlet temperature no greater than 85°F

Method of Investigation

14

3. The baseline model must be used to create different containment configurations

4. Perform CFD simulations to find most efficient containment configuration

5. Create matrix based on power, max inlet temperature, and total supply flow rate

6. Utilize Minitab© to create predictive equations and identify optimal operating point

7. Based on results, either validate or reoptimize

Method of Investigation (cont.)

15

Optimization Procedure

16

Combination of CFD & DOE (w/factorial analysis)

17

CFD Models

Without Hot Aisle Containment

Full Hot Aisle ContainmentHot aisle containment

FullPartial*None

Cold aisle containmentFull

Partial*None

6 Models

• Industry’s best practices employed to create models Containment leakageEquipment gaps & interferencesSupply tile locationsChiller supply temperature setpoint rangeTotal flow rate range

* Partial configurations enclosed only high density zones

Data Center Model

18

IT Equipment8 rows36 racks 550 kW total IT power (total heat load)

2 rows of 10kW racks2 rows of 5kW racks4 rows of high density 40kW racks

Cooling Equipment2 CRAH units16 In-Row coolers

ContainmentVarious containment strategies

hot aisle

cold aisle cold aisle

hot aisle

cold aisle

hot aisle

CR

AH

CR

AH

hot aisle

Cabinet Inlet Temperature Layout

19

Data and Results

20

CFD Results

21

Flow rate effect on max inlet temperature

√

CHWST effect on max inlet temperature

√√

Power Consumption

22

𝑃 𝑓𝑎𝑛2

𝑃 𝑓𝑎𝑛1=( �̇� 2

�̇� 1)

3

Tsupply = chiller supply temperature (CHWST)

COP = chiller coefficient of performance

= total heat load

Fan power consumption is calculated from the use of the Fan Affinity Laws

= fan power ; = total supply flow rate

Chiller power consumption is calculated from a correlation developed by HP

𝐶𝑂𝑃=0.0068 (𝑇 𝑠𝑢𝑝𝑝𝑙𝑦)2+0.0008(𝑇 ¿¿ 𝑠𝑢𝑝𝑝𝑙𝑦 )+0.458 ¿

𝑃 h𝑐 𝑖𝑙𝑙𝑒𝑟=�̇�

𝐶𝑂𝑃

(𝑻𝒐𝒕𝒂𝒍𝒑𝒐𝒘𝒆𝒓 )𝑷𝒕𝒐𝒕𝒂𝒍=𝑷 𝒇𝒂𝒏+𝑷 𝒄𝒉𝒊𝒍𝒍𝒆𝒓

Optimization

23

Regression analysis employed to obtain predictive equations for:

Total system power consumption

Maximum server inlet temp. 

(r2 =94.17% & 99.89%)

= total supply flow rateTsupply = CHWST

Findings

24

• Both chiller supply temperature setpoint & ACU total supply flow rate impact data center total power consumption

• CHWST has much greater effects on data center thermal efficiency

Parameter ValueTotal Supply Flow Rate, cfm 95,200CHWST, °F 70Total Power Consumption, kW 218Maximum Inlet Temperature, °F 85

• The supply temperature setpoint of the chiller was found to significantly change the power consumption of the data center

• The total supply flow rate was found to only slightly change the power consumption of the data center

• A combined CFD & DOE (w/factorial analysis) methodology was developed for design & optimization of a data center

• 1st law analysis was employed to determine cooling system’s optimal setting for improved energy efficiency

Findings (cont.)

25

26

• Containment is beneficial in reducing air inlet temperatures, leading to an improvement in energy efficiency

• CFD programs like 6SigmaRoom can be effectively used to identify optimal operating points of total supply flow rate and chilled water supply temperature setpoint for a data center

• Hybrid cooling is effective in handling high density IT zones of a data center

Conclusions

Acknowledgements• My research advisors, Dr. Kamran Fouladi and Dr.

Aaron Wemhoff of the Mechanical Engineering Department along with Dr. Joseph Pigeon from the Mathematics and Statistics Department

• Thomas Wu and Aitor Zabalegui from Future Facilites Inc.

• The National Science Foundation (NSF) Research Experience for Undergraduates (REU)

• NSF I/UCRC in Energy-Smart Electronic Systems