1 Randall Poet A C Systems. Server Load = CRAC Capacity Server Airflow = CRAC Airflow 2

Cooling:Best Practices and Economizers

Cooling:Best Practices and Economizers

1

Randall Poet

A C Systems

Server Load = CRAC Capacity Server Airflow = CRAC Airflow

2

Ideal Situation

The Conventional Approach – Base CaseRaise the Return / Supply Air TempContain the Cold Aisle Add Intelligent ControlThe Set-upSummary

3

Agenda

Hot aisle / Cold aisle Blanking panels in racks Blanking panels between racks Blanking Panels between racks and floor Cable cut-outs covered Relatively clean underfloor area Proper location of CRAC units Vapor barrier around space

4

Start with Best Practices

20 Racks◦ 8 kW each◦ 35F delta T

5

Base Case – 75F Return Air Control 4 CRACs (N+1)

54 kW each 21F delta T design

20 Racks◦ 8 kW each◦ 35F delta T

6

Base Case – 75F Return Air Control 3 CRACs

Running 54 kW each 21F delta T design

Unit Airflow – 24,000 CFM Rack Airflow – 14,600 CFM Bypass Airflow – 9,400 CFM

7

Base Case Operating Scenario

75F

89F

54F

54F

CFM 100%

8

The Airflow Pattern

9

Typical Air Cooled DXEnergy Consumption

Evap Fan MotorCompressors

Condenser Fan Motors

Compressors run at full capacity Fans run at full speed

10

Typical Energy Consumption

Evap Fan MotorCompressors

Condenser Fan Motors

System kW 75F RA

Compressors 35.4

Evap Fans 8.7

Condenser Fans

5.4

Total 49.5


11

Agenda


12

New Operating Scenario85F

99F

64F

64F

CFM 100%

13

Increased Capacity at Full Airflow

14

Increased Capacity at Full Airflow

Compressors run at reduced capacity or unloaded Fan motors run at full speed

15

Operating Systems Comparison

System kW 75F RA 85F RA

Compressors 35.4 24.9

Evap Fans 8.7 8.7

Condenser Fans 5.4 5.4

Total 49.5 39.0

Base 78.8%

Sanity Check –100 dF OAT, 6400’ ASL

16

Sanity Check

17


18

Agenda


19

Contain the Cold Aisle85F

94F

62F

62F

CFM 90%

At the higher RA temperature, the contained system has very similar operating costs as the non-contained

Fan motors run at full speed but at a reduced CFM and HP due to the higher static pressure

Compressors run at reduced capacity or unloaded but slightly higher than the non-contained

20

Operating Systems ComparisonSystem kW 75F RA 85F RA 85F RA

Contained

Compressors 35.4 24.9 25.8

Evap Fans 8.7 8.7 7.5

Condenser Fans

5.4 5.4 5.4

Total 49.5 39.0 38.7

Base 78.8% 78.2%

21

Why use Containment??

System operating costs are similarContainment partitions and doors cost $$

22

Hot Spots in Racks due toWrap-Around

23

No Leakage into the Cold AIsle

24

Higher Temperatures without Containment

25

Containment of Cold Aisle

Issues to Consider Fire Detection / Suppression

◦ Wide variation between municipalities◦ If local Fire Inspector involved early, typically goes

well Curtains usually eliminate this issue

Installation◦ Will be site specificy

Irregular row length/height, gaps, etc.◦ Site conditions critical, one size does NOT fit all

Issues To Consider ADA

◦ CAC space is for Service Personnel (Section 4.1.1) What about cooling for components in rest

of room?◦ Best solution today may be ducted return from hot

aisle◦ Perf tiles near other equipment requiring cooling

(eg. UPS) 85⁰ Room temperature?

HAC vs CAC Main Purpose of Cooling in Data Center?

◦ Cool the equipment Data Centers commonly on raised floor

◦ CAC allows current investment to be used HAC typically requires in row cooling

◦ So, refrigerant or chilled water and condensate intermingled with IT equipment and racks

Which aisle does majority of work take place in?◦ CAC hot aisle likely in 85⁰F range

Can use perf tiles in other space◦ HAC hot aisle likely in 100⁰F range


29

Agenda


30

Closer to Matching the Load to the Cooling

92F

97F

62F

62F

CFM 70%

Variable Capacity Compressors

Variable Speed Fans Intelligent Control

The airflows and capacities/loads are more closely matched

Fan motors run at reduced speed, CFM and HP based on the demand in the contained area

Compressors run at reduced capacity or unloaded but slightly higher than the non-contained

31

Operating Systems ComparisonSystem kW 75F RA 85F RA 85F RA

Contained92F RA Contained

& Controlled

Compressors 35.4 24.9 25.8 25.5

Evap Fans 8.7 8.7 7.5 3.0

Condenser Fans 5.4 5.4 5.4 5.4

Total 49.5 39.0 38.7 33.9

Base 78.8% 78.2% 68.5%

The Conventional Approach – Base CaseRaise the Return / Supply Air TempContain the Cold Aisle Add Intelligent Control (Creating SmartAisle)The Set-upSummary

32

Agenda

33

Sensor Location• Server centric solution, meaning that it focuses on the inlet

temperature to the servers• Self adapting to environment changes due to server utilization,

equipment location changes and outside variables• Can adapt to situations with no containment, end containment, and

full containment

Rack Sensors• Rack Sensors without doors

can be mounted on the frame of the racks.

• Temperature differences were within .5°F

Cold Aisle Sensors• Sensors can also be mounted at the top

of cold aisles when rack mounting is unavailable

36

Supply Compensation

• Compensation is the magical link between controlling sensors• The controller evaluates the Rack Sensors and Fan Speed• If the fan is operating at 100% and remains above the cold

aisle set point• Then the supply temperature set point will slowly lower to

drive the correct cold aisle temperature

Lower Supply

Fan Speed100%

CW Valve Open %

100%90%80%70%60%50%40%30%20%10%0%

Fan Speed %

Controller and CRAC Operation

Incre

ase in

kW

Unit ON

IT Load IncreasesRack Temperature Sensor detects inlet rack temperature

Fan Speed IncreasesSo that cold aisle temperature is maintained at customer temperature Setpoint

Supply TemperatureThe increase in fan speed will result in a warmer supply air temperature which is detected by the supply air sensor that will increase cooling to maintain supply air setpoint

Cold Aisle ContainmentiCOM will automatically adjust to changes that result in a temperature increase or decrease

38

Controller and CRAC Operation• Advanced freeze protection routine

• Allow all units to reduce fan speed to 60%• Fan speed and compressor capacity (or CW valve) managed for

best unit efficiency and performance• Multiple remote sensors

• Controller can use averaged and maximum/minimum values to individually control multiple CRAC systems

• CRAC systems work as a team• All remote sensors used• Increase capacity of other applicable adjacent units if one is at

maximum and unable to handle the load• Automatically adjust for units not in service


39

Agenda

Issues to Consider Fire Detection / Suppression

◦ Wide variation between municipalities◦ If local Fire Inspector involved early, typically goes

well Curtain eliminates this issue

Installation◦ Will be a local responsibility◦ You need to develop relationship with local Cable

Contractor or similar company for installation Irregular row length/height

◦ Site Survey critical◦ Most expensive component

Issues To Consider ADA

◦ Consultant raised issue at BAIS project◦ Successfully defended CAC space is for Service

Personnel (Section 4.1.1) What about cooling for components in rest

of room?◦ Best solution today is ducted return from hot aisle◦ perf tile by IT component

85⁰ Room temperature

Issues To Consider Purpose of Cooling in Data Center?

◦ Control equipment inlet temps Data Centers often on raised floor

◦ CAC allows current investment to be used Which aisle does majority of work take place

in?◦ CAC hot aisle likely in 85⁰F range

Could use perf tile in other space◦ HAC hot aisle likely in 100⁰F range

Best practices are a must if improved efficiency is a goal

Running warmer temperatures in the space will improve the cooling system operating efficiency

Containment improves the availability of the servers by eliminating hot spots

Intelligently controlling fan speed and compressor capacity “balances” the system to operate at it’s most efficient level

43

Summary

Summary Efficiency is the story, not containment Containment can be done in several ways,

none necessarily fit all situations Each critical space is unique and merits

individual planning Dynamic Control is (currently )the final

element

…Shifting on to Economizers and Equipment Considerations…

45

90.1 Economizer Map

Definitions of Cooling Efficiency EER (Energy Efficiency Ratio)

◦ Total Cooling Capacity (BTUH)/Total power input (watts)

◦ Full-load value on 95F design day◦ Commercial return air conditions of 80F/50%

SCOP (Sensible Coefficient of Performance)◦ Sensible Cooling Capacity (kW)/Total power input

(kW)◦ Full-load value on 95F design day◦ Data Center return air conditions of 75F/45%

47

ASCOP – Efficiency MetricAnnualized Sensible Coefficient of

Performance(location specific)

Bin efficiency x bin hours/total annual hours Factors hours at each efficiency operating

point Factors part-load efficiency Factors economizer hours

48

Single System –Outside Air Economizer

49

Large or Multiple Systems –Outside Air Economizer

50

Single System withGlycol Economizer

51

Precooler on Chilled Water System

52

Chilled Water withCooling Tower Economizer

53

What if we could do a refrigerant based economizer cycle?

54

Energy Consuming ComponentsAir-Cooled System

55

Compressor energy reduction is low-hanging fruit

Digital Scroll Technology Continuous variable capacity

compressor technology without inverter drive from 10% to 100% of capacity

In commercial use since 2004 Available in single compressor

and tandem compressor configurations

High reliability versus Inverter drive systems

No electrical harmonics introduced

56

EC fans in unit

57

EC fans in raised floor

58

EconoPhase Pumped Refrigerant Economizer

59

Condenser

Check Valve

Compressors ON

EvaporatorElectronic expansion

valve

Summer operation

Check Valve

Check Valve

Refrigerant

Pumps OFF

SolenoidValve

EconoPhase

Outdoor Ambient 95°F

KW @ 70% Load 24.1

Circuit 2

Circuit 1

EconoPhase – Partial Economization

60

Condenser

Check Valve

Compressor

EvaporatorElectronic expansion

valve

Fall / Evening operation

Check Valve

Check Valve

Refrigerant

Pump

SolenoidValve

EconoPhase


KW @ 70% Load 15.1

Circuit 2

Circuit 1

ON

OFF

OFF

ON

EconoPhase - Full Economization

61

Condenser

Check Valve

Compressor

EvaporatorElectronic expansion valve

Winter operation

Check Valve

Check Valve

Refrigerant

Pump

SolenoidValve

EconoPhase


KW @ 70% Load 3.7

Circuit 2

Circuit 1

ON

OFF

ON

OFF

Technology

Electronic Expansion Valve EC Plug FansTandem compressors w/ Digital Scroll New Evaporator Coil design

◦ Staged “A” Coil◦ Greater surface area

Microchannel Condenser◦ Indoor Unit Communicates to Condenser

Refrigerant Economizer

Advantages ofRefrigerant Economizer

No additional heat exchangers No dust or contamination concerns No need for added ductwork or structural

changes to building for economizer No dampers to maintain Quick changeover between compressor

mode and economizer mode No water usage No water freeze or coil freeze concerns

63

45F 55F 65F 75F 85F 95F1.8

2.3

2.8

3.3

3.8

4.3

4.8

5.3

5.8

6.3

6.8

Outdoor Ambient, F

Eff

icie

ncy

, S

CO

PLiebert DSE Full-Load Efficiency SCOP (kW/kW) @ 85F return air

Liebert DSE DA125A @ 50% load

ASHRAE 90.1 @ 75F

Efficiency Plateau

+92%

Liebert DSE DA125A @ 100%

load

+45%

+115%

Traditional Refrigeration

With Refrigerant Economizer Maximum capacity, max airflow

65

0 10 20 30 40 50 600

2

4

6

8

10

12

14

16

0

20

40

60

80

100

120

140

SC

OP

, kW

/kW

Se

ns

ible

kW

= Sensible Capacity, kW

= SCOP

Ambient, ºF

Designing for LEED

Energy efficiencyNo water use for coolingMERV 13 filtersNo outside air contaminationR-410A refrigerant/lower charge

Utility Rebates

66

Condenser Improvements

67

Air Cooled Condenser Improvements

Microchannel vs. Fin and Tube Variable Speed Fans Control Based on Refrigerant Pressure

68

Aluminum Fins & Tubes Developed & used in Automotive A/C for

25+yrs

69

Microchannel Coil

70

Microchannel Coil Advantage

1” Thin coil replaces 2” to 6” fin/tube coil◦ Lower airside ∆P◦ Reduced maximum fan wattage for heat transfer◦ Reduced refirgerant quantity

6”1”

EC Axial Fan Technology EC (Electronically Commutated) Motor

◦ Every fan/motor is designed for variable speed. Superior part load efficiency

◦ 3-Phase AC power feed, but efficiency of a DC motor

◦ 20% reduction of fan speed = 49% savings in energy

71

Sound Quiet fan blade design

72

Sound Level Factors & Options

Sound varies with fan speed Fan speed is proportional to % compressor

load and outdoor ambient temperature◦ Will you ever hear a lightly loaded system

condenser? Even more quiet options

◦ Upsizing condenser will keep fan speed in nearly “inaudible” range!

73

Why do they cool things with air, anyway??

74

75

Why indeed? How about pumped refrigerant?

Refrigerant piping anifold suppliespumped refrigerant

Cam levers activates engagement and disengagement of server

Refrigerant connection points at top of the rack on left side. All plumbing runs along left side.

Cooling Architecture

Each server is cooled by an individual cold plate◦ Cold plates and insertion mechanism is designed for use of standard 1U

servers Circulating refrigerant is the cooling medium

Rack

Server

Heat exchanger &

circulation unit

Cold PlateTo/From Chiller

Cold Plate Highly flexible Backed with pressure

plate Forms to the server lid

In-Rack Cooling withPumped Refrigerant

The rack is 45U tall by 800mm wide by 1200mm deep.

36 1U server slots with cold plate Three non-cooled server slots for

“cool” devices like switches, etc. Can be configured for a cooling

capacity of 20kW or 40kW.

78

Thank You.Questions?

Documents

1 Randall Poet A C Systems. Server Load = CRAC Capacity Server Airflow = CRAC Airflow 2