Data Center Energy Efficiency

Sonoma Mountain VillageNovember 29, 2007William Tschudiwftschudi@lbl.gov

Acknowledgement of LBNL Acknowledgement of LBNL sponsorssponsors

California Energy Commission – PIER California Energy Commission – PIER programprogram

Pacific Gas and Electric CompanyPacific Gas and Electric Company New York State Energy and New York State Energy and

Development Agency (NYSERDA)Development Agency (NYSERDA) US - Environmental Protection AgencyUS - Environmental Protection Agency US – Department of EnergyUS – Department of Energy

Energy Commission 101Energy Commission 101

The state’s primary energy policy and The state’s primary energy policy and planning agency charged with:planning agency charged with:

Licensing thermal power plants 50 megawatts and larger

Adopting appliance and building energy efficiency standards

Forecasting state energy use

Maintaining energy information and performing analysis

Proposing to the Governor integrated state energy policies

Conducting public interest energy research

PIER ProgramPIER Program

Established in California by legislation Established in California by legislation in 1997 in 1997

$62.5 million collected annually from $62.5 million collected annually from investor-owned electricity utility investor-owned electricity utility ratepayers for “public interest” energy ratepayers for “public interest” energy research, development and research, development and demonstration (RD&D) projectsdemonstration (RD&D) projects

PIER Program ObjectivesPIER Program Objectives

Improve energy cost/valueImprove energy cost/value

Improve environment, public health, Improve environment, public health, and safetyand safety

Improve electricity Improve electricity reliability/quality/sufficiencyreliability/quality/sufficiency

Strengthen the economyStrengthen the economy

Provide consumer choiceProvide consumer choice

Address research gapsAddress research gaps

PIER Segments of ResearchPIER Segments of Research

Buildings End-Use Energy Efficiency Buildings End-Use Energy Efficiency Energy-Related Environmental Research Energy-Related Environmental Research Energy Systems Integration Energy Systems Integration Environmentally-Preferred Advanced Environmentally-Preferred Advanced

Generation Generation Industrial/Agricultural/Water End-Use Energy Industrial/Agricultural/Water End-Use Energy

Efficiency Efficiency Renewable Energy Technologies Renewable Energy Technologies Climate Change ProgramClimate Change Program Transportation ResearchTransportation Research

Data Center research activitiesData Center research activities

Research RoadmapResearch Roadmap Benchmarking and 23 data center case Benchmarking and 23 data center case

studies studies Self-benchmarking protocol Self-benchmarking protocol Power supply efficiency studyPower supply efficiency study UPS systems efficiency studyUPS systems efficiency study Standby generation lossesStandby generation losses Performance metrics – Computation/wattPerformance metrics – Computation/watt EPA report to CongressEPA report to Congress DOE Save Energy Now programDOE Save Energy Now program

Energy efficiency varies Energy efficiency varies

There are many features in data center There are many features in data center design and operation that must all be design and operation that must all be optimized in order to achieve highest optimized in order to achieve highest efficiencyefficiency

There is no silver bulletThere is no silver bullet Changing paradigms may be Changing paradigms may be

necessarynecessary– DC power (Edison was right!)DC power (Edison was right!)– Liquid coolingLiquid cooling– Use of outside air for coolingUse of outside air for cooling

Server Load/ComputingOperations

Cooling Equipment

Power Conversion & Distribution

AlternativePower

Generation

• High voltage distribution

• Use of DC power

• Highly efficient UPS systems

• Efficient redundancy strategies

• Load management

• Server innovation

Energy efficiency opportunities Energy efficiency opportunities are everywhereare everywhere

• Better air management

• Move to liquid cooling

• Optimized chilled-water plants

• Use of free cooling

• On-site generation

• Waste heat for cooling

• Use of renewable energy/fuel cells

Benchmark results helped to Benchmark results helped to find best practicesfind best practices

The ratio of IT equipment power to the The ratio of IT equipment power to the total (or its inverse) is an indicator of total (or its inverse) is an indicator of relative overall efficiency. Examination relative overall efficiency. Examination of individual systems and components of individual systems and components in the centers that performed well in the centers that performed well helped to identify best practices.helped to identify best practices.

Ratio of electricity delivered Ratio of electricity delivered to IT equipmentto IT equipment

IT Power to Total Data Center Power

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

Data Center Number

Ra

tio

Average .57

Higher is better

Source: LBNL Benchmarking

Performance varies

Data Center Server Load

51%

Data Center CRAC Units

25%

Cooling Tower Plant4%

Electrical Room Cooling

4%

Office Space Conditioning

1%

Lighting2%

Other13%

Computer Loads67%

HVAC - Air Movement

7%

Lighting2%

HVAC - Chiller and

Pumps24%

The relative percentages of the energy actually doing computing varies considerably.

Best HVAC PracticesBest HVAC Practices

Air ManagementAir Management Air EconomizersAir Economizers Humidification Humidification

ControlControl Centralized Air Centralized Air

HandlersHandlers Low Pressure Low Pressure

Drop SystemsDrop Systems Fan EfficiencyFan Efficiency

Cooling Plant Cooling Plant OptimizationOptimization

Water Side Water Side EconomizerEconomizer

Variable Speed Variable Speed ChillersChillers

Variable Speed Variable Speed PumpingPumping

Direct Liquid Direct Liquid CoolingCooling

Best Electrical PracticesBest Electrical Practices

UPS systemsUPS systems Self-generationSelf-generation AC-DC AC-DC

distributiondistribution Standby Standby

generationgeneration

Sonoma climate and efficiencySonoma climate and efficiency

Grapes aren’t the Grapes aren’t the only thing that only thing that benefits from this benefits from this climateclimate

Majority of cooling Majority of cooling can be achieved can be achieved without chillerswithout chillers

0

50

100

150

200

250

300

350

400

450

500

Hrs/YR

28 35 42 49 56 63 70 77 84 91 98 105 112

Temperature (F)

Berkeley Weather

hr/yr0 10 0

11 20 021 30 331 40 50441 50 232551 60 315361 70 154271 80 84681 90 29791 100 75

101 110 15111 120 0

Range F

Most hours of the year can be cooled without chillers

LEEDLEEDTM TM criteria and data centerscriteria and data centers

The US Green Building Council established The US Green Building Council established LEED criteria for commercial buildingsLEED criteria for commercial buildings

Points are given for sustainable elementsPoints are given for sustainable elements– e.g. bikeracks, bamboo floors, waterless urinalse.g. bikeracks, bamboo floors, waterless urinals

While these are wonderful to consider for any While these are wonderful to consider for any building, the real action in data centers is with building, the real action in data centers is with energy and water consumptionenergy and water consumption

A different point rating system is neededA different point rating system is needed

LEEDLEEDTM TM type criteria points for type criteria points for data centersdata centers

LBNL and industry are developing proposed LBNL and industry are developing proposed criteria to be used in ranking data centerscriteria to be used in ranking data centers

Consider different metrics:Consider different metrics:– Ratio of IT power to total powerRatio of IT power to total power– Efficiency of power deliveryEfficiency of power delivery– Efficiency of chilled water plantEfficiency of chilled water plant– Air management effectiveness Air management effectiveness – Environmental conditionsEnvironmental conditions– Mission of the data centerMission of the data center– etc.etc.

Completion planned for August, 2008Completion planned for August, 2008

LEEDLEEDTM TM type criteria type criteria developmentdevelopment

Green Grid, Uptime Institute, ASHRAE, Green Grid, Uptime Institute, ASHRAE, Critical Facilities Roundtable Critical Facilities Roundtable

EPA and DOE supportiveEPA and DOE supportive Other industry input welcomeOther industry input welcome

LBNL data center LBNL data center demonstrationsdemonstrations

““Air management” demonstrationAir management” demonstration

Outside air economizer demonstrationOutside air economizer demonstration

Use of infrared thermographyUse of infrared thermography

DC powering demonstrationDC powering demonstration

Silicon Valley Leadership Group – “chill-off”Silicon Valley Leadership Group – “chill-off”

Measured fan energy savingsMeasured fan energy savings – – 75%75%

If mixing of cold supply If mixing of cold supply air with hot return air air with hot return air can be eliminated-can be eliminated-fan speed can be fan speed can be reducedreduced

Better temperature control can Better temperature control can allow raising the temperature allow raising the temperature in the entire data center!in the entire data center!

Cold Aisle NW - PGE12813

40

45

50

55

60

65

70

75

80

85

90

6/13/2006 12:00 6/14/2006 0:00 6/14/2006 12:00 6/15/2006 0:00 6/15/2006 12:00 6/16/2006 0:00 6/16/2006 12:00

Time

Tem

per

atu

re (

deg

F)

Low

Med

High

Baseline Alternate 1

Setup

Setup

Alternate 2

ASHRAE Recommended Range

Ranges during demonstration

See the problem areasSee the problem areas

Infrared thermography and CFD modeling can be used as visualization tools

Inverter

In Out

Bypass

Battery/ChargerRectifier

Internal Drive

External Drive

I/O

Memory Controller

Processor

SDRAM

Graphics Controller

DC/DCAC/DC

DC/DC

AC/DC Multi output PS

Voltage Regulator Modules

5V

12V

3.3V

12V 1.5/2.5V

1.1V-1.85V

3.3V

3.3V

12V

PWM/PFCSwitcher

Unregulated DCTo Multi Output Regulated DC

Voltages

Data center power conversions

AC voltage conversions

With over 25 industry partners direct DC powering of servers was demonstrated

Typical AC distribution Typical AC distribution todaytoday

DC/ACAC/DC480 VACBulk Power

Supply

UPS PDU

AC/DC DC/DC VRM

VRM

VRM

VRM

VRM

VRM

12 V

Loadsusing

LegacyVoltages

Loadsusing

SiliconVoltages

12 V

5 V

3.3 V

1.2 V

1.8 V

0.8 VServer

PSU

480 Volt AC

Facility-level DC Facility-level DC distributiondistribution

AC/DC480 VACBulk Power

SupplyDC UPS

orRectifier

DC/DC VRM

VRM

VRM

VRM

VRM

VRM

12 V

Loadsusing

LegacyVoltages

Loadsusing

SiliconVoltages

12 V

5 V

3.3 V

1.2 V

1.8 V

0.8 VServer

PSU

380 VDC380V.DC480

Volt AC

AC system loss compared to DCAC system loss compared to DC

DC/ACAC/DC480 VACBulk Power

Supply

UPS PDU

AC/DC DC/DC VRM

VRM

VRM

VRM

VRM

VRM

12 V

Loadsusing

LegacyVoltages

Loadsusing

SiliconVoltages

12 V

5 V

3.3 V

1.2 V

1.8 V

0.8 VServer

PSU

AC/DC480 VACBulk Power

SupplyDC UPS

orRectifier

DC/DC VRM

VRM

VRM

VRM

VRM

VRM

12 V

Loadsusing

LegacyVoltages

Loadsusing

SiliconVoltages

12 V

5 V

3.3 V

1.2 V

1.8 V

0.8 VServer

PSU

380 VDC

7-7.3% measured improvement

2-5% measured improvement

Rotary UPS

ChallengeChallenge

Can Sonoma Mountain Village’s data Can Sonoma Mountain Village’s data center demonstrate these energy center demonstrate these energy saving technologies? saving technologies?

LBNL is seeking partnering sites to LBNL is seeking partnering sites to demonstrate DC powerdemonstrate DC power

Thank youThank you

website: website:

http://hightech.lbl.gov/datacenters/http://hightech.lbl.gov/datacenters/