DC Power Demonstration for HPC - Energy Efficient High ... · Facility-Level DC Distribution 480...

UCSD Project GreenLight: Dr. Tom DeFanti, P.I., Dr. Greg Hidley

LBNL: William Tschudi, PE

EPRI: Dennis Symanski

DC Power Demonstration for HPC

Why Consider DC Power?

• Power Distribution Conversion Losses

• Power Quality Issues

• Capital Cost Reduction

• Ease of Adopting DC Renewable Energy

• Other Infrastructure Energy Savings

– DC Variable Speed Drives

– DC Lighting

Conversion losses vary and

can be quite large

45%

50%

55%

60%

65%

70%

75%

80%

85%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

% of Nameplate Power Output

% E

ffic

ien

cy

Average of All Servers

Uninterruptible Power Supplies (UPS)

Power Supplies in IT equipment

Factory Measurements of UPS Efficiency

70%

75%

80%

85%

90%

95%

100%

0% 20% 40% 60% 80% 100%

Percent of Rated Active Power Load

Eff

icie

nc

y

Flywheel UPS

Double-Conversion UPS

Delta-Conversion UPS

(tested using linear loads)

source: LBNL research

HPC centers are not Data Centers

• Little use of UPS – One of the Losses goes away

• Some distribute high voltage AC close to load

• Some HPC vendors convert at rack level and distribute to rack

• Other benefits Still Available

Inverter

In Out

Bypass

Battery/ChargerRectifier

Internal Drive

External Drive

I/O

Memory Controller

m Processor

SDRAM

Graphics Controller

DC/DCAC/DC

DC/DC

AC/DC Multi outputPower Supply

Voltage Regulator Modules

5V

12V

3.3V

12V 1.5/2.

5V

1.1V-

1.85V

3.3V

3.3V

12V

PWM/PFCSwitcher

Unregulated DCTo Multi Output Regulated DC

Voltages

DC Power Demonstration Seeks to Reduce Data Center Power Conversions

Power Distribution Unit (PDU)

ServerUninterruptible Power Supply (UPS)

AC DC AC DC

Facility-Level DC Distribution

AC/DC480 VAC

Bulk Power

Supply

DC UPS

or

Rectifier

DC/DC VRM

VRM

VRM

VRM

VRM

VRM

12 V

Loads

using

Legacy

Voltages

Loads

using

Silicon

Voltages

12 V

5 V

3.3 V

1.2 V

1.8 V

0.8 VServer

PSU

380 VDC

6

380V.DC

Rotary UPS

BLDGPWR

Overview of 380VDC Power Distribution

DC Power is not New

• Many uses in Industry, Military, Transportation• DC Codes and Standards Exist for this

Application• UL Rated Equipment Exists• 380 V. Power Supplies Coming to Market

• Delta• IBM

• Unique Opportunity for World Wide Standard – For Voltages and Connectors

• Telecom Industry has used 48 V. DC Reliably Since the Beginning

IT specific connectors have been developed

Organizations Already Involved with >48 VDC

International Electrotechnical Commission (IEC)European Telecommunications Standards Institute (ETSI)International Telecommunications Union (ITU)- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -Electric Power Research InstituteLawrence Berkeley National Lab- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -The Green GridDC Power PartnersEmerge Alliance- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -Photovoltaic IndustryElectric Car IndustryLight Rail Industry- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -NTT/Orange-France Telecomm/Syracuse Univ/UC San Diego/Korea Telecom/---

The UCSD GreenLight Project: Instrumenting the Energy Use of Computational Science

• Measure, Monitor, & Web Publish Real-Time Sensor Outputs– Via Service-oriented Architectures

– Allow Researchers Anywhere To Study Computing Energy Cost

– Enable Scientists To Explore Tactics For Maximizing Work/Watt

• DC Powered Systems in Operation for Nearly 3 Months without incident

On-line Dashboard

“Tap”for details

Power utilization

Multiple perspectives

Enterprise reach

Zoom-in Analysis

2010.08.20 Calit2/UCSD [http://greenlight.calit2.net]

History over several days.

Zoom on desired time range.

Hint on each sample point.

Automatic average area.

Multiple sensors per asset with up to 1 min samplingresolution.

UCSD is Installing Zero Carbon EmissionSolar and Fuel Cell DC Electricity Generators

San Diego’s Point Loma Wastewater Treatment Plant Produces Waste Methane

UCSD 2.8 Megawatt Fuel Cell Power Plant Uses Methane

2 Megawatt of Solar Power Cells

Installed

Available Late 2011

Zero Carbon GreenLight Experiment:Direct DC-Powered Modular Data Center

• Concept—Avoid DC to AC to DC Conversion Losses– Computers Use DC Power Internally

– Solar and Fuel Cells Produce DC

– Both Plug into the AC Power Grid

– Can We Use DC Directly (With or Without the AC Grid)?

• DC Generation from Renewable Can Be Intermittent – Depends on Source

• Solar, Wind, Fuel Cell, Hydro

– Can Use Sensors to Shut Down or Sleep Computers

– Can Use Virtualization to Halt/Shift Jobs

• DC Generation Projects Just Starting– Collaboration with Oracle, LBNL, EPRI, Direct Power, Intel, Delta, Emerson

– NSF GreenLight Year 2 and Year 3 Funds

– California Energy Commission support

DC Fuel Cell 2800kWattsContainer<200kWatt

DC Power Partners

For more information

• http://greenlight.calit2.net

• http://hightech.lbl.gov/datacenters