Embed Size (px)
Citation preview
Evolution of Facebook’sdata center design
Veerendra Mulay, PhDStrategic Engineering and Development
1 Introduction
2 Electrical design
3 Mechanical design
4 Design evolution
5 RDDC
Agenda
Introduction
Facebook Greenfield DatacenterGoal Design and build the most efficient datacenter eco-system possible
Control Application Server configuration Datacenter design
Sites Prineville, OR Forest City, NC Luleå, Sweden Altoona, IA
Prineville, OR
Forest City, NC
Luleå, Sweden
Altoona, IA
Electrical design
Electrical Overview Eliminate 480V to 208V transformation Used 480/277VAC distribution to IT equipment
Remove centralized UPS Implemented 48VDC UPS System
Result a highly efficient electrical system and small failure domain
Typical Power Prineville Power
Utility Transformer480/277 VAC
99.999%Availability
Total loss up to server:
2% loss
6% - 12% loss
3% loss
208/120VAC
AC/DC
DC/AC
ASTS/PDU
SERVER PS
StandbyGenerator
10% loss(assuming 90% plus PS)
Utility Transformer480/277 VAC
99.9999% Availability
2% loss
480/277VAC
Total loss up to server:
FB SERVER PS
StandbyGenerator
48VDC DC UPS(Stand-by)
5.5% loss
UPS480VAC
21% to 27% 7.5%
DC UPS Backup Scheme
Server PS
Back-up Converter
48VDC-12VDC
AC to DC
277VAC-12VDC
12VDC
MOBO
48VDC Standby
277VAC48VDC
Battery Charger
4x12V Batt
4x12V Batt
4x12V Batt
4x12V Batt
4x12V Batt
48VDC Battery CabinetNormal Power
Backup Power
Battery Cabinet
Custom DC UPS
56kW or 85kW
480VAC, 3-phase input
45 second back-up
20 sealed VRLA batteries
Battery Validation System
Six 48VDC Output
Two 50A 48VDC aux outputs
Reactor Power Panel Custom Fabricated RPP
Delivers 165kW, 480/277V, 3-phase to CAB level
Contains Cam-Lock connector for maintenance wrap around
Line Reactor
Reduces short circuit current < 10kA
Corrects leading power factor towards unity (3% improvement)
Reduces THD for improved electrical system performance (iTHD 2% improvement)
Power consumption = 360 Watt
Mechanical design
Overview
Removed Centralized chiller plant HVAC ductwork
System Basis of Design ASHRAE Weather Data: N=50 years TC9.9 2008: Recommended Envelopes
Built-up penthouse air handling system
Server waste heat is used for office space heating
Typical Datacenter Cooling
Forest City Datacenter Cooling
Ductless Return Air
Fan Wall
Misting System
100% Outside Air Intake
DUCTLESSSUPPLY
Return Air
Filter Wall
Return Air Plenum
Ductless Relief Air
Relief Air
DATA CENTER
SUPPLY DUCT
CT CHILLER AHUDATA CENTER
Return Ductwork
DX System
Prineville Datacenter Cooling
Fan Wall
Misting System
100% Outside
Air Intake
Hot AisleReturn Air
Filter Wall
Relief Air Corridor
CommonReturn Air Plenum
Relief Air Fan
Supply Air Corridor
Data Center
Mixed Air Corridor
Server Cabinets
Server Cabinets
Hot Aisle
Hot Aisle Relief Air
Hot AisleCommonCold Aisle
PRN1 Datacenter Cooling
Outside air intake corridorVertical drainable louvers
Air mixing section Evaporative cooling / humidification corridor
DX Cooling CoilForest City Only
50 year extreme weather conditions
Fan array outlet / supply air corridorFan wall array inlet corridor
Cold aisle pressurization –ductless supply
Design evolution
80ºF inlet
65% humidity
20ºF ΔT
PRN1A1B
85ºF inlet
80% humidity
22ºF ΔT
85ºF inlet
90% humidity
22ºF ΔT
85ºF inlet
80% humidity
22ºF ΔT
PRN1C1D FRC1A1B LLA1A1B
Basis of Design Comparison
Wetted Media
Typical Generator Topology
LLA Reduced Generator Topology
Rapid Deployment Data Center
RDDCFLAT PACKEDPANELIZED ASSEMBLIES
PRE MANUFACTURED UNIT ASSEMBLIES
RDDC Pre-fab Assemblies
A
MECH UNITS
MECH UNITS
BELECSKIDS
MECH UNITS
MECH UNITS
RDDC building plans
RDDC
▪ Unitized IT Modules
RDDC
▪ Section
Unit IT Unit IT a b c
RDDC
▪ Power Modules
Luleå, Sweden
RDDC Flat Pack Solution
RA
CK
RA
CK
16’-0”16’-0”
14’-0
”14
’-0”
12’-0”12’-0”
Flat pack solution
Flat pack solution
RA
CK
RA
CK
Flat pack solution
Leaner simplification
ALTOONA
www.opencompute.org
(c) 2007 Facebook, Inc. or its licensors. "Facebook" is a registered trademark of Facebook, Inc.. All rights reserved. 1.0
