Embed Size (px)
Citation preview
DENS: Data Center Energy-Efficient
Network-Aware Scheduling
Dzmitry Kliazovich University of Luxembourg
Pascal Bouvry University of Luxembourg
Samee Ullah Khan North Dakota State University
Dec 20, 2010
GreenCom 2010
Why energy is important? Increased computing demand
Data centers are rapidly growing Consume 10 to 100 times more energy per square foot
than a typical office building
Energy cost dynamics Energy accounts for 10% of data center operational
expenses (OPEX) and can rise to 50% in the next few years
Accompanying cooling system costs $2-$5 million per year
December 20, 2010 2Pascal Bouvry ([email protected])
GreenCom 2010
Distribution of data center energy consumption
December 20, 2010 3Pascal Bouvry ([email protected])
GreenCom 2010
Data center architectures Three-tier data center architecture
Most Widely Used Nowadays Access, Aggregation, and Core layers Scales to over 10,000 servers
December 20, 2010
S S S S S S S S S S S S S S S S S S S S S S S S S S S S S S S S S S S S
CoreNetwork
AggregationNetwork
AccessNetwork
S
Links
10 GE 1 GE
Nodes
L3 Switch L2/L3 Rack Switch Computing Server
4Pascal Bouvry ([email protected])
GreenCom 2010
Data center components Servers’ Energy Model
Pascal Bouvry ([email protected]) 5December 20, 2010
Fmin
Fmax
0
0.2
0.4
0.6
0.8
1
Pfixed
Ppeak
CPU FrequencyServer load
Pow
er c
onsu
mpt
ion
CPUmemory modules, disks, I/O resources
Idle server consumes about 66% of the peak load for all CPU frequencies
GreenCom 2010
Data center components Switches
Most common Top-of-Rack (ToR) switches typically operate at Layer-2 interconnecting gigabit links in the access network
Aggregation and core networks host Layer-3 switches operating at 10 GE (or 100 GE)
Links Transceivers’ power consumption depends on the quality of signal
transmission in cables and is proportional to their cost 1 GE links consume 0.4W for 100 meter transmissions over twisted
pair 10 GE links consume 1W for 300 meter transmission over optical fiber
Supported power management modes DVFS, DNS, or both
Pascal Bouvry ([email protected]) 6December 20, 2010
GreenCom 2010
Simulator components Switches’ Energy Model
Pascal Bouvry ([email protected]) 7December 20, 2010
Chassis~ 36%
Linecards~ 53%
Port transceivers~ 11%
GreenCom 2010
DENS methodology DENS achieves balance between
Energy consumed by the data center Individual job performances Job QoS requirements Data center traffic demands
DENS is architecture specific
Pascal Bouvry ([email protected]) 8December 20, 2010
CoreNetwork
AggregationNetwork
AccessNetwork
Rack Rack Rack
Module
Data Center Architecture
GreenCom 2010
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.2
0.4
0.6
0.8
1
L s(l)
Server load, l
DENS methodology Computing server selection
Pascal Bouvry ([email protected]) 9December 20, 2010
Penalize Under-loaded
Servers
Favor High Server
Utilization
Avoid Overloading
GreenCom 2010
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Q(q
)
Queue size, q/Qmax
DENS methodology Computing server selection
Pascal Bouvry ([email protected]) 10December 20, 2010
Favor Low Congestion
Levels
Penalize Congested
Queues
GreenCom 2010
00.2
0.40.6
0.81
0
0.5
10
0.1
0.2
0.3
0.4
0.5
Server load, lQueue load, q
Fs(l
,q)
DENS methodology Computing server selection
Pascal Bouvry ([email protected]) 11December 20, 2010
Maximize Server Load,Minimizing Network
Congestion
GreenCom 2010
Performance Evaluation GreenCloud simulator is developed Three-tier data center topology
1536 nodes, 32 racks, 4 core and 8 aggregation switches
Pascal Bouvry ([email protected]) 12December 20, 2010
CoreNetwork
AggregationNetwork
AccessNetwork
Computing Server
1 RU Rack Switch
L3 Switch
TaskSchedulerTask
Scheduler
Links
10 GE 1 GE
WorkloadGenerator
CloudUserCloud
User
Data Center
Data CenterCharacteristics
TaskScheduler
TaskComAgent
L3 Energymodel
L2 Energymodel
ServerCharacteristics
TaskComSink
Connect ()
SchedulerServerEnergy model
S S S S S S S S S
WorkloadTrace File
GreenCom 2010
Performance Evaluation Server workload distribution
Pascal Bouvry ([email protected]) 13December 20, 2010
0 200 400 600 800 1000 1200 1400 16000
0.2
0.4
0.6
0.8
1
Server #
Ser
ver
load
Green scheduler
DENS scheduler
Round-robin scheduler
Redistributing Computing
Load
GreenCom 2010
0 5 10 15 20 25 30 350
0.5
1
1.5
Top-of-Rack switch #
Com
bine
d up
link
load
leve
l
Performance Evaluation Network workload distribution
Pascal Bouvry ([email protected]) 14December 20, 2010
Green scheduler
DENS scheduler
Round-robin scheduler
Redistributing Load to
Satisfy QoS
Network congestion
GreenCom 2010
Performance Evaluation Top-of-Rack (ToR) switch load
Pascal Bouvry ([email protected]) 15December 20, 2010
0 5 10 150
0.2
0.4
0.6
0.8
1
Simulation time (s)
ToR
sw
itch
uplin
k lo
ad le
vel
0 5 10 150
200
400
600
800
1000
Simulation time (s)
Upl
ink
queu
e si
ze (
pack
ets)
Network congestion
Green scheduler
DENS scheduler
Green scheduler
DENS scheduler
GreenCom 2010
Data center energy consumption
Pascal Bouvry ([email protected]) 16December 20, 2010
Parameter
Power Consumption (kW·h)
Round Robinscheduler
GreenScheduler
DENSscheduler
Data centerServersNetwork switches
417.5K353.7K63.8K
203.3K (48%)161.8K (45%)41.5K (65%)
212.1K (50%)168.2K (47%)43.9K (68%)
EnergyQoS
Performance Evaluation
GreenCom 2010
Conclusions We acknowledge
Funding form Luxembourg FNR in the framework of GreenIT project
Research fellowship provided by the European Research Consortium for Informatics and Mathematics (ERCIM)
Related publications "GreenCloud: A Packet-level Simulator of Energy-aware Cloud Computing
Data Centers," in Journal of Supercomputing, special issue on Green Networks, 2011.
“GreenCloud: A Packet-level Simulator of Energy-aware Cloud Computing Data Centers,” IEEE Global Communications Conference (GLOBECOM), Miami, FL, USA, December 2010.
Pascal Bouvry ([email protected]) 17December 20, 2010
GreenCloud
