Embed Size (px)
Citation preview
Always race to sleep?
i.e. how we managed to confuse ourselves
by talking about two kinds of race to sleep
Always race to sleep?
i.e. how we managed to confuse ourselves
by talking about two kinds of race to sleep
Single node – amortize static power cost
Many nodes – minimize parallelization overhead
1. Background: Common system power and work rate behaviors
2. Analysis: Required conditions for race to sleep
3. Empirical data: When are those conditions met in Hadoop?
Two common behaviors for system power
Resource proportional
Not resource proportional
0
100
200
0 0.5 1
node CPU utilization (%)
total power single node (W)
0
10000
20000
0 50 100
cluster size (#nodes)
cluster power many nodes (W)
Three common behaviors for system work rate
Linear speed up0
100
200
0 0.5 1node CPU utilization (%)
crypto puzzles solved per second
0
50000
100000
0 50 100cluster size (#nodes)
sorted records per second
Parallelization overhead
Bottleneck elsewhere
0
100
200
0 500 1000cluster size (#nodes)
queries served per second
1. Background: Common system power and work rate behaviors
2. Analysis: Required conditions for race to sleep
3. Empirical data: When are those conditions met in Hadoop?
Work rate
Work rate Power
Work rate PowerPower efficiency= work rate / power
Work rate PowerPower efficiency= work rate / power
Work rate PowerPower efficiency= work rate / power
Yes
Time benefit, no energy benefit
Yes
Increasing efficiency
Race to sleep?i.e. operate at highest work rate?
No
Decreasing efficiency
Somewhat
Turning point exists
Energy benefit , no time benefit
Energy benefit , no time benefit
Yes and no
Yes and no
Work rate PowerPower efficiency= work rate / power
Yes
Yes
Increasing efficiency
Race to sleep?i.e. operate at highest work rate?
No
Decreasing efficiency
Somewhat
Turning point exists
Yes and no
Yes and no
Time benefit, no energy benefit
Energy benefit , no time benefit
Energy benefit , no time benefit
Work rate PowerPower efficiency= work rate / power
Race to sleep?i.e. operate at highest work rate?
Go faster if
% increase in work rate≥
% increase in power
Go slower otherwise
Work rate PowerPower efficiency= work rate / power
Race to sleep?i.e. operate at highest work rate?
Required
condition for
race to sleepGo faster if
% increase in work rate≥
% increase in power
Go slower otherwise
Work rate PowerPower efficiency= work rate / power
Race to sleep?i.e. operate at highest work rate?
e.g.
Old work rate = ANew work rate = 1.1A
Old power = BNew power = 1.05A
Old power efficiency = A / B
New power efficiency = (1.1 / 1.05) × (A / B)= (1.1 / 1.05) × old power eff.
Required
condition for
race to sleepGo faster if
% increase in work rate≥
% increase in power
Go slower otherwise
1. Background: Common system power and work rate behaviors
2. Analysis: Required conditions for race to sleep
3. Empirical data: When are those conditions met in Hadoop?
Hadoop sort
10GB terasort format
HDFS read
10GB
HDFS write
10GB
Hadoop shuffle
10GB
0
10000
20000
0 2 4 6 8 10 12#nodes
records / s
0
5
10
15
0 2 4 6 8 10 12#nodes
MB / s
0
5
10
15
0 2 4 6 8 10 12#nodes
MB / s
0
5
10
15
0 2 4 6 8 10 12#nodes
MB / s
Work rate
Hadoop sort
10GB terasort format
HDFS read
10GB
HDFS write
10GB
Hadoop shuffle
10GB
0
10000
20000
0 2 4 6 8 10 12#nodes
records / s
0
5
10
15
0 2 4 6 8 10 12#nodes
MB / s
0
5
10
15
0 2 4 6 8 10 12#nodes
MB / s
0
5
10
15
0 2 4 6 8 10 12#nodes
MB / s
Work rate
Hadoop sort
10GB terasort format
HDFS read
10GB
HDFS write
10GB
Hadoop shuffle
10GB
Power efficiency
0
50
100
150
0 2 4 6 8 10 12#nodes
records / J
0
50
100
0 2 4 6 8 10 12#nodes
KB / J
0
50
100
0 2 4 6 8 10 12#nodes
KB / J
0
50
100
0 2 4 6 8 10 12#nodes
KB / J
0
10000
20000
0 2 4 6 8 10 12#nodes
records / s
0
5
10
15
0 2 4 6 8 10 12#nodes
MB / s
0
5
10
15
0 2 4 6 8 10 12#nodes
MB / s
0
5
10
15
0 2 4 6 8 10 12#nodes
MB / s
Work rate
Hadoop sort
10GB terasort format
HDFS read
10GB
HDFS write
10GB
Hadoop shuffle
10GB
Power efficiency
0
50
100
150
0 2 4 6 8 10 12#nodes
records / J
0
50
100
0 2 4 6 8 10 12#nodes
KB / J
0
50
100
0 2 4 6 8 10 12#nodes
KB / J
0
50
100
0 2 4 6 8 10 12#nodes
KB / J
Race to sleep
No
Yes
No
Yes
That was multi-node power efficiency
Single-node power efficiency is a different picture
Always race to sleep?
Always race to sleep?
Maybe the question should be
Always use as much resources as possible?
Always race to sleep?
Maybe the question should be
Always use as much resources as possible?
Take away:
Single node – amortize static power cost (awake nodes should race to sleep)
Many nodes – minimize parallelization overhead (as few nodes awake as possible)
Increase resource if resulting % work rate increase ≥ % power increase
Work rate PowerPower efficiency= work rate / power
Yes
Yes
Increasing efficiency
Race to sleep?i.e. operate at highest work rate?
No
Decreasing efficiency
Somewhat
Turning point exists
Yes and no
Yes and no
Time benefit, no energy benefit
Energy benefit , no time benefit
Energy benefit , no time benefit
Other junk …
Power efficiency = energy efficiency
Power efficiency = joule
unitswork
Watt
sunitswork
sjoules
sunitswork
ratenconsumptioenergy
ratework
/
/
/
Energy efficiency = joule
unitswork
joules
unitswork
nedconsumptioenergy
donework
E.g. Sort job. 100 million records sorted in 100 seconds, consuming 200W on average.
Power efficiency = joule
records
sjoules
srecordsmillion
averageonW
sinsortedrecordsmillion 5000
/200
/1
200
100100
Energy efficiency = joule
records
joules
sortedrecordsmillion
sW
sortedrecordsmillion 5000
20000
100
100200
100
Power efficiency = energy efficiency
Power efficiency = joule
unitswork
Watt
sunitswork
sjoules
sunitswork
ratenconsumptioenergy
ratework
/
/
/
Energy efficiency = joule
unitswork
joules
unitswork
nedconsumptioenergy
donework
E.g. Sort job. 100 million records sorted in 100 seconds, consuming 200W on average.
Power efficiency = joule
records
sjoules
srecordsmillion
averageonW
sinsortedrecordsmillion 5000
/200
/1
200
100100
Energy efficiency = joule
records
joules
sortedrecordsmillion
sW
sortedrecordsmillion 5000
20000
100
100200
100
=
Power efficiency = energy efficiency
Power efficiency = joule
unitswork
Watt
sunitswork
sjoules
sunitswork
ratenconsumptioenergy
ratework
/
/
/
Energy efficiency = joule
unitswork
joules
unitswork
nedconsumptioenergy
donework
E.g. Sort job. 100 million records sorted in 100 seconds, consuming 200W on average.
Power efficiency = joule
records
sjoules
srecordsmillion
averageonW
sinsortedrecordsmillion 5000
/200
/1
200
100100
Energy efficiency = joule
records
joules
sortedrecordsmillion
sW
sortedrecordsmillion 5000
20000
100
100200
100
Power efficiency = energy efficiency
Power efficiency = joule
unitswork
Watt
sunitswork
sjoules
sunitswork
ratenconsumptioenergy
ratework
/
/
/
Energy efficiency = joule
unitswork
joules
unitswork
nedconsumptioenergy
donework
E.g. Web service. 100 million queries per second, consuming 200W.
Power efficiency = joule
queries
sjoules
squeriesmillion
W
sinservedqueriesmillion 500000
/200
/100
200
1100
Energy efficiency = joule
records
joules
servedqueriesmillion
sW
servedqueriesmillion 500000
200
100
1200
100
