IEE/CEEM 2012-2013 Seminar: Thomas Wenisch, "Power Management from Smartphones to Data Centers."

7/29/2019 IEE/CEEM 2012-2013 Seminar: Thomas Wenisch, "Power Management from Smartphones to Data Centers."

1/62

2013 Thomas Wenisch

Power Management

from Smartphones to Data CentersThomas WenischMorris Wellman Faculty Dev. Asst. Prof. of CSE

University of Michigan

Acknowledgements:Luiz Barroso, Anuj Chandawalla,

Laurel Emurian, Brian Gold, Yixin Luo,

Milo Martin, David Meisner,

Marios Papaefthymiou, Steven Pelley,

Kevin Pipe, Arun Raghavan,

Chris Sadler, Lei Shao, Wolf Weber


2/62

2013 Thomas Wenisch

A Paradigm Shift In Computing

2

0.001

0.01

0.1

1

10

100

1000

10000

100000

1000000

1985 1990 1995 2000 2005 2010 2015 2020

Transistors (100,000's)

Power (W)

Performance (GOPS)

Efficiency (GOPS/W)

Limits on heat extraction

Limits on energy-efficiency of operations

IEEE ComputerApril 2001

T. Mudge


3/62

2013 Thomas Wenisch

A Paradigm Shift In Computing

3

0.001

0.01

0.1

1

10

100

1000

10000

100000

1000000

1985 1990 1995 2000 2005 2010 2015 2020

Transistors (100,000's)

Power (W)

Performance (GOPS)

Efficiency (GOPS/W)

Era of High Performance Computing Era of Energy-Efficient Computingc. 2000

Limits on heat extraction

Limits on energy-efficiency of operations

Stagnates performance growth

IEEE ComputerApril 2001

T. Mudge


4/62

2013 Thomas Wenisch

Four decades of Dennard Scaling

P = C V2 f

Increase in device count

Lower supply voltages

Constant power/chip

Dennard et. al., 1974 Robert H. Dennard, picture from

Wikipedia

4


5/62

2013 Thomas Wenisch

Leakage Killed Dennard Scaling

Leakage:

Exponential in inverse of Vth

Exponential in temperature Linear in device count

To switch well

must keep Vdd/Vth > 3

Vddcant go down

5


6/62

2013 Thomas Wenisch

No more free lunch

Need system-level approaches to turn increasing transistor counts into customer value

without exceeding thermal limits

Energy efficiency is the new performance

Todays talk

Computational Sprinting

Improving responsiveness for mobile systemsby briefly exceeding thermal limits

Power mgmt. for Online Data Intensive Services

Case study of server power management

for Googles Web Search 6


7/62 2013 Thomas Wenisch


Arun Raghavan*, Yixin Luo+, Anuj Chandawalla+,

Marios Papaefthymiou+, Kevin P. Pipe+#,

Thomas F. Wenisch+

, Milo M. K. Martin*

University of Pennsylvania, Computer and Information Science*

University of Michigan, Electrical Eng. and Computer Science+

University of Michigan, Mechanical Engineering#

7



Computational Sprinting and Dark Silicon

A Problem: Dark Silicon a.k.a. The Utilization Wall

Increasing power density; cant use all transistors all the time

Cooling constraints limit mobile systems

One approach: Use few transistors for long durations

Specialized functional units [Accelerators, GreenDroid] Targeted towards sustained compute, e.g. media playback

Our approach: Use many transistors for short durations

Computational Sprinting by activating many dark cores

Unsustainable power for short, intense bursts of compute

Responsiveness for bursty/interactive applications

8Is this feasible?



Sprinting Challenges and Opportunities

Thermal challenges

How to extend sprint duration and intensity?

Latent heat fromphase change materialclose to the die

Electrical challenges

How to supply peak currents? Ultracapacitor/battery hybrid

How to ensure power stability? Ramped activation (~100s)

Architectural challenges

How to control sprints? Thermal resource management How do applications benefit from sprinting?

6.3x responsiveness for vision workloads

on a real Core i7 testbed restricted to a 10W TDP

9



Power Density Trends for Sustained Compute

10

How to meet thermal limit despite

power density increase?

0

0

power

time

time

tem

perature

TmaxThermal limit

> 10x



Option 1: Enhance Cooling?

11

Mobile devices limited to passive cooling

0

tempera

ture

time

Tmax



Option 2: Decrease Chip Area?

12

Reducescost, but sacrifices

benefits from Moores law



Option 3: Decrease Active Fraction?

13

How do we extract application performance

from this dark silicon?



Design for Responsiveness Observation: today, design for sustained performance

But, consider emerging interactive mobile apps*Clemons DAC11, Hartl ECV11, Girod IEEE Signal Processing11+

Intense compute bursts in response to user input, then idle

Humans demand sub-second response times*Doherty IBM TR 82, Yan DAC05, Shye MICRO09, Blake ISCA10+

14

Peak performance during bursts

limits what applications can do



COMPUTATIONAL SPRINTING

Designing for Responsiveness

15



Parallel Computational Sprinting

Tma

x

power

temperature

16



Tma

x

power

temperature

Effect ofthermal capacitance


17



Tma

x

power

temperature



18



Tma

x

power

temperature



19



Tma

x

power

temperature

Effect ofthermal capacitanceState of the art:

Turbo Boost 2.0

exceedssustainable power

with DVFS

(~25% for 25s)


20



Hardware Testbed

21

Quad-core Core i7 desktop

Heat sink removed

Fan tuned for 10W TDP

Power profile Idle: 4.5 W

Sustainable (1 core 1.6GHz): 9.5 W

Efficient sprint (4 core 1.6GHz): ~20 W

Max sprint (4 core 3.2GHz): ~50 W

Temperature profile Idle: ~45C

Max safe: 78C

20g copper heat spreader on package

Can absorb ~225J heat over 30C temperature rise

Models a system capable of 5x max sprint intensity



Power & Temperature Response

22

0

10

20

30

40

50

60

-5 0 5 10 15 20 25 30 35 40

sustained

sprint-3.2GHz

sprint-1.6GHz

40

50

60

70

0

20

40

60

Power(W

)

Tem

p(C)

Max sprint: 3s @ 3.2GHz, 19s @ 1.6GHz

-5 0 5 10 15 20 25 30 35 40

sustained

sprint-3.2GHz

sprint-1.6GHz



Responsiveness & Energy Impact

23

0

2

4

6

8

3.2 1.6 3.2 1.6 3.2 1.6 3.2 1.6 3.2 1.6 3.2 1.6

normalizedspeed

up

sobel disparity segment kmeans feature texture

sobel disparity segment kmeans feature texture

0

0.5

1

1.5

3.2 1.6 3.2 1.6 3.2 1.6 3.2 1.6 3.2 1.6 3.2 1.6

norm

alizedenergy

Idle

Sprint

Idle

Sprint

Sprint for responsiveness (3.2GHz): 6.3x speedup

Race-to-idle (1.6GHz): 7% energy savings (!!)


24/62

2013 Thomas Wenisch

Extending Sprint Intensity & Duration:

Role of Thermal Capacitance

Current systems designed forthermal conductivity

Limited capacitance close to die

To explicitly design for sprinting,

add thermalcapacitance near die

Exploit latent heat fromphase change material (PCM)

Die

Die

PCM

24


25/62

2013 Thomas Wenisch

PCM Heat Sink Prototype

25

Aluminum foam mesh filled with Paraffin wax

Relatively form-stable; melting point near 55C

Working on a fully-sealed prototype w/ thermocouples

1

2


26/62

2013 Thomas Wenisch

Demo of PCM melting

26Nickel-plated Copper fins; paraffin wax


27/62

2013 Thomas Wenisch

Impact of PCM prototype

27

40

50

60

70

Temp

(C)

PCM extends max sprint duration by almost 3x

80

90 air

empty

water

wax

50 100 150 200 250 300 350 400

Elapsed Time


28/62

2013 Thomas Wenisch

Power Management of

Online Data-Intensive Services

David Meisner, Christopher M. Sadler, Luiz A. Barroso,

Wolf-Dietrich Weber, Thomas F. Wenisch

The University of Michigan *Facebook Google, Inc.

International Symposium on Computer Architecture 201128


29/62

2013 Thomas Wenisch

Power: A first-class data center constraint

29

Improving energy & capital efficiency is a critical challenge

Source: US EPA 2007

Source: Mankoff et al, IEEE Computer 2008

Annual data center CO2:17 million households

2.5% of US energy

$7.4 billion/yr.Installed base

grows 11%/yr.

Facility

Electricity

Servers

Source: Barroso 10

Lifetime Cost of a Data Center

Peak power determines

data center capital costs


30/62

2013 Thomas Wenisch

Online Data-Intensive Services [ISCA 2011]

Challenging workload class for power management Process TBs of data with O(ms) request latency

Tail latencies critical (e.g., 95th, 99th-percentile latency)

Provisioned by data set size and latency notthroughput

Examples:web search, machine translation, online-ads

Case Study: Google Web Search

First study on power management for OLDI services

Goal: Identify which power modes are useful

30


31/62

2013 Thomas Wenisch

The need for energy-proportionality

31

~75%

~50%

~20%

How to achieve energy-proportionality at each QPS level?


32/62

2013 Thomas Wenisch

Two-part study of Web Search

Part 1: Cluster-scale throughput study

Web Search on O(1,000) node cluster

Measured per-component activity at leaf level

Use to derive upper-bounds on power savings

Determine power modes of interest/non-interest

Part 2: Single-node latency-constrained study

Evaluate power-latency tradeoffs of power modes

Can we achieve energy-proportionality with SLA slack

32Need coordinated, full-system active low-power modes


33/62


34/62

2013 Thomas Wenisch

Doc 36

Background: Web Search operation

34

Root

Leaf

Intermediary

LevelQuery:Ice Cream

Ice

Index Docs Index Docs Index Docs Index Docs

Cream

Doc 24

Doc

11, 200Doc

36,50

Doc

11,50, 36,200

Doc

50, 76, 200, 323

Doc

76, 323

Doc

76, 323

What if we turn off

a fraction of the cluster?Doc 50


35/62

2013 Thomas Wenisch

Web Search operation

35

Query:Vanilla Ice Cream

Index Docs Index Docs Index Docs Index Docs

Doc 76

What if we turn off

a fraction of the cluster?


36/62

2013 Thomas Wenisch

Web Search operation

36

Index Index Docs

Doc 76What if we turn off

a fraction of the cluster?

50% of Max QPS

20

A I

Cluster-level techniquescause data unavailability

Disk

CPU

Mem

A I

Sy

stem

Cluster


37/62

2013 Thomas Wenisch

Study #1: Cluster-scale throughput

Web Search experimental setup O(1,000) server system

Operated at 20%, 50%, 75% of peak QPS

Traces of CPU util., memory bandwidth, disk util.

Characterization

Goal: find periods to use low-power modes Understand intensityand time-scale of utilization

Analyze using activity graphs

37


38/62

2013 Thomas Wenisch

CPU utilization

38

50% of Max QPS

20

40

80

100

Time Scale

1ms 10ms100 s

PercentofTim

e

60

100ms 10s

10%Idle

30%

50%

1s

How often canwe use the mode?What is the intensity of activity?

We can use a mode with a 2xslowdown 45% of the time

on time scales of 1ms or less


39/62

2013 Thomas Wenisch

CPU utilization

39

50% of Max QPS

Very little Idleness > 10ms

1 ms granularity sufficient

20

40

80

100

Time Scale

1ms 10ms100 s

PercentofTim

e

60

100ms 10s

10%Idle

30%

50%

1s


40/62

2013 Thomas Wenisch

CPU utilization

40

50% of Max QPS

Very little Idleness > 10ms

1 ms granularity sufficient

20

40

80

100

Time Scale

1ms 10ms100 s

PercentofTim

e

60

100ms 10s

10%Idle

30%

50%

1s

50% of Max QPS

20

A I

CPU active/idle mode opportunityfrom a bandwidth perspective

Disk

CPU

Mem

A I

Sy

stem

Cluster


41/62

2013 Thomas Wenisch

Memory bandwidth utilization

41

50% of Max QPS

20

40

80

100

Time Scale

1s 10s100 ms

PercentofTim

e

60

100s 1000s

10%Idle

30%

50%

No SignificantIdleness

Significant Periodsof Under-utilization


42/62

2013 Thomas Wenisch

Memory bandwidth utilization

42

50% of Max QPS

20

40

80

100

Time Scale

1s 10s100 ms

PercentofTim

e

60

100s 1000s

10%Idle

30%

50%

No SignificantIdleness

Significant Periodsof Under-utilization

50% of Max QPS

20

A I

Insufficient idleness for memoryidle low-power mode

CPU

A I

System

Cluster

Disk transition times too slowSee paper for details

Disk

Mem


43/62

2013 Thomas Wenisch

Study #2: Leaf node latency

Goal: Understand latency effect of power modes

Leaf node testbed

Faithfully replicate production queries at leaf node

Arrival time distribution critical for accurate modeling

Up to 50% error from Nave loadtester

Validated power-performance model Characterize power-latency tradeoff on real HW

Evaluate power modes using Stochastic Queuing

Simulation (SQS) *EXERT 10+

43


44/62

2013 Thomas Wenisch

75% QPS

Full-system coordinated idle modes

Scarce full-system idleness in 16-core systems

PowerNap *ASPLOS 09+ with batching *Elnozahy et al 03+

44

20% QPS

25

50

75

100

95th-Percentile Latency Increase4x 6x8x 10x

2x

Power(Percentofpeak)

1x Avg. Query Time

0.1x Avg. Query Time

25

50

75

100

95th-Percentile Latency4x 6x 8x 10x2x

Power(Pe

rcentofpeak)

1x Avg. Query Time


Transition Time

Transition Time


45/62

2013 Thomas Wenisch

75% QPS

Full-system coordinated idle modes

Scarce full-system idleness for multicore

PowerNap *ASPLOS 09+ with batching *Elnozahy et al 03+

45

20% QPS

25

50

75

100

95th-Percentile Latency Increase4x 6x 8x 10x2x


1x Avg. Query Time


25

50

75

100

95th-Percentile Latency4x 6x 8x 10x2x

Power(Pe

rcentofpeak)

1x Avg. Query Time


Transition Time

Transition Time

20

A I

Batching + full-system idlemodes ineffective

CPU

A I

Cluster

Disk

Mem

System


46/62

2013 Thomas Wenisch

Full-system coordinated active scaling

We assume CPU and memory scaling with P~f2.4

Optimal Mix requires coordination of CPU/memory modes

46

20% QPS

1.6x 1.8x 2x

Memory Only

Full System

25

50

75

100

95th-Percentile Latency Increase

1.2x 1.4x1x


CPU Only25

50

75

100


1.2x 1.4x 1.6x 1.8x1x

Power(Pe

rcentofpeak)

2x

75% QPS

Memory Only

Full System

CPU Only


47/62

2013 Thomas Wenisch

Full-system coordinated active scaling

We assume CPU and memory scaling with P~f2.4

Optimal Mix requires coordination of CPU/memory modes

47

20% QPS

1.6x 1.8x 2x

Memory Only

Full System

25

50

75

100


1.2x 1.4x1x


CPU Only25

50

75

100


1.2x 1.4x 1.6x 1.8x1x

Power(Pe

rcentofpeak)

2x

75% QPS

Memory Only

Full System

CPU Only

A I

Single-component active low-power modes insufficient

I

Cluster

Disk

A

System

CPU

Mem


48/62

2013 Thomas Wenisch

Comparing power modes

Allow SLA slack deviation from 95th-percentile latency

48

2040

60

80100

1x SLA

2x SLA5x SLA

Averag

eDiurnalPowe

r

(Percentofpeak)


49/62

2013 Thomas Wenisch

Comparing power modes

Allow SLA slack deviation from 95th-percentile latency

49

2040

60

80100

1x SLA

2x SLA5x SLA

Averag

eDiurnalPowe

r

(Percentofpeak)

A I

Core-level power modesprovide negligible power savings

I

Clu

ster

Disk

A

System

Mem

CPU

Only coordinated active modes achieve proportionality


50/62

2013 Thomas Wenisch

OLDI power management summary

OLDI workloads challenging for power management

Cluster-scale study

Current CPU power modes sufficient

Massive opportunity for active modes for memory Need faster idle and active modes for disk

Latency-constrained study

Individual idle/active power modes do not achieve proportionality

PowerNap + batching provides poor latency-power tradeoffs

50

Need coordinated, full-system active low-power modes

to achieve energy proportionality for OLDI workloads


51/62

2013 Thomas Wenisch

For more informationhttp://www.eecs.umich.edu/~twenisch

51

Sponsors


52/62

2013 Thomas Wenisch

Backup Slides

52


53/62

2013 Thomas Wenisch

Typical data center utilization

Low utilization (20%) is endemic Provisioning for peak load

Performance isolation

Redundancy

53

Source: Barroso & Hlzle, Google 2007

0%

20%

40%

60%

80%

100%

10

%

20

%

30

%

40

%

50

%

60

%

70

%

80

%

90

%

100

%Fractionoftime

CPU utilization (%)

ITWeb 2.0

Customer traces supplied by HP Labs

But, historically, vendors optimize & report peak power


54/62

2013 Thomas Wenisch Most idle periods are < 1 Sec in length

Idle periods are short

54

20%

40%

60%

80%

100%

0%

100ms 1s 10s 100s10ms

Idle Period Length (L)

%

IdleTimeinPeriodsL

[Meisner 09]

DNS

Shell

Mail

Web

HPCBackup


55/62

2013 Thomas Wenisch

Background: PowerNap *ASPLOS09, TOCS11+Full System Idle Low-Power Mode

Full-system nap during


56/62

2013 Thomas Wenisch

0%

20%

40%

60%

80%

100%

0% 20% 40% 60% 80% 100%

Avg.Power

(%

maxpower)

% utilization

DVFS = 100%

DVFS = 40%

DVFS = 20%

PowerNap = 100 msPowerNap = 10 ms

PowerNap = 1 ms0%

20%

40%

60%

80%

100%

0% 20% 40% 60% 80% 100%

Avg.Power

(%

maxpower)

% utilization

DVFS = 100%

DVFS = 40%

DVFS = 20%

PowerNap = 100 msPowerNap = 10 ms

PowerNap = 1 ms

Average power

56

Pwrcpu

Pwrcpu

Pwrcpu

TtTt

Tt

DVFS saves rapidly, but limited by Pwrcpu

PowerNap becomes energy-proportional as Tt0


57/62

2013 Thomas Wenisch

Response time

57

1.0

1.5

2.0

2.5

3.0

3.5

4.0

0% 20% 40% 60% 80% 100%

Relativeresponsetime

% utilization

DVFS

PowerNap = 100 ms

PowerNap = 10 ms

PowerNap = 1 ms

Tt

Tt

Tt

DVFS response time penalty capped by fmin

PowerNap penalty negligible for Tt 1ms


58/62

2013 Thomas Wenisch

PowerNap Hardware

58

Transition Time (us)

NapPower(W

)

Nap power is ~10W, but PSU uses additional 25W

PSU also limiting factor for transition

1 10 100 1000

CPU

DRAM

PSU

0

10

20

30

40

CPU

DRAM

NIC

SSD

PSU


59/62

2013 Thomas Wenisch

PowerNap & multicore scaling

Request-level parallelism &

core scaling thwart PowerNap

Full-system idleness vanishes

even at low utilization

Per-core idleness unaligned

Parking (Package C6) saves little Automatic per-core C1E on HLT already very good

59

0%

20%

40%

60%

80%

100%

1 2 4 8 16 32PowerSavingsvs.C1E

30%

utilization

Cores per Socket

Core Parking

Socket Parking

PowerNap

Need to create PowerNap opportunity via scheduling


60/62

2013 Thomas Wenisch

Background: MemScale *ASPLOS11+Active Low-Power Mode for Main Memory

Goal: Dynamically scale memory frequency to conserve energy

Hardware mechanism:

Frequency scaling (DFS) of the channels, DIMMs, DRAM devices

Voltage & frequency scaling (DVFS) ofthe memory controller

Key challenge:

Conserving significant energy while meeting performanceconstraints

Approach:

Online profiling to estimate performance and bandwidth demand

Epoch-based modeling and control to meet performance constraints

System energy savings of 18%

with average performance loss of 4% 60


61/62

2013 Thomas Wenisch 6161

MemScale frequency and slack management

Time

Epoch 1 Epoch 2 Epoch 3 Epoch 4

High Freq.

Low Freq.

MC, Bus + DRAM

CPU Pos. Slack Neg. Slack Pos. Slack ProfilingTarget

Actual

Calculate slack vs. targetEstimate performance/energy via models


62/62

In Brief: Computational Sprinting [HPCA 2012]

Many interactive mobile apps are bursty

Power density trend leading to dark silicon (esp. mobile)

Today, we design for sustained performance

Our goal: design for responsiveness


Intensely, but briefly exceed Thermal Design Power (TDP)

Buffer heat via thermal capacitance using phase change material

temp MeltingPoint

Re-solidification

Tmax

Die

PCM

Documents

IEE/CEEM 2012-2013 Seminar: Thomas Wenisch, "Power Management from Smartphones to Data Centers."