Empya:Saving Energy in the Face of Varying Workloads

Christopher Eibel, Thao-Nguyen Do, Robert Meißner,and Tobias Distler

System Software GroupFriedrich-Alexander-Universität Erlangen-Nürnberg (FAU)

The 2018 IEEE International Conference on Cloud Engineering (IC2E ’18)

April 20, 2018

Introduction & Motivation

Execution platforms vs. energy demand in data centersProgramming and execution platforms are generally not energy awareDynamic applications in data centers are faced with varying workloadsResources are often statically assigned

Ô Consequence: Lots of energy is being wasted

Example application: key–value storeReceives and processes user requestswith basic operations (e.g., get(key))

Programmers may choose between twoconfiguration options:

Ô Staticenergy: Lower performance when load is highÔ Staticperf : Wasting energy when load is low

IC2E ’18 Empya: Saving Energy in the Face of Varying Workloads Introduction & Motivation 2

Introduction & Motivation

Execution platforms vs. energy demand in data centersProgramming and execution platforms are generally not energy awareDynamic applications in data centers are faced with varying workloadsResources are often statically assigned

Ô Consequence: Lots of energy is being wasted

Example application: key–value storeReceives and processes user requestswith basic operations (e.g., get(key))

Programmers may choose between twoconfiguration options:

Ô Staticenergy: Lower performance when load is highÔ Staticperf : Wasting energy when load is low

IC2E ’18 Empya: Saving Energy in the Face of Varying Workloads Introduction & Motivation 2

Introduction & Motivation

Execution platforms vs. energy demand in data centersProgramming and execution platforms are generally not energy awareDynamic applications in data centers are faced with varying workloadsResources are often statically assigned

Ô Consequence: Lots of energy is being wasted

Example application: key–value storeReceives and processes user requestswith basic operations (e.g., get(key))

Programmers may choose between twoconfiguration options:

Ô Staticenergy: Lower performance when load is highÔ Staticperf : Wasting energy when load is low

IC2E ’18 Empya: Saving Energy in the Face of Varying Workloads Introduction & Motivation 2

0

5

10

15

20

Staticperf Staticenergy

Power demand at 70 kOps/s

Pow

erdem

and[W

]

0

100

200

300

400

Staticperf Staticenergy

Maximum throughput

Throughput[kOps/s]

Introduction & Motivation

Execution platforms vs. energy demand in data centersProgramming and execution platforms are generally not energy awareDynamic applications in data centers are faced with varying workloadsResources are often statically assigned

Ô Consequence: Lots of energy is being wasted

Example application: key–value storeReceives and processes user requestswith basic operations (e.g., get(key))

Programmers may choose between twoconfiguration options:

Ô Staticenergy: Lower performance when load is highÔ Staticperf : Wasting energy when load is low

IC2E ’18 Empya: Saving Energy in the Face of Varying Workloads Introduction & Motivation 2

0

5

10

15

20

Staticperf Staticenergy

Power demand at 70 kOps/s

Pow

erdem

and[W

]

0

100

200

300

400

Staticperf Staticenergy

Maximum throughput

Throughput[kOps/s]

Goals

Control the energy–performance tradeoffPropose platform that¶ frees programmers from taking care of energy optimizations· uses available techniques at hardware and software level¸ adapts dynamically to varying workloads

General Approach

EMPYA: energy-aware middleware platform for dynamic applications

Key design principles for EMPYA¶ Energy-efficiency awareness

Avoid high CPU utilization because of disproportionatepower-to-performance ratioNot necessarily select configuration with full resource allocation

· Multi-level awarenessExploit available techniques at multiple levelsCoordinate techniques:Best energy efficiency with respect to required performance

¸ Energy awarenessIntegrated regulator making energy-aware reconfigurationsNo additional services

IC2E ’18 Empya: Saving Energy in the Face of Varying Workloads Approach 3

EMPYA – Exploiting Techniques at Different Levels

IC2E ’18 Empya: Saving Energy in the Face of Varying Workloads Approach 4

Ô Vary #threadsÔ Mapping of application

components to threadsÔ Vary #(un)active coresÔ Mapping of application

threads to active coresÔ Vary upper power limitsÔ Instruct hardware to

enforce them

Platform

Level

OS

Level

Hardware

Level

Energy

Regulator

EMPYA – Exploiting Techniques at Different Levels

IC2E ’18 Empya: Saving Energy in the Face of Varying Workloads Approach 4

Ô Vary #threadsÔ Mapping of application

components to threads

Ô Vary #(un)active coresÔ Mapping of application

threads to active coresÔ Vary upper power limitsÔ Instruct hardware to

enforce them

Platform

Level

OS

Level

Hardware

Level

Energy

Regulator

EMPYA – Exploiting Techniques at Different Levels

IC2E ’18 Empya: Saving Energy in the Face of Varying Workloads Approach 4

Ô Vary #threadsÔ Mapping of application

components to threadsÔ Vary #(un)active coresÔ Mapping of application

threads to active cores

Ô Vary upper power limitsÔ Instruct hardware to

enforce them

Platform

Level

OS

Level

Hardware

Level

Energy

Regulator

EMPYA – Exploiting Techniques at Different Levels

IC2E ’18 Empya: Saving Energy in the Face of Varying Workloads Approach 4

Ô Vary #threadsÔ Mapping of application

components to threadsÔ Vary #(un)active coresÔ Mapping of application

threads to active coresÔ Vary upper power limitsÔ Instruct hardware to

enforce them

Platform

Level

OS

Level

Hardware

Level

Energy

Regulator

EMPYA – Exploiting Techniques at Different Levels

IC2E ’18 Empya: Saving Energy in the Face of Varying Workloads Approach 4

Ô Vary #threadsÔ Mapping of application

components to threadsÔ Vary #(un)active coresÔ Mapping of application

threads to active coresÔ Vary upper power limitsÔ Instruct hardware to

enforce them

Platform

Level

OS

Level

Hardware

Level

Energy

Regulator

EMPYA – Exploiting Techniques at Different Levels

IC2E ’18 Empya: Saving Energy in the Face of Varying Workloads Approach 5

¶ ActorsEach actor maintainsits own stateCommunication viamessage passingActor is independentof executing thread

Ô Implementation:Akka toolkit

· Power limitingRunning average power limit (RAPL)Originally developed for power limiting (e.g., temperature issues)Enables power and energy measurementsPower capping very powerful for reducing the energy demand

Platform

Level

OS

Level

Hardware

Level

Energy

Regulator

A0 A1 A2

ThreadPool0

ThreadPool1

C0 C1 C2 C3

Active Active Inactive Active

PowerControl

EnergyAccounting

EMPYA – Exploiting Techniques at Different Levels

IC2E ’18 Empya: Saving Energy in the Face of Varying Workloads Approach 5

¶ ActorsEach actor maintainsits own stateCommunication viamessage passingActor is independentof executing thread

Ô Implementation:Akka toolkit

· Power limitingRunning average power limit (RAPL)Originally developed for power limiting (e.g., temperature issues)Enables power and energy measurementsPower capping very powerful for reducing the energy demand

Platform

Level

OS

Level

Hardware

Level

Energy

Regulator

A0 A1 A2

ThreadPool0

ThreadPool1

C0 C1 C2 C3

Active Active Inactive Active

PowerControl

EnergyAccounting

EMPYA – Exploiting Techniques at Different Levels

IC2E ’18 Empya: Saving Energy in the Face of Varying Workloads Approach 5

¶ ActorsEach actor maintainsits own stateCommunication viamessage passingActor is independentof executing thread

Ô Implementation:Akka toolkit

· Power limitingRunning average power limit (RAPL)Originally developed for power limiting (e.g., temperature issues)Enables power and energy measurementsPower capping very powerful for reducing the energy demand

Platform

Level

OS

Level

Hardware

Level

Energy

Regulator

A0 A1 A2

ThreadPool0

ThreadPool1

C0 C1 C2 C3

Active Active Inactive Active

PowerControl

EnergyAccounting

EMPYA – Energy Regulator

Self-adapting system with continuous feedback loopMonitor application performanceEmit dynamic HW/SW reconfigurations

Energy-profile databaseConfiguration characteristicsWorkload-specific power values

Energy policiesPrimary performance goal(e.g., throughput)Secondary performance goal(e.g., latency)

IC2E ’18 Empya: Saving Energy in the Face of Varying Workloads Approach 6

PlatformLevel

OSLevel

HardwareLevel

Energy Regulator

Observer

Configurator

ControlUnit

Profiles

Energy Policy

getValues()

reconfigure()

EMPYA – Energy Regulator

Self-adapting system with continuous feedback loopMonitor application performanceEmit dynamic HW/SW reconfigurations

Energy-profile databaseConfiguration characteristicsWorkload-specific power values

Energy policiesPrimary performance goal(e.g., throughput)Secondary performance goal(e.g., latency)

IC2E ’18 Empya: Saving Energy in the Face of Varying Workloads Approach 6

PlatformLevel

OSLevel

HardwareLevel

Energy Regulator

Observer

Configurator

ControlUnit

Profiles

Energy Policy

getValues()

reconfigure()

ID Configuration Performance Power#Threads #Cores Cap Throughput Latency usage

α 24 8 None390.5 kOps/s 0.42 ms 51.2 W

70.4 kOps/s 0.37 ms 19.3 W

λ 12 6 22 W224.8 kOps/s 0.62 ms 22.0 W

50.5 kOps/s 0.25 ms 15.3 W

ω 1 1 10 W 20.6 kOps/s 0.22 ms 10.0 W15.1 kOps/s 0.21 ms 9.7 W

EMPYA – Energy Regulator

Self-adapting system with continuous feedback loopMonitor application performanceEmit dynamic HW/SW reconfigurations

Energy-profile databaseConfiguration characteristicsWorkload-specific power values

Energy policiesPrimary performance goal(e.g., throughput)Secondary performance goal(e.g., latency)

IC2E ’18 Empya: Saving Energy in the Face of Varying Workloads Approach 6

PlatformLevel

OSLevel

HardwareLevel

Energy Regulator

Observer

Configurator

ControlUnit

Profiles

Energy Policy

getValues()

reconfigure()

energy policy {application = key-value-store;throughput_min_ops_per_sec = 10k;throughput_priority = pri;latency_max_msec = 0.5;latency_priority = sec;

}

ID Configuration Performance Power#Threads #Cores Cap Throughput Latency usage

α 24 8 None390.5 kOps/s 0.42 ms 51.2 W

70.4 kOps/s 0.37 ms 19.3 W

λ 12 6 22 W224.8 kOps/s 0.62 ms 22.0 W

50.5 kOps/s 0.25 ms 15.3 W

ω 1 1 10 W 20.6 kOps/s 0.22 ms 10.0 W15.1 kOps/s 0.21 ms 9.7 W

Evaluation – Evaluation Setup

HardwareClient and server machines, switched 1Gbps EthernetIntel Xeon E3-1245 v3 & Xeon E3-1275 v5 processors8 cores with Hyper-Threading enabled, 3.40GHzSpeed Step and TurboBoost enabled

Application classesUse case A: Key–value store with mixed operations (get, set, exists)Use case B: MapReduce running single, different jobs

IC2E ’18 Empya: Saving Energy in the Face of Varying Workloads Evaluation 7

XeonClient machine

Key1 Values1

Key2 Values2

Keyn Valuesn

......

Data Map Shuffle Reduce Output

Evaluation – Use Case A: Key–Value StoreStaticperf vs. EMPYAThroughput as primary performance goal

0 60 120 180 240 300 360 4200

100

200

300

400

Time [s]

Throughput[kOps/s] Staticperf

EMPYA

0 60 120 180 240 300 360 4200

102030405060

Time [s]

Pow

er[W

]

StaticperfEMPYA

IC2E ’18 Empya: Saving Energy in the Face of Varying Workloads Evaluation 8

Evaluation – Use Case A: Key–Value StoreStaticperf vs. EMPYAlatencyThroughput as primary and latency as secondary performance goal

0 60 120 180 240 300 360 4200

0.5

1.0

1.5

2.0

Time [s]

Latency

[ms]

StaticperfEMPYAdefaultEMPYAlatency

0 60 120 180 240 300 360 4200

102030405060

Time [s]

Pow

er[W

]

StaticperfEMPYAdefaultEMPYAlatency

IC2E ’18 Empya: Saving Energy in the Face of Varying Workloads Evaluation 8

Evaluation – Use Case B: MapReduceStaticenergy/Staticperf vs. EMPYAPerformance goal: Specifying maximum execution-time penalties

0

1

Energy

Staticperf Staticenergy Empya5% Empya15% Empya30%

0

1

2

Runtime

1507.7J

517.5J

752.9J

594.2J

529.6J

2982.5J

1456.6J

2124.9J

1827.7J

1815.1J

2402.4J

887.1J

1537.8J

1346.1J

889.7J

2527.7J

843.9J

1271.6J

1244.3J

947.1J

1967.3J

724.8J

1328.0J

923.4J

817.9J

643.0J

242.4J

423.3J

397.9J

367.5J

1415.8J

432.8J

511.4J

481.3J

443.2J

64.9s

84.3s

66.2s

72.9s

83.5s

102.1s

196.0s

109.8s

117.5s

123.7s

62.2s

119.0s

63.3s

64.3s

77.2s

67.6s

113.6s

70.6s

74.8s

92.8s

60.1s

98.1s

61.5s

68.4s

75.1s

15.3s

32.5s

15.9s

16.0s

20.4s

41.9s

59.1s

44.2s

45.2s

51.7s

sort kmeans wc topn mean join overlap

IC2E ’18 Empya: Saving Energy in the Face of Varying Workloads Evaluation 9

Conclusion

EMPYASelf-adaptive middleware platform enforcing HW and SW reconfigurationsExploiting actors and operating-system functionalityPower capping as an effective power- and energy-reduction measure

Key–value store: Up to 34% less power demandMapReduce: Energy savings of 22–64%

Future and ongoing workMaking decisions in a distributed manner for multiple machinesCarefully increasing the configuration space → heterogeneity

C. Eibel, C. Gulden, W. Schröder-Preikschat, and T. DistlerStrome: Energy-Aware Data-Stream ProcessingIn Proceedings of the 18th IFIP International Conference on DistributedApplications and Interoperable Systems (DAIS 2018), 2018.

IC2E ’18 Empya: Saving Energy in the Face of Varying Workloads Conclusion 10

Conclusion

EMPYASelf-adaptive middleware platform enforcing HW and SW reconfigurationsExploiting actors and operating-system functionalityPower capping as an effective power- and energy-reduction measure

Key–value store: Up to 34% less power demandMapReduce: Energy savings of 22–64%

Future and ongoing workMaking decisions in a distributed manner for multiple machinesCarefully increasing the configuration space → heterogeneity

C. Eibel, C. Gulden, W. Schröder-Preikschat, and T. DistlerStrome: Energy-Aware Data-Stream ProcessingIn Proceedings of the 18th IFIP International Conference on DistributedApplications and Interoperable Systems (DAIS 2018), 2018.

IC2E ’18 Empya: Saving Energy in the Face of Varying Workloads Conclusion 10

Thank you for your attention.

Questions?