Cutting the Electric Bill for Internet-Scale Systems

Kevin Leeds

The Concept

Energy costs differ by location Reroute customer requests to

cheaper locations

$50/MWh $30/MWh

Customer

CloserCheaper energy

Why This Is Possible

Electricity prices vary per region Price variation between coal and hydro

Also by hour (up to factor of 10) Solar power, wind power, tide

Trends are detectable Power grids may be over/under-

utilized

Potential Savings

Google spends over $38M on electricity per year.

3% energy cost reduction saves ~ $1M

Energy Elasticity

Energy Consumed / Load on cluster

Idealistic: 0 load = 0% peak power Realistic: 0 load ~ 60% peak

power (current state-of-the-art) Without energy elasticity, no

savings can be gained

Constraints Affect Outcomes

Elasticity Existing systems: ~2% savings Fully-elastic system: ~13% savings

Bandwidth constraints on fully-elastic No constraints: ~30% savings 95/5 constraints: ~13% savings

Client-server distances Constrained: ~45% max savings Non-constrained: ~35% max savings

Wholesale Electricity Markets

Regional Transmission Organization (RTO) Runs several parallel wholesale markets

Day-ahead markets (expectations for the day) Real-time markets (real-time price calculations)

Changes in price Demand rises, more expensive sources of

energy are called upon (congestion exists)

Price changes per hour

Price Differences Between Regions

Cluster Power Calculations

Variables F = Fixed power V = Variable power E/r = Derived Constants PUE = Power Utilization Efficiency n = # of servers in cluster u = average CPU utilization

What about routing energy increases?

Shouldn’t be significant 1 kJ/Google Query 2 mJ/Packet passing through router

Savings Experiment Findings

Cost Savings Experiment (Change in Per-Cluster Cost)

Reaction Delays

Faster reaction to changes = lower cost

Future Work

Weather Differentials Cool centers with outside air (AZ vs

MN) Implementing Joint Optimization

Systems already reroute traffic based on bandwidth costs, performance, and reliability – add local energy costs

Questions/Comments

Thank you!