42
© Copyright IBM Corporation 2012 IBM Systems and Technology Group Nick Gruendler Blades and Modular Systems Systems and Technology Group

IBM Systems and Technology Group Nick Gruendler Blades and ... · IBM Systems and Technology Group IBM Power Technology Symposium 2011 ... Looking at the graphs on the previous slides,

  • Upload
    others

  • View
    0

  • Download
    0

Embed Size (px)

Citation preview

Page 1: IBM Systems and Technology Group Nick Gruendler Blades and ... · IBM Systems and Technology Group IBM Power Technology Symposium 2011 ... Looking at the graphs on the previous slides,

© Copyright IBM Corporation 2012

IBM Systems and Technology Group Nick GruendlerBlades and Modular Systems

Systems and Technology Group

Page 2: IBM Systems and Technology Group Nick Gruendler Blades and ... · IBM Systems and Technology Group IBM Power Technology Symposium 2011 ... Looking at the graphs on the previous slides,

2 © Copyright IBM Corporation 2012

IBM Systems and Technology Group

IBM Power Technology Symposium 2011

ABSTRACT

Building on last year’s symposium presentation, this presentation will show:- The challenges faced by system x,

- The progress IBM has made in product energy efficiency, and

- What is still needed to further energy saving goals, including: • 1) Component technology and design points (both dc/dc and ac/dc); • 2) Efficiency standards; • 3) Regulatory environment and their impact to product design

(Energy Star, China and Korean regulations, etc).

Page 3: IBM Systems and Technology Group Nick Gruendler Blades and ... · IBM Systems and Technology Group IBM Power Technology Symposium 2011 ... Looking at the graphs on the previous slides,

3 © Copyright IBM Corporation 2012

IBM Systems and Technology Group

IBM Power Technology Symposium 2011

Top 10 CIO Business and Technology Priorities in 2012

10 -Security10 -Expanding into new markets and geographies

9 -ERP Applications9 -Improving marketing and sales effectiveness

8 -CRM8 -Attracting and retaining the workforce

7 4IT Management7 -Improving profitability (margins)

6 -Legacy Modernization6 8Improving efficiency

5 2Virtualization5 9Delivering operational results

4 8Collaboration technologies (workflow)4 4Creating new products and services (innovation)

3 1Cloud computing (SaaS, IaaS, PaaS)3 3Reducing enterprise costs

2 3 Mobile technologies2 2Attracting and retaining new customers

1 5Analytics and business intelligence1 1Increasing enterprise growth

Rank’12 ‘11

Top 10 Technology PrioritiesRank’12 ‘11

Top 10 Business Priorities

Source: Gartner Executive Programs (January 2012)

Gartner CIO Study

Page 4: IBM Systems and Technology Group Nick Gruendler Blades and ... · IBM Systems and Technology Group IBM Power Technology Symposium 2011 ... Looking at the graphs on the previous slides,

4 © Copyright IBM Corporation 2012

IBM Systems and Technology Group

IBM Power Technology Symposium 2011

How is the “Power” Problem Appearing Today?Gartner Data Center Conference – 12/05- Attendee Polls

1) Over 50% of poll respondents cited Excessive Heat or Insufficient Power as their key issueQuestion: What is the greatest facility problem with your primary data center?

0

5

10

15

20

25

30

35

ExcessiveHeat

InsufficientPower

InsufficientRaised

Floor Area

PoorLocation

ExcessiveFacility Cost

Other

12/04 n=112

12/05 n=182

2) 15% respondents cited specialty cooling solutionsQuestion: What is your primary source of cooling? n=180

• Centralized cooling plant only 29%• Distributed cooling (CRAC units) 57%• CRAC units plus specialty cooling for hot spots 12%• Specialty cooling only (water cooled; rack enclosure, etc.) 3%

Source: Gartner Data Center Conference, 12/05 http://www.gartner.com/2_events/conferences/lsc24.jsp

Over the past few years, electrical cost has been relatively flat (1). But the absolute power consumption is still high and is still increasing:

- “Datacenter energy consumption growing at a rate of 12% per year.” (2)

- “If the internet was rated as a country, it’s total energy consumption would be 5th in the world, just after Japan and above Russia”. (2)

Price spikes in electricity expected in the near future:

- Running out of power capacity from the utility grid

- Current issues relating to power and cooling intensifying in the near term

Datacenters are heating up because power in = heat out.Density requires White Space / drives Lack of SpaceIn 2008, 50% of data centers had insufficient power and cooling

Relative Power Consumption of the Internet:

(1) http://www.eia.doe.gov/cneaf/electricity/epm/table5_6_b.html

(2) Peer 1, http://www.psfk.com/2011/04/data-center-power-usage-infographic.html

Page 5: IBM Systems and Technology Group Nick Gruendler Blades and ... · IBM Systems and Technology Group IBM Power Technology Symposium 2011 ... Looking at the graphs on the previous slides,

5 © Copyright IBM Corporation 2012

IBM Systems and Technology Group

IBM Power Technology Symposium 2011

Power Saving Features Within a Server

BRW: Systemx_Power_Savings.ppt

The power savings & efficiency seen at the system level is a

combined effect of many individual server features.

Page 6: IBM Systems and Technology Group Nick Gruendler Blades and ... · IBM Systems and Technology Group IBM Power Technology Symposium 2011 ... Looking at the graphs on the previous slides,

6 © Copyright IBM Corporation 2012

IBM Systems and Technology Group

IBM Power Technology Symposium 2011

Composite Effect of Power Saving Features

The composite effect of each power efficiency feature is used to optimize for efficiency on workloads and benchmarks (e.g. SPECpower shown).

It is important to understand what portion of the performance/watt curve each feature affects and to what degree.

*SPECpower benchmark shown for illustrative purposes

BR

W:

Sys

tem

x_P

ower

_Sav

ings

.ppt

Relative influence –from previous page

Page 7: IBM Systems and Technology Group Nick Gruendler Blades and ... · IBM Systems and Technology Group IBM Power Technology Symposium 2011 ... Looking at the graphs on the previous slides,

© 2012 IBM Corporation

Sustainability on a Smarter Planet

IBM Energy Efficiency TrendsIBM Technology SymposiumOctober 2012

Page 8: IBM Systems and Technology Group Nick Gruendler Blades and ... · IBM Systems and Technology Group IBM Power Technology Symposium 2011 ... Looking at the graphs on the previous slides,

8 © 2012 IBM Corporation

Sustainability on a Smarter Planet: Green Infrastructure

1.Server Generations2.Server Components3.UEFI and OS Settings4.Ambient Temperature and DC Considerations5.Green IT6.Regulatory Roadmap

IBM x86 Energy Efficiency Trends

Page 9: IBM Systems and Technology Group Nick Gruendler Blades and ... · IBM Systems and Technology Group IBM Power Technology Symposium 2011 ... Looking at the graphs on the previous slides,

9 © 2012 IBM Corporation

Sustainability on a Smarter Planet: Green Infrastructure

Server GenerationsIBM’s intention is to present data here that shows how:

– Generally, maximum power requirements are trending slowly upwards (as more and more components go into x86 servers, e.g. more Cores, more DIMMs, more disks, more PCI cards) – but customers are getting more for less in terms of power consumption.

– Idle power requirements are trending downwards.

– Performance per Watt is improving – therefore there are benefits with newer servers.

This data currently focuses on the 2 Socket EP server.

The overall objective is to demonstrate efficiency gains over time..

This is the data collation stage; no recommendations are currently provided.

Page 10: IBM Systems and Technology Group Nick Gruendler Blades and ... · IBM Systems and Technology Group IBM Power Technology Symposium 2011 ... Looking at the graphs on the previous slides,

10 © 2012 IBM Corporation

Sustainability on a Smarter Planet: Green Infrastructure

– Generational Comparison, Power

Configuration used here:Software settings optimised for optimal efficiency. Performance and power metrics were measured running a Java application.

Idle Power (W)

x3650 (M1)

x3650 M2x3650 M3

x3650 M4

100

120

140

160

180

200

220

240

7%

53%

Max Power (W)

x3650 (M1)

x3650 M2

x3650 M3

x3650 M4

250.00

270.00

290.00

310.00

330.00

350.00

370.00

390.00

9%

22%

These two graphs demonstrate how, for a static configuration:

– Idle power draw decreases dramatically over the server generations.

– Max power also decreases, but not as dramatically, for this static configuration.

– Obvious Conclusion – older servers are less efficient.Blue = improvement from M3 to M4

Yellow = improvement from M1 to M4

(M1) = HarpertownM2 = Nehalem

M3 = WestmereM4 = Sandy Bridge

Page 11: IBM Systems and Technology Group Nick Gruendler Blades and ... · IBM Systems and Technology Group IBM Power Technology Symposium 2011 ... Looking at the graphs on the previous slides,

11 © 2012 IBM Corporation

Sustainability on a Smarter Planet: Green Infrastructure

– Generational Comparison, Performance

Relative Efficiency

x3650 (M1)

x3650 M2

x3650 M3

x3650 M4

0.0

1.0

2.0

3.0

4.0

5.0

6.0

7.0

6.6x1.6x

These two graphs demonstrate how, for a static configuration

– Performance provided over server generations has increased by almost 4x.

– For a fixed configuration, as performance increases, power consumption decreases and the relative efficiency of the latest generation of servers (x3650 M4) is 6.6x that of the oldest compared here, the x3650 (M1), like-for-like.

Blue = improvement over Generation

Yellow = improvement from M1 to M4

Relative Performance

x3650 (M1)

x3650 M2

x3650 M3

x3650 M4

1.0

1.5

2.0

2.5

3.0

3.5

4.0

3.9x1.3x

2x

1.5x

Page 12: IBM Systems and Technology Group Nick Gruendler Blades and ... · IBM Systems and Technology Group IBM Power Technology Symposium 2011 ... Looking at the graphs on the previous slides,

12 © 2012 IBM Corporation

Sustainability on a Smarter Planet: Green Infrastructure

– Performance vs. Power Utilisation TrendingFor all server generations reviewed, power draw has risen almost linearly in line with server utilisation. No large steps in power vs. utilisation exist. Therefore higher utilisation is positive.

System Power Versus Performance (SPECpower_ssj normalized)

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

80.00%

90.00%

100.00%

0% 20% 40% 60% 80% 100%% Power Load

% P

erfro

man

ce

Jun '09 (X5570)apr '10 (X5670)apr '11 (X5675)apr '12 (E5-2660)'05 Dual Core

Page 13: IBM Systems and Technology Group Nick Gruendler Blades and ... · IBM Systems and Technology Group IBM Power Technology Symposium 2011 ... Looking at the graphs on the previous slides,

13 © 2012 IBM Corporation

Sustainability on a Smarter Planet: Green Infrastructure

– Data InterpretationLooking at the graphs on the previous slides, we can gauge how much less power will be consumed for a given workload and how many fewer servers will be needed. For example:

A given workload requires 100 x3650 M3 servers.

If the x3650 M3 servers are upgraded to x3650 M4, how many servers are required andwhat is the reduction in maximum power consumption?

The x3650 M4 provides 1.3x the performance of an x3650 M3, therefore, the sameworkload can be completed with 76 servers (100 / 1.3).

The maximum power for a x3650 M3 server is 316W AC; total power for 100 servers is31,600W.

The maximum power for a x3650 M4 server is 287W AC, total power for 76 servers is 21,812W or 69% of the power that is drawn by the 100 M3 servers.

Power saving with M4 servers is 19,002W, or 30% of the power drawn by the x3650 M3s.

The opposite exercise can also be performed. That is, for a fixed power budget, how many more M4 servers can be installed vs. M3 servers? Answer: 10 additional M4 servers can be installed and the 110 servers will yield a performance boost of 43% over 100 M3s.

This position would only improve if it was even older x86 servers that were being considered.

Page 14: IBM Systems and Technology Group Nick Gruendler Blades and ... · IBM Systems and Technology Group IBM Power Technology Symposium 2011 ... Looking at the graphs on the previous slides,

14 © 2012 IBM Corporation

Sustainability on a Smarter Planet: Green Infrastructure

Generational Xeon and x3650 Server Performance(Using a blend of SPECint and SPECfp Rates)

0

50

100

150

200

250

E5140(M1, R1)

E5345(M1, R2)

E5440(M1, R3)

E5540(M2)

E5640(M3)

E5620(M3)

E5-2609(M4)

– Generational Comparison, PerformanceThis is a second view illustrating the performance improvements across the different Xeon generations, using specific Xeons (no hard set server configuration assumed).

5.6x

Page 15: IBM Systems and Technology Group Nick Gruendler Blades and ... · IBM Systems and Technology Group IBM Power Technology Symposium 2011 ... Looking at the graphs on the previous slides,

15 © 2012 IBM Corporation

Sustainability on a Smarter Planet: Green Infrastructure

x3650 Servers - Maximum RAM Footprint

0100200300400500600700800

x3650 M1(R1)

x3650 M1(R2)

x3650 M1(R3)

x3650 M2 x3650 M3 x3650 M4

GB

– Generational Comparison, RAMAs servers have moved through the generations, RAM footprints have increased dramatically through the inclusion of more physical DIMM slots and the introduction and support of higher capacity DIMMs.

At the same time, costs of memory has generally been falling.

16x

Page 16: IBM Systems and Technology Group Nick Gruendler Blades and ... · IBM Systems and Technology Group IBM Power Technology Symposium 2011 ... Looking at the graphs on the previous slides,

16 © 2012 IBM Corporation

Sustainability on a Smarter Planet: Green Infrastructure

– Generational Comparison, PSUsThis view shows how the PSUs in this 2 Socket Rack server have changed over the different generations. Obviously PSU capability does not reflect power draw, but this gives us a good view of the different ratings of the PSUs and how the wattage of the PSUs has not risen as dramatically as performance, or RAM footprint. It is fair for us to conclude that customers are getting more dramatic increases in capability for less dramatic increases in power.

x3650 Power Supply Ratings

0

200

400

600

800

1000

x3650 M1(R1)

x3650 M1(R2)

x3650 M1(R3)

x3650 M2 x3650 M3 x3650 M4

Watts

Page 17: IBM Systems and Technology Group Nick Gruendler Blades and ... · IBM Systems and Technology Group IBM Power Technology Symposium 2011 ... Looking at the graphs on the previous slides,

17 © 2012 IBM Corporation

Sustainability on a Smarter Planet: Green Infrastructure

Server ComponentsSpecific server components have demonstrable improvements in their power / performance or power / capacity or efficiency perspective. The key components to look at are:

– Intel Xeons, using collated MI, IBM can show that 80w Xeons are favoured by some customers over more expensive 95w and 130w Xeons.

– PSUs are becoming more efficient each generation.

– Regulators: DC/DC converter efficiency improvement trend.

– Memory DIMMs – The transition from 1.5v DIMMs to 1.35v DIMMs across the board provides a 20% reduction in power consumption for no price or performance penalty. Capacities are rising dramatically, far outstripping power requirements.

– Integrated networking – move to lower power LOM

– Increased usage of SSDs: power to spin media removed

Page 18: IBM Systems and Technology Group Nick Gruendler Blades and ... · IBM Systems and Technology Group IBM Power Technology Symposium 2011 ... Looking at the graphs on the previous slides,

18 © 2012 IBM Corporation

Sustainability on a Smarter Planet: Green Infrastructure

- System PSU Efficiency Trend

2U Server Power Supply Efficiency Trends(by system/PS release date)

65%

70%

75%

80%

85%

90%

95%

100%

Jan-04May-05

Oct-06Feb-08

Jul-09Nov-10

Apr-12Aug-13

Dec-14

50% Load PS Eff iciency

Avg PS Eff iciency

100% Load Eff iciency

20% Load Eff iciency

10% Load Eff iciency

x346

x3650

x3650 M2x3650 M3 x3650 M4

Future

80Plus Titanium Average

80Plus Platinum Average

80Plus Gold Average

Improvements in power conversion designs and components have allowed for continuing increases in PSU efficiency. The efficiencies are approaching maximums for conversion topologies and only incremental gains may continue.

Page 19: IBM Systems and Technology Group Nick Gruendler Blades and ... · IBM Systems and Technology Group IBM Power Technology Symposium 2011 ... Looking at the graphs on the previous slides,

19 © 2012 IBM Corporation

Sustainability on a Smarter Planet: Green Infrastructure

Major improvement in conversion efficiencies have occurred as the industry put significant focus on decreasing the losses within the power circuitry, while, simultaneously, the size of the PSUs has been shrinking to accommodate an increasing number of server components.Note: the future target of 80PLUS Titanium, while feasible, is a goal that will require costs to decrease for high volume production and will be near the maximum obtainable with standard AC input PSUs.

- System PSU Efficiency Trend

80Plus Eff Spec (% PSU Load)System Release Date Power Supply 80PLUS Rating 10% 20% 50% 100%x346 Sep 2004 625W None - - - -x3650 (M1) Jul 2006 835W Bronze n/a 81% 85% 81%x3650 M2 Apr 2009 675W Gold n/a 88% 92% 88%x3650 M3 Mar 2010 675W Platinum n/a 90% 94% 91%x3650 M3 Apr 2011 675W Platinumx3650 M4 Mar 2012 550 / 750 / 900W PlatinumFuture Target TBD 750 / 900W Titanium 90% 94% 96% 91%

Page 20: IBM Systems and Technology Group Nick Gruendler Blades and ... · IBM Systems and Technology Group IBM Power Technology Symposium 2011 ... Looking at the graphs on the previous slides,

20 © 2012 IBM Corporation

Sustainability on a Smarter Planet: Green Infrastructure

- Right-Sizing PSU for Efficiency / CostPSU Efficiency

70.00%

75.00%

80.00%

85.00%

90.00%

95.00%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

550W750W900W

PSU Efficiency versus Output Power

50.00%

55.00%

60.00%

65.00%

70.00%

75.00%

80.00%

85.00%

90.00%

95.00%

0.00 100.00 200.00 300.00 400.00 500.00 600.00 700.00 800.00 900.00 1000.00

550W750W900W

The PSU efficiency curves are similar when shown onthe same scale:

(% efficiency vs. # load).

Viewing PSU efficiency vs.output power shows that, forlightly loaded or configuredsystems, the 550W PSU may provide the best efficiency.

Standardizing on the 900W PSU, at idle this can be a difference of over 7W of input power and over 3W with the load data shown in section 1.1.

2 x PSU Idle (W) Load (W)550W 108.8 286.6750W 116.1 290.2900W 116.4 289.5

Page 21: IBM Systems and Technology Group Nick Gruendler Blades and ... · IBM Systems and Technology Group IBM Power Technology Symposium 2011 ... Looking at the graphs on the previous slides,

21 © 2012 IBM Corporation

Sustainability on a Smarter Planet: Green Infrastructure

CPU and Memory represent the two highest loads within a server and are a major focus for reducing the power conversion loses for the DC/DC converters powering these devices.

Due to a wide load range required in CPU DC/DC designs (one design must support 60W to 135W Xeons) the gains are not as large as for the memory regulators. Major efficiency gains were achieved on both CPU and RAM VRegs when the original x3650 was introduced.

- DC / DC Regulator Trends

x3650 DC/DC Regulator Trend

78%80%82%84%86%88%90%92%94%96%

x3650 x3650M2 x3650M3 x3650M4 Future

System

Volta

ge R

egul

ator

Eff

icie

ncy

CPU Vreg Avg

Mem Vreg Avg

Page 22: IBM Systems and Technology Group Nick Gruendler Blades and ... · IBM Systems and Technology Group IBM Power Technology Symposium 2011 ... Looking at the graphs on the previous slides,

22 © 2012 IBM Corporation

Sustainability on a Smarter Planet: Green Infrastructure

- Memory DIMMs1.35v DIMMs benefit from the following characteristics:

– No performance impact when compared to 1.5v DIMMs.

– No price premium when compared to 1.5v DIMMs.

– A 20% reduction in power consumption when compared to 1.5v DIMMs (for a fixed frequency, the DIMM power draw is the square of the operating voltage, hence 20%).

– 1,600MHz capable DIMMs forthe Sandy Bridge platform are1.5v options and bring an improvement to overallsystem performance (20%).

Page 23: IBM Systems and Technology Group Nick Gruendler Blades and ... · IBM Systems and Technology Group IBM Power Technology Symposium 2011 ... Looking at the graphs on the previous slides,

23 © 2012 IBM Corporation

Sustainability on a Smarter Planet: Green Infrastructure

- Memory DIMM Power as a Function of SpeedAs the speed that an individual DIMM operates at reduces, so does the amount of power that is consumed by that DIMM – however, as expected, there is a performance impact. Based on the workload requirements and number of DIMMs installed, it could be possible to make powers savings without impacting performance too negatively.

Page 24: IBM Systems and Technology Group Nick Gruendler Blades and ... · IBM Systems and Technology Group IBM Power Technology Symposium 2011 ... Looking at the graphs on the previous slides,

24 © 2012 IBM Corporation

Sustainability on a Smarter Planet: Green Infrastructure

- NetworkingThe IBM x3650 M4 moves to more efficient NICs

– NIC now consumes 4.3 watts maximum for 4 ports, compared to 3.8 watts for 2 ports.

– Energy Efficient Ethernet (EEE) – shuts down the link circuitry during periods of inactivity, and when new data needs the link, it is powered up very quickly with no data loss.

– Platform Power Management (PPM) ready,which allows the DMA coalescing synchronisation between multiple controllers / adapters.

120

130

140

150

160

170

180

Idle 250mb/s Traffic

Interrupt Moderation

DMA Coalescence

130

174

166

140

Power in Watts

8W

26W

Page 25: IBM Systems and Technology Group Nick Gruendler Blades and ... · IBM Systems and Technology Group IBM Power Technology Symposium 2011 ... Looking at the graphs on the previous slides,

25 © 2012 IBM Corporation

Sustainability on a Smarter Planet: Green Infrastructure

- Solid State Disks (SSDs)

Trend toward energy efficient storage (no spinning media).

– Use somewhat hampered by limited storage capacity.

Page 26: IBM Systems and Technology Group Nick Gruendler Blades and ... · IBM Systems and Technology Group IBM Power Technology Symposium 2011 ... Looking at the graphs on the previous slides,

26 © 2012 IBM Corporation

Sustainability on a Smarter Planet: Green Infrastructure

- BIOS and OS Power Saving SettingsMore UEFI profile choices are available to customers:|

Power Savings versus Performancenothing comes for free, so you can expect performance impacts by changingsettings to be more power savings focused – i.e. memory speed, QPI links, etc

Options that are available from within the OS:

– Windows 2008 R2 – moved from a Performance to a Balanced profile by default, providing a good set of power saving features. Other tweaks are available

– Linux (RHEL and SLES) – CPUFreq sub-system Governors have been available since RHEL 5 / SLES 9 but the hardware / BIOS must support these features (support for P-states)..

Objective is to help customers understand the “big hitters” in this power saving space, plus a set of overall recommendations.

Page 27: IBM Systems and Technology Group Nick Gruendler Blades and ... · IBM Systems and Technology Group IBM Power Technology Symposium 2011 ... Looking at the graphs on the previous slides,

27 © 2012 IBM Corporation

Sustainability on a Smarter Planet: Green Infrastructure

- UEFI Settings

Remote memory bandwidth can reduce as much as 11% when QPI speed is dropped from 8GT/s to 6.4 GT/s and a further 10% when dropped to 5.6 GT/s.

2 to 5W AC when set to “minimal power” setting

QPI Link Frequency

Latency can increase by as much as 19%. Will only affect performance for applications that have short bursts of activity.Can enable higher processor frequency through Turbo mode when fewer Cores are active.

Up to 37W AC / Socket savings when maximally enabled, which would not be the case as maximally enabled C-states can introduce other problems, as have been observed by customers

C-states, ACPI C-state limit

No performance penalty unless frequency is limited through OS power daemons or power themes.

Up to 55W AC / Socket savings when enabled

Processor Performance States (P-states)

Majority of applications benefit with all prefetchers enabled.However, 4% increase on Java applications if hardware prefetcher, adjacent cache line prefetcher, and DCU streamer prefetcherdisabled, DCU IP prefetcher left enabled.

NonePrefetchersPerformance ImpactPower ImpactUEFI Setting

What impact do individual UEFI settings on power and performance?

Page 28: IBM Systems and Technology Group Nick Gruendler Blades and ... · IBM Systems and Technology Group IBM Power Technology Symposium 2011 ... Looking at the graphs on the previous slides,

28 © 2012 IBM Corporation

Sustainability on a Smarter Planet: Green Infrastructure

- UEFI Settings

23% reduction in memory bandwidth if disabled. Disabling is not recommended for most applications.For certain non-NUMA applications, disabling NUMA can produce ~4% increase in performance.

NoneNUMA

Depending on processor SKU, can generate up to 29% (E5-2680) improvement in performance for single threaded applications and up to 15% (E5-2680) for multi-threaded and multi-programmed applications if enabled.

30% power savings at maximum utilisation if disabled.

Turbo

4% efficiency increase on some workloads when set to “efficiency favour performance”

Power/Performance Bias

No impact to memory latency or throughput.Up to 1W AC per idle DIMM when enabled

Memory Power Management

31% reduction in memory performance when speed is changed from 1,600MHz to 1,066MHz.

5 to 15W AC when set to “minimal power” setting

Memory Frequency

Performance ImpactPower ImpactUEFI Setting

These are the “big hitters” that can affect both server power savings in a positive way andperformance in a negative way. Start with the default UEFI operating mode (Efficiency –Favour Performance) and tweak settings to assess impact across a variety of workloads.

Page 29: IBM Systems and Technology Group Nick Gruendler Blades and ... · IBM Systems and Technology Group IBM Power Technology Symposium 2011 ... Looking at the graphs on the previous slides,

29 © 2012 IBM Corporation

Sustainability on a Smarter Planet: Green Infrastructure

- OS Power Savings OptionsModern OSs all provide power savings options.

Windows controls power savings through “power plans”.– This feature has been available since Windows 2003.– There are 3 built-in plans, plus the ability for users to customise their own plan.

Linux controls power savings through “governors” that are part of the CPUFreqsubsystem.– The CPUFreq subsystem has been available since the Linux 2.6.0 kernel (RHEL 5 and

SLES 9).– IBM developerWorks article “Reduce Linux power consumption, Part 1: The CPUfreq

subsystem” at http://www.ibm.com/developerworks/linux/library/l-cpufreq-1/index.html provides an overview of the CPUFreq subsystem and its governors.

– See also the RHEL 6 Power Management Guide at http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/6/html-single/Power_Management_Guide/index.html

For both Windows and Linux, the underlying hardware must support P-states and they must be enabled in the BIOS. All IBM System x servers released in the last 5 years support P-states; therefore that includes many of the servers within consideration.

If servers are running on older OSs / hardware, that could be a case for evergreening.

Page 30: IBM Systems and Technology Group Nick Gruendler Blades and ... · IBM Systems and Technology Group IBM Power Technology Symposium 2011 ... Looking at the graphs on the previous slides,

30 © 2012 IBM Corporation

Sustainability on a Smarter Planet: Green Infrastructure

– OS Power Savings OptionsThe best power savings option will depend upon the workload. In defining a standard, confirming that the default power options are enabled and then consider if further investigation leads to a conclusion that the Power Save (Windows) and powersave (Linux) options are potential candidates for global defaults, exceptions being allowed, if justified.

In general, IBM recommends running with the default OS power savings option.– For Windows Server 2008 R2, the Balanced power plan.– For Linux, the ondemand Governor.

Tweaks for specific workloads may be warranted.

Page 31: IBM Systems and Technology Group Nick Gruendler Blades and ... · IBM Systems and Technology Group IBM Power Technology Symposium 2011 ... Looking at the graphs on the previous slides,

31 © 2012 IBM Corporation

Sustainability on a Smarter Planet: Green Infrastructure

Ambient Temperature and DC ConsiderationsIBM’s focus with this input is on air cooling – how IBM manage cooling efficiency as a function of system design enhancements. Water cooling also forms a part of IBM’s data centre strategies, but that will not be not covered in this input (unless specifically asked for).

With regards water-cooling in the DC, IBM’s perspective is that this is not and probably will not ever be a requirement, but there is a TCO case that shows that having water (even retro-fitting it to a DC) can pay back over a 4 year period.

Trends and Expectations:

– European COC 40° C inlet support – “Starting 2012 new IT equipment should be able to withstand the air inlet temperature and relative humidity ranges of 5°C to 40°C and 5% to 80% RH, non-condensing respectively, and under exceptional conditions up to +45°C as described in ETSI EN 300 019 Class 3.1”.

– IBM M4 and PureSystems support ASHRAE Class A3 Operation up to 40° C due to new airflow control algorithms – the systems now have more sensors, including ambient, CPU, memory, IO and PSUs which allow better control of fans.

– For idle systems, throttle power and cooling to absolute minimum requirements.

– Ensure efficient and uncompromised system performance over wider range of computing environments.

Page 32: IBM Systems and Technology Group Nick Gruendler Blades and ... · IBM Systems and Technology Group IBM Power Technology Symposium 2011 ... Looking at the graphs on the previous slides,

32 © 2012 IBM Corporation

Sustainability on a Smarter Planet: Green Infrastructure

– Impact of Ambient Temperature, PowerThe inlet temperature can impact a server’s power consumption in a fairly consistent manner in an idle state and by running a simulated workload – up by 17% at 20° C and 7% at 35° C.

Ambient Inlet Temperature vs. Server (x3550 M3) Power Draw(Same Idle State and Same Simulated Workload)

0

50

100

150

200

250

300

20° C 25° C 27° C 30° C 35° C

Inlet Temperature

WattsIdleSimulated Workload

Page 33: IBM Systems and Technology Group Nick Gruendler Blades and ... · IBM Systems and Technology Group IBM Power Technology Symposium 2011 ... Looking at the graphs on the previous slides,

33 © 2012 IBM Corporation

Sustainability on a Smarter Planet: Green Infrastructure

– Impact of Ambient Temperature, Airflow

Server airflow also increases as the inlet temperature increases.

Ambient Temperature - degrees C

10 15 20 25 30 35

Ser

ver A

ir Fl

owra

te In

crea

se

1.0

1.2

1.4

1.6

1.8

2.0

2.2

2.4

2.6

2.8

Page 34: IBM Systems and Technology Group Nick Gruendler Blades and ... · IBM Systems and Technology Group IBM Power Technology Symposium 2011 ... Looking at the graphs on the previous slides,

34 © 2012 IBM Corporation

Sustainability on a Smarter Planet: Green Infrastructure

– Supported Ambient TemperaturesThe table below shows supported ambient temperatures for IBM’s x86 servers:

The latest M4 generation of 2 Socket servers supports both lower and higher ambient temperatures. The same is true of PureFlex system, but is not true of all IBM Blades.

2 Socket EP Blade Ambient Temp Altitude Ambient Temp Altitude

HS20 10° C to 35° C 0 to 3,000 ft 10° C to 28° C 0 to 7,000 ft

HS21 10° C to 35° C 0 to 3,000 ft 10° C to 28° C 0 to 7,000 ft

HS22 10° C to 35° C 0 to 3,000 ft 10° C to 28° C 0 to 7,000 ft

HS23 10° C to 35° C 0 to 3,000 ft 10° C to 28° C 0 to 7,000 ft

2 Socket EP Rack

x3650 - M1 10° C to 35° C 0 to 3,000 ft 10° C to 28° C 0 to 7,000 ft

x3650 - M2 10° C to 35° C 0 to 3,000 ft 10° C to 28° C 0 to 7,000 ft

x3650 - M3 10° C to 35° C 0 to 3,000 ft 10° C to 28° C 0 to 7,000 ft

x3650 - M4 (E5-2643, E5-2670) 10° C to 35° C 0 to 3,000 ft 10° C to 28° C 0 to 10000 ft

x3650 - M4 (E5-2609, E5-2640, E5-2650) 5° C to 40° C 0 to 3,000 ft 10° C to 28° C 0 to 10000 ft

4 Socket EX Rack

x3850 10° C to 35° C 0 to 3,000 ft 10° C to 28° C 0 to 7,000 ft

x3850 M2 10° C to 35° C 0 to 3,000 ft 10° C to 28° C 0 to 7,000 ft

x3850 X5 / 7145 (E7520, E7540, X7560) 10° C to 35° C 0 to 3,000 ft 10° C to 28° C 0 to 7,000 ft

x3850 X5 / 7143 (E7-4807, E7-4830, E7-4860) 10° C to 35° C 0 to 3,000 ft 10° C to 28° C 0 to 7,000 ft

Page 35: IBM Systems and Technology Group Nick Gruendler Blades and ... · IBM Systems and Technology Group IBM Power Technology Symposium 2011 ... Looking at the graphs on the previous slides,

35 © 2012 IBM Corporation

Sustainability on a Smarter Planet: Green Infrastructure

– Server Technologies to Support CoolingIBM’s System x servers include a number of technologies to benefit the cooling challenge, including:

–System Layout – “Ocean Front” cooling for CPUs and Memory, ducted and zonal airflow control of cooling.

–Heatsinks – optimised for efficiency, vapour chambers or heat pipes for Xeon cooling.

– Interface Materials – high performance thermal conductivity interface materials between Xeons and heatsinks.

–Fan technologies – sensors used for active fan control, IBM System x and suppliers engineered high efficiency fans for a 10+% increase in performance.

–Cooling Control Schema – patented cooling algorithm to optimise cooling fan power, system power, component temperatures.

Page 36: IBM Systems and Technology Group Nick Gruendler Blades and ... · IBM Systems and Technology Group IBM Power Technology Symposium 2011 ... Looking at the graphs on the previous slides,

36 © 2012 IBM Corporation

Sustainability on a Smarter Planet: Green Infrastructure

Green StatementsIBM’s strategy for energy efficiency including information on data centres, stewardship and sustainability is located at http://www.ibm.com/ibm/green/.IBM releases an annual report highlighting environmental programmes, including product energy improvements and efficiency.Climate protection information is located at http://www.ibm.com/ibm/environment/climate/.Specific information on product energy efficiency is located at http://www.ibm.com/ibm/environment/products/energy.shtml

Page 37: IBM Systems and Technology Group Nick Gruendler Blades and ... · IBM Systems and Technology Group IBM Power Technology Symposium 2011 ... Looking at the graphs on the previous slides,

37 © 2012 IBM Corporation

Sustainability on a Smarter Planet: Green Infrastructure

The Green500 List

Full list downloadable from http://www.green500.org/lists/2012/06/top/green500_top_201206.xls

Page 38: IBM Systems and Technology Group Nick Gruendler Blades and ... · IBM Systems and Technology Group IBM Power Technology Symposium 2011 ... Looking at the graphs on the previous slides,

38 © 2012 IBM Corporation

Sustainability on a Smarter Planet: Green Infrastructure

ENERGY STAR and Qualified ServersIBM has a long history with the ENERGY STAR program. ENERGY STAR, managed by the US Department of Energy and the US Environmental Protection Agency (EPA), is a voluntary programme which sets energy efficiency and labelling requirements for different product types to identify and promote energy-efficient products.

Energy Star information – http://www-03.ibm.com/systems/hardware/energy_star/index.html

– For System x – http://www-03.ibm.com/systems/hardware/energy_star/systemx.html

In March 2001, IBM became the first company to win an ENERGY STAR “Excellence in Corporate Commitment Award”, recognising IBM's overall commitment and contributions to energy conservation and efficiency across the company's operations and in the design of its products.

Page 39: IBM Systems and Technology Group Nick Gruendler Blades and ... · IBM Systems and Technology Group IBM Power Technology Symposium 2011 ... Looking at the graphs on the previous slides,

39 © 2012 IBM Corporation

Sustainability on a Smarter Planet: Green Infrastructure

- ENERGY STAR RoadmapTier 1 standard is in effect.

The EPA excluded Blade servers from the version 1 programme.

Tier 2 specification is targeted for 2013 implementation and will include:

Required specific Perf/W testing for EPA data gathering.

Lower Idle power allowances for components.

Increased PSU efficiency targets.

Blade servers to be added to qualifying products list.

Tier 3: EPA to Implement minimum required Perf/W metric.

Page 40: IBM Systems and Technology Group Nick Gruendler Blades and ... · IBM Systems and Technology Group IBM Power Technology Symposium 2011 ... Looking at the graphs on the previous slides,

40 © 2012 IBM Corporation

Sustainability on a Smarter Planet: Green Infrastructure

Server Power Component RoadmapPSUs– 80Plus Platinum efficiency is the "standard" offering.– 80Plus Titanium specification released in Aug 2011

Technology/cost improvements needed to move adoption forward.

– Alternate Power Distribution Schemes picking up steam(customer inquiries increasing)HVDC277VacAdditional standards needed

DC/DC Voltage Regulators:Continued incremental increase in conversion efficiency.

Page 41: IBM Systems and Technology Group Nick Gruendler Blades and ... · IBM Systems and Technology Group IBM Power Technology Symposium 2011 ... Looking at the graphs on the previous slides,

41 © Copyright IBM Corporation 2012

IBM Systems and Technology Group

IBM Power Technology Symposium 2011

THANK YOU

Contact InformationNick GruendlerChief Engineer –Blades and Modular Power [email protected]

Page 42: IBM Systems and Technology Group Nick Gruendler Blades and ... · IBM Systems and Technology Group IBM Power Technology Symposium 2011 ... Looking at the graphs on the previous slides,

42 © Copyright IBM Corporation 2012

IBM Systems and Technology Group

IBM Power Technology Symposium 2011

Copyright information

©Copyright IBM Corporation 2012

IBM Global ServicesRoute 100Somers, NY 10589U.S.A.

Produced in the United States of America05-11All Rights Reserved

IBM and the IBM logo are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both.

Other company, product and service names may be trademarks or service marks of others.

Use of the information herein is at the recipient's own risk. Information herein may be changed or updated without notice. IBM may also make improvements and/or changes in the products and/or the programs described herein at any time without notice.

References in this publication to IBM products or services do not imply that IBM intends to make them available in all countries in which IBM operates.