When running Linux on IBM System z When running Linux on IBM System z makes sense...makes sense...Xmakes sense...makes sense...Xcents

Velocity Software Inc. / Hill Gang Velocity Software Inc. / Hill Gang –– July 14thJuly 14th

Len DiegelLen Diegel

L @ l it ftL @ l it ftLen@velocitysoftware.comLen@velocitysoftware.com

Agenda

• Today’s “legacy system”! Times have changed.h i ff i i ll i• How changes in IT are affecting your installation.

– IT Trends & Directions– TCO Considerations

• Measure, Track, and Report - Why bother?• Results – What should you expect?• Tools of the trade: The good, the bad, & the sad

Background • This session is based on actual customer experience & p

history as it relates to migrations from x86 (commodity) server “farms” to Linux on z.Th i d bl th t d l th• The issues and problems they encountered along the way.

• Their efforts to resolve those issues and problems.• How they adapted overcame and survived ! !• How they adapted, overcame, and . . .survived ! !

Definition: “Legacy System”

“A legacy system is an old computer system or application program that continues to be used because the user (typically an organization) does not want to replace or redesign it.”

en.wikipedia.org/wiki/en.wikipedia.org/wiki/

“Legacy can be a burden or heritage g y gwith dignity”…

Karl-Heinz Strassemeyer (IBM Corp)

IT in 2010 has a new set of issues.

Unfortunately, the “legacy systems” of the 90s are creating more issues to deal with than thecreating more issues to deal with than the mainframe of the 70s. A nightmare to manage & administer. Limited to horizontal capacity increases. (see below) Increasingly expensive to maintain.

Today’s legacy systems are the byproduct of the Today’s legacy systems are the byproduct of the “add another rack” IT generation of the 90’s“add another rack” IT generation of the 90’sadd another rack IT generation of the 90 s . . .add another rack IT generation of the 90 s . . .

Challenges to the IT Professional . .

Environmentals –Space, Heat, Power, “Green” Efforts

AdministrativeMaintenance, Upgrades, Charge Back, Capacity

Planning Performance DR Data Security and morePlanning, Performance, DR, Data Security, and more Internal IT PressuresMigrations, Consolidations, Compliance Issues, g , , p ,

Maximizing Resources, Flexible Capacity Industry Strategies & DirectionsCloud / Storm / SUN-down / Sky-BLUE/ Cloudburst

What everyone wants from IT . .

Secure, high speed access to data any time from anydata any time, from any

device, and from any where.

Even from a dive boat in fTurks Caicos working with Neale on breakfast for this

eventevent . .

What IT Execs are thinking:

Response from 100 IT executives at companies using Linux operatingsystem on IBM mainframes.

93% of respondents projected that their use of IBM's IFL (Integrated93% of respondents projected that their use of IBM s IFL (IntegratedFacility for Linux) would increase or remain steady.

The two main reasons:

1. Advantage of computing capacity available on their mainframe'sand IFLs.

22 LiLi thth i fi f ldld bb tt ff tiff ti thth thth2.2. LinuxLinux onon thethe mainframemainframe wouldwould bebe moremore costcost--effectiveeffective thanthan otherotherplatformsplatforms..

* http://finance.yahoo.com/news/Survey-Predicts-Continued-prnews-15547427.html?.v=1

What is the greatest problem with your primary data center ?

“P“Power and Cooling will be a top 3 issue with all CIOs in the next 6 “Power and Cooling will be a top 3 issue with all CIOs in the next 6 to 12 months.” Michael Bell to 12 months.” Michael Bell –– Gartner GroupGartner Group

$250

$300

Installed Base(M Units)

Spending(US$B)

40

45

50Power and cooling costs x8Server mgmt and admin costs x4New server spending

$100

$150

$200

15

20

25

30

35

$0

$50

0

5

10

15

Source: IDC, Virtualization 2.0: The Next Phase in Customer Adoption, Doc #204904, Dec 2006

IBM’s Global CEO Study• Over 1500 CEOs were interviewed across all geosOver 1500 CEOs were interviewed across all geos.• Most intensive study conducted in this space.• More than 80% see an increase in complexity complexity as their p yp y

number one IT issue. Fewer than 50% say they are ready.• Challenges differ from region to region:

US / G R l i– US / Government Regulations– Japan / Power shift to emerging markets– China / “Thinking Global” as opposed to closed society.

• Conclusion: “No matter what the industry, those that can manage and react to complexitycomplexity will have the advantage over those that can’t ”over those that can t.

IBM’s Mainframe Cloud Agenda

• Focus on Mainframe Strengths– Extend System z’s massive virtualization capabilities

• Leverage System z efficiencies• Offer System z (mainframe) customers value beyond that

i th l tf / l ti d d iseen via other platforms / solutions and doing so transparently to the broader community. (90 Day Free Trials on IFLs)

“Keeping your head above the clouds” STG Tech Conference 09Keeping your head above the clouds STG Tech Conference 09

IBM’s Cloud Website: http://www.ibm.com/ibm/cloud/

System z for Cloud

“As the IT world prepares for cost“As the IT world prepares for cost--cutting in 2009 servercutting in 2009 server“As the IT world prepares for cost“As the IT world prepares for cost--cutting in 2009 servercutting in 2009 serverAs the IT world prepares for costAs the IT world prepares for cost--cutting in 2009, server cutting in 2009, server virtualization seems like one of the few technologies bound for virtualization seems like one of the few technologies bound for continued growth and success. . . . continued growth and success. . . . The IBM System z may be the The IBM System z may be the b ib i l i li i l f il bl d ”l i li i l f il bl d ”

As the IT world prepares for costAs the IT world prepares for cost--cutting in 2009, server cutting in 2009, server virtualization seems like one of the few technologies bound for virtualization seems like one of the few technologies bound for continued growth and success. . . . continued growth and success. . . . The IBM System z may be the The IBM System z may be the b ib i l i li i l f il bl d ”l i li i l f il bl d ”best enterprisebest enterprise--class virtualization platform available today . . .”class virtualization platform available today . . .”best enterprisebest enterprise--class virtualization platform available today . . .”class virtualization platform available today . . .”

Enterprise Strategy Group – January 2009 Report “IBM System z: The Enterprise Server Virtualization Platform.”

IBM’s Project “BIG Green”• Consolidation of several thousand Linux servers to 30 IBM• Consolidation of several thousand Linux servers to 30 IBM

z10s. (may end up being fewer than 20..)• 80% Reduction in Energy gy• 85% Reduction in Floor Space

– 1,600 for z10s vs 11,600 square feet for Distributed• Categories as a % of Gross Savings:

FacitilitesFacitilitesConnectLaborSoftwareHW MaintHW Purch

Internal IT Study for TCO

• Determine consolidation ratio for given workloads (Linux, Windows, etc.) 20

Server Migration : x86 vs. zLinux

• Estimate annual cost to operate virtualized servers for 3-5 years.C hi d l 10

15

0

• Compare this to stand-alone provisioning or public Cloud offerings like Amazon

• N< PPR*PUR*CPF supports 0

5

10 x86/otherVirtual/IFL

• N< PPR*PUR*CPF supports that the more servers you can consolidate to virtual, the lower your cost.

01stQtr

2ndQtr

3rdQtr

4thQtr

Consolidation MathConsolidation Math

N= # of virtual servers / Proc Perf Ratio / Proc Util Ratio / Cores per FrameN= # of virtual servers / Proc Perf Ratio / Proc Util Ratio / Cores per Frame

Response Time Comparison100

nds

708090

100

me

in S

econ

40506070

zVMVMware

spon

se T

im

10203040 VMware

N b f Vi t l M hi

Res

010

10 20 30 40 50 60 70 80 90

Number of Virtual Machines

Throughput Comparison160

120

140

160

60

80

100zVMVMware

tal T

PS

20

40

60 VMware

N b f Vi t l M hi

Tot

010 20 30 40 50 60 70 80 90

Number of Virtual Machines

But keep in mind, not everything was meant to run on z . .

Cost Per Image for Linux Serversin

$k

350

400

oad

Imag

e i

200

250

300S/A ServersPublic Virt.P i t Vi t

Per W

orkl

o

100

150

200 Private Virt.zLinux AddzLinux New

Cos

t P

0

50

IT Solutions

Detailed Cost Breakdown - 5 yr TCO

Platform 100 x3250 100 Amazon Large EC2

VMware & Five x3950 – 8ways

7 IFLs on z10

Hardware $5 000 000 $2 800 000 $1 080 000 $2 030 000Hardware $5,000,000 $2,800,000 $1,080,000 $2,030,000

Software $21,490,000 $20,840,000 $2,380,000 $1,520,000

Facilities and Administration

$11,020,000 $4,020,000 (admin only)

$2,760,000 $1,210,000

Total Cost for 100 Images

$37,510,000 $27,740,000 $6,220,000 $4,780,000

Cost per image $375,100 $277,400 $62,200 $47,600

IBM SWG White Paper / April 2009 & IBM Internal Study of 4000 xSeries Distributed Servers

System z Rising

• More than 1800 customers are using System z for Li

18002000

Linux.• In 09, IBM added 54 new

“mainframe” customers 120014001600

mainframe customers for the purpose of Linux

• Customer experience and l h 400

600800

1000

results support the current strategy to migrate Linux applications to System z. 0

200400

2002 2004 2006 2008pp y 2002 2004 2006 2008The number of Linux IFLs sold/year

doubled between 2006 and 2008.

Unique Value of the IFLs

• Run at full MIPS even on sub-capacity models 1.72 391%

4.4 1157% IFL Value Increase• If/when you upgrade, the

IFLs move up in power at no additional charge.

4.4 1157%

81012

IFL Value Increase

1100% over G555% over z9BC

no additional charge.• IFL’s have increased in

power by roughly 55% h i

0246

G6 z80z90z89z99z9 z10

IFL Capacity%IFL Increase

over each previous generation or 1100% since G5.

IFL Value Increase

6 8009008909909 10

Machine Type

Note: IBM delivered more IFL MIPS in 1Q10 than in 2009.

Horizontal vs. Vertical Scale (x86 vs. z)Horizontal vs. Vertical Scale (x86 vs. z)

What Horizontal vs. Vertical Scale means to you (x86 vs. z)

• How about adding a second engine to a car at the flip of a switch . . . without stopping.

• Increasing horse power from 180 HP to 450 HP at the flip of a switch . . . without stopping.

• Or being able to do both on the fly without• Or, being able to do both . . . on the fly . . . without stopping.

• System z allows you to increase your capacity either horizontally or vertically and it’s the only platform to achieve this flexibility.

But don’t get yourself in a “hole” . .

Some applications run better on z than others.

’ d dIt’s a good idea to understand the difference when ff

migrating to Linux on z.

What applications make a best fit?W bS h MQ S iWebSphere MQ Series DB2 Connect CICS Transaction Gateway , IMS Connect for Java SAP High Availability, High Availability,

h / d d hh / d d hWebSphere and JAVA applications developmentWebSphere Application Server (WAS), PortalDominoN t k I f t t FTP NFS DNS t

High I/O Bandwidth, High I/O Bandwidth, Security and DR Requirements Security and DR Requirements Simplifying the Support Model Simplifying the Support Model Shortening endShortening end toto end Path Reqsend Path ReqsNetwork Infrastructure, FTP, NFS, DNS etc..,

Oracle DatabaseApplications requiring top end disaster recovery model

Shortening endShortening end--toto--end Path Reqs end Path Reqs Reduction in Network Traffic Reduction in Network Traffic Collocation of Applications Collocation of Applications Consolidation of diverse loadsConsolidation of diverse loadsComServer and Communications Controller for

LinuxVirtualization and Security Services InfoSphere/Cognos

Consolidation of diverse loadsConsolidation of diverse loads

Communicate Pro (VoIP) “Selecting Projects for System z” “Selecting Projects for System z” Bill Reeder, IBM Corp.Bill Reeder, IBM Corp.

What applications make a good fit?

UDB (DB2) Informix, (IDS)

Evaluate Server ChoicesEvaluate Server ChoicesCorrect Application Availability Correct Application Availability S ti A li tiS ti A li tiApache web serving

SAMBA TIM/TAM (LDAP Services)

Supporting Applications Supporting Applications Political Issues within the Org. Political Issues within the Org. Potential Porting IssuesPotential Porting Issues

( )TSMExisting Linux Workloads

What applications are difficult to move?

ISV and IBM applications that have not yet ported their application to run on Linux on System z

Applications that by design run at VERY High sustained utilization which I will define here as >95%.

Stand-Alone single applications as the only Linux for System z applications

Applications that are to internally sensitive to try and move

This URL is a link to the IBM software running and supported to run under Linux on System z.http // 1 ibm com/ser ers/eser er/ series/os/lin /soft are htmlhttp://www-1.ibm.com/servers/eserver/zseries/os/linux/software.html

What workloads are moving to IFLs?

IBM Corp. 2010

Why bother to Measure TrackWhy bother to Measure, Track, and Report ?p

Len’s Travel Tips and other ways to reduce TCO . .

Len’s Travel Tips (continued)

The question becomes . . . . . .

When you hit the next “bump” on the road to better performance,

do you keep buying new tires, or do you get the front end aligned ? ? ?

Responsible IT professionals should always consider

• Performance Management to ensure service levels are met.

• Capacity Planning to ensure future needs are met.• Operational Alerts that detect issues such as p

looping, exceeding disk capacity, etc., and doing so for hundreds/thousands of servers concurrently.

• Charge back and accounting information to allow your business to charge for resources consumed.

Real Life Situations

• Financial in NY Area - Crit Sit / 2 months / SRM parms wrong – SLES8 server wouldn’t initialize

• State of Montana - Crit Sit / 1 month / Linux Server• State of Montana - Crit Sit / 1 month / Linux Server Looping, 1 IFL offline. POC Failed.

• University Crit Sit - Response Time issues.• Financial in DFW Area - System spikes every 5 minutes .

. Wrong choice of monitors . .• Industrial – Decided to buy 20G of Storage rather than use y g

best practices and set the correct parms.. • Industrial #2 – “We keep adding IFLs.” (now leaving the

platform )platform . .)

Large Insurance Company• Pain Points (history)• Pain Points (history) –

– Too many servers– Space Power CoolingSpace, Power, Cooling– Inability to Allocate Resources Where Needed– New Apps - Time/Expense of Basic Trials and Testing

• Today – (strategy deployed in less than 4 months)– +700 Servers on 2 Boxes / Dynamic Capacity Options– 100,000 Active Users– Zero Production Outages Since 2005

$3M/Y i S i– $3M/Year in Savings

Some installations are smaller:

• One CEC (system)• Two IFLs (one production, one test)• 20 Linux Servers replaced, more planned• Proof of Concept – 90 Days (free trial – including zVPS)• Savings – space, power, admin, etc.

Typical QuotesTypical Quotes• "My TSM debugging could not have been done without Velocity.

IBM’s DB2 Level 2 support was wondering where I got some of theIBM s DB2 Level 2 support was wondering where I got some of the info ;-)” …. and that level of detail…

• “Using any other monitor would cost me at least 5% of my IFL That• Using any other monitor would cost me at least 5% of my IFL. That means one (1) IFL for every 20 that I install just to monitor my system.”

• “They gave me their monitor for free.”– What about the drag on your system?

Wh t b t d di ti th Tb t f t f th ft ?– What about dedicating more than a Tbyte of storage for the software?– What did it cost you in time to install? (Did it install?)– What does it give you for data, health checks, support, etc…?

B li IT WASN’T “F ”– Believe me, IT WASN’T “Free”.

Or, one or more of the following:

• “Time to add another IFL.”• “It’s just the way Linux runs on System z.”• “You might consider using System p instead.”• “You might consider leaving that appl. on the x86.”• “This was your idea. You might consider a new job.”• “I told you this would never work on a mainframe.”• “DRT Dead Right There ”• “DRT…. Dead Right There.”

However, accomplishing the basics does create certain challenges . . As you move g yforward, what should you expect?

It’s hardly likely that you would trust your system’s performance to an astrologist . . .y p f g

Prerequisites

• A single suite of products that can deliver whatever is necessary to manage performance on z/VM and zLinux.– Keeps cost to a minimum.– Simplifies service, upgrades, education, etc.

Si l d i t f– Single vendor interface• A vendor with a history for delivering best-of-breed

products and support. p pp• A cross industry install base with references in all major

geographies.

Velocity Software Inc.• Founded in 1988 by former IBMers• Founded in 1988 by former IBMers.• Velocity products are used by hundreds of installations

worldwide; many are IBM showcase customers.• IBM uses Velocity’s products internally and makes frequent

references to our products in their Redbooks.• We have a working relationship with IBM System z labs inWe have a working relationship with IBM System z labs in

Endicott(z/VM), Poughkeepsie(zOS), and Boeblingen(zVSE).• Velocity has participated in every VM Early Support Program

since 1988 and has been a PartnerWorld member since 89since 1988 and has been a PartnerWorld member since 89.• Velocity presents research at many conferences including:

– SHARE, GSE, CMG, WAVV, and Local Linux User Groups

Velocity Software Delivers :

l• Total Data Accuracy– When necessary, identifies, prorates and corrects inaccurate data.

• Complete Data CollectionComplete Data Collection– Variety of Platforms (NT, Linux, SUN, HP, etc)– Standard Interfaces (SNMP and NETSNMP)

C l 100 1000 f– Concurrently across 100s or 1000s of servers.

• A 100% Data Capture Ratio– Best in the industry . . . Why measure incomplete data . . ?y y p

• Minimum impact on processors by monitoring agents.– Velocity’s target is .1% or less of ONE processor

z/VM – Linux Requirement Velocity Software Product All Other Vendors

Not All Performance and Systems Management Tools are Created Equal . . . .

z/VM Linux Requirement Velocity Software Product All Other Vendors

Support All Linux YES NO

Charge Back Accounting YES NO

Capacity Planning Info. YES NO

Low Cost Operation YES NO

L O h d ALow Overhead Agents YES NO

Performance Serv. (zTUNE) YES NO

Performance Education YES NOYES NO

Included in IBM Redbooks YES NO

Standard Open Linux Agent YES NO

Network Perf. Monitoring YES NO

Accounting – VM Level YES NO

Accounting Linux Process YES NOAccounting – Linux Process YES NO

Accounting – Linux Applic. YES NO

MXG and MICS Interfaces YES NO

“Trying to determine performance issues without Velocity’s suite, is like asking a doctor to give you an y , g g yexamination while wearing your clothes.”

And, the results are pretty much the same.

D O AD.O.A.

Quote from an IBM D.E. at WAVV 2010

zVPS Overview

• zVPS is Velocity’s Performance Suite. It is a collection of several components including:p g– zMON– zVWS– zVIEW– zPRO (optional)zPRO (optional)– zTUNE (optional)

zVPS Overview (cont)

• Provides real time metrics on all facets of server performance including z/VM, Linux, and other di ib d (SUN Wi d )distributed servers (SUN, Windows, etc.)

• Full function capacity planning and interfaces to planning tools such as MXG and MICSplanning tools such as MXG and MICS

• Complete and accurate charge back accounting information for Linux applications and z/VM.pp

• Operational alerts for up to 1000s of virtual servers and immediate detection of performance iissues.

zVPS Example:IBM System z

zVPS can monitor and measure the

activity of Linux on

z/VM

activity of Linux on System z in addition to distributed servers

& Distributed Servers (x86)

SUNLinux (IFL)Linux (IFL)

like SUN Solaris, Linux, HP, Windows,

and more.,

zVPS Linux

,Windows

otherNet: You can determine the level of use forNet: You can determine the level of use for otherNet: You can determine the level of use for Net: You can determine the level of use for any server in your network.any server in your network.

Data can be used for Trend Analyses

l h

90100

IFL CPU Utilization – 12 Months

607080

Prod 1

20304050 Prod 2

Test 1

01020

Jan Mar May Jul Sep NovJan Mar May Jul Sep Nov

Hmm, CPU spikes every 10 minutesHmm, CPU spikes every 10 minutes

Which user class is causing the problem?

In this case it’s one of the Linux Servers.

Drill further -> guest Suselnx2

Drill more…

Auto Arrange

Identify the offending process (“smallstr”)fy ff g p ( )

Unique Capabilities of Velocity Software

zPROzPRO

Cl i f Li S S l i VSE CMS

Cloud Enablement Tool for Systems Administrators

• Cloning for Linux Servers, Solaris, VSE, CMS users and more.

• Provides administrators the ability to define• Provides administrators the ability to define servers based on “golden images”.

• Manages z/VM security (RACF)• Manages z/VM security (RACF)• Provides an easy-to-use, intuitive front-end to

Dirmaint via a Tab approach and syntax checkingDirmaint via a Tab approach and syntax checking for defining directory entries.

• Documentation / Command search capabilitiesDocumentation / Command search capabilities

zTUNE

• Configuration analysis to ensure best possible configuration for performance.g p

• Dozens of Automated “health checks” for both Linux and z/VM.

• Analyses of your Capacity and Performance data.• Recommendations to improve performance and to p p

optimize resources whenever it’s requested.

Velocity works as an extension of the local team.

Summary

• We are committed to providing the best z/VM performance products available.

• Our zVPS Product Suite is a complete set of Performance and• Our zVPS Product Suite is a complete set of Performance and Systems Management tools designed exclusively for z/VM and Linux on IBM System z

• We provide excellent product support services, and education.• Our install base is both global and diverse.

l i S f i h i d l d i hi b• Velocity Software is the recognized leader in this space by most of IBM’s z/VM – Linux “showcase” accounts.

Proven Performance from Velocity Software

The next step is yours . . .