Developing Sizing Guide Solutions for the IBM eServer Sizing Guide Estimator

...understanding the concepts and processes of building an application-

specific sizing guide

Joseph Puhl iSeries Performance Tools – Workload Estimator Development Helen Olson-Williams iSeries Performance Analyst – Customer Benchmark Center Ronald Young iSeries Performance Analyst – iSeries Development December 2003

Page 1 of 53

Table of Contents Preface .........................................................................................................................................3

Overview .................................................................................................................................3 Common Terms: .....................................................................................................................3

1. Planning...................................................................................................................................5 2. Preparation ..............................................................................................................................8 2.1 Education ................................................................................................................................8 2.2 Questions................................................................................................................................8 2.3 Assumptions .........................................................................................................................10 3. Define the test plan...............................................................................................................11 3.1 Determine application workload............................................................................................11 3.2 Establish the hardware environment.....................................................................................13 3.3 List of Performance Tests .....................................................................................................14 3.4 Write the scripts ....................................................................................................................17 3.5 Build the database ................................................................................................................18 4. Test Execution ......................................................................................................................20 5. Analysis .................................................................................................................................24 5.1 Determine CPU Requirements .............................................................................................24 5.2 Determine Memory Requirements ........................................................................................28

Example of a memory-sizing exercise ..................................................................................30 5.3 To Size Disk Requirements ..................................................................................................32 5.4 A Word About Scalability ......................................................................................................38 6. Generation of Sizing Guide Solution ..................................................................................40 6.1 General Considerations ........................................................................................................40 6.2 Create “Help” Documentation ...............................................................................................40 7. Validation...............................................................................................................................42 8. Publication to Web................................................................................................................43 9. Final Words ...........................................................................................................................44 Appendix A: Run Procedures ..................................................................................................45 Appendix B: Page Faults..........................................................................................................46 Appendix C: Collection Services.............................................................................................47 Trademarks and Disclaimers ...................................................................................................53

Preface Overview One of the recurring challenges for IBM® Business Partners and application providers involves the accurate sizing of servers that will be running their suite of applications. Because of this large and still growing challenge, IBM has enhanced the IBM eServertm Sizing Guide Estimator (Estimator), which is an easy-to-use “sizer” of I and p Seriestm servers that run various workloads. The basic function of the Estimator is to present to the user a series of screens with a few simple questions about how the application will be used. Then, using the answers supplied, computing resources are calculated. Based on the calculated resources, and other factors, a server is selected/recommended which is a good fit for the work defined by the user-answered questions. This paper describes the process for quantifying the resource requirements of an application so that the information can be used to describe a Sizing Guide Description which, in turn, will be used as input to the Estimator. This process of building a customized sizing tool, referred to in this document as the Sizing Guide Solution, also involves identifying the critical sizing questions to be presented to the end user. The resulting formulas will be used by the Estimator in sizing a server for the intended application, and supporting application-specific messages and “Help” text available within the Estimator when sizing the application. An all inclusive, step-by-step guide is beyond the scope of this document. The intent here is to inform Business Partners of the key efforts and processes of gathering input data on user and transaction volumes to build Sizing Guide Descriptions and Sizing Guide Solutions for their applications. NOTES: The discussions in this paper, while adequate for the purposes of this brief introduction,

are anything but “intensively thorough.” Whole chapters, even entire books, have been written on the challenges and nuances of scripting transactions and many other topics discussed here. This “lite” discussion is intended, instead, to make the reader aware that there are various methodologies and that certain sizing issues should be dwelt upon in a way that yields adequate load demand for the Sizing Guide Estimator to be effective.

This document should be used in conjunction with the paper: “Developing Sizing Guide Solutions for the IBM Sizing Guide Estimator,” authored by Joseph Puhl and found at: www.developer.ibm.com/welcome/eserver/e3.pl

Common Terms: While an explanation of terms might more normally be relegated to an appendix, these terms are integral to the ease of comprehending the core information provided in this paper, so the reader is encouraged to read through the following list BEFORE continuing with this document. Application Provider / Business Partner: Any enterprise whose products and services

(e.g., ERP software, custom programming, consulting) are intended to assist an end-user organization with its Information Services needs—also referred to in the industry as a solution developer or an Independent Service Providers (ISVs).

Arm: A casual reference to a single disk drive (can be used interchangeably with “drive”).

Page 3 of 53

Baseline: A type of test that allows for tuning the server, tweaking the test and run procedures, verifying the workload’s suitability, and deciding the correct configuration for a balanced system.

Balanced configuration: A server hardware configuration in which no resources are excessively underutilized or overutilized.

Benchmark: Can mean several different things. Quite commonly, it refers to Industry Standard Benchmarks such as TPC-C, or SpecWeb. The term can also be used in a more generic sense to refer to any carefully designed test that either gathers data for sizing purposes or to prove that a server will perform as promised.

CPW/CIW: Over time, IBM analysts have identified two sets of characteristics that appear to represent a large number of environments on iSeries servers. Many applications tend to follow the same patterns as CPW (Commercial Processing Workload). These applications tend to have many jobs running brief transactions in an environment that is dominated by IBM system code performing database operations. Other applications tend to follow the same patterns as CIW (Compute Intensive Workload). These applications tend to have somewhat fewer jobs running transactions which spend a substantial amount of time in the application, itself. The term "Compute Intensive" does not mean that commercial processing is not done. It simply means that more CPU power is typically expended in each transaction because more work is done at the application level instead of at the IBM licensed internal code level. See “iSeries Performance Capabilities Reference Version 5, Release 2” (Appendix A) for a thorough description of these two measurements. This manual is found at: http://publib.boulder.ibm.com/pubs/pdfs/as400/V4R5PDF/as4ppcp5.pdf.

DASD (Direct Access Storage Device): A term commonly used to denote a disk drive. Database: A group of data whose contents are structured to be quickly accessed,

managed, and updated. The most common database type is a relational database, a tabular database where data is described in a way that it can be reshuffled and accessed in multiple ways.

Exploratory Tests: Provide general information for defining the conditions for the formal measurements and can help determine upon which servers the measurements should be made.

Page Fault: A virtual memory interrupt indicating that the next instruction or segment of data is not already located in physical memory. Instead, it has to be swapped in from disk.

Run/Test: A distinct performance test. These terms are used interchangeably in this document.

Scalability: Indicates the extent to which a server can be expanded to support business growth or increased utilization.

Workload: The defined activity that takes place during a test (may include batch and on-line activity). This is also referred to in this document as the “application processing workload.”

Sizing Guide Description: This identifies the critical sizing questions to be presented to the end user who will be using the Sizing Guide Solution.

Sizing Guide Estimator / Estimator: A shortened name for the more formal reference to “IBM eServer Sizing Guide Estimator.” These terms are used interchangeably in this document.

Sizing Guide Solution: The resulting (customized) estimator tool is created after the Business Partner uses the IBM eServer Sizing Guide Estimator. This tool continues to work hand-in-hand with the Sizing Guide Estimator. The process to create a Sizing Guide Solution is the subject of this document. (This may be referred to as the Estimator, Sizer or Sizing Guide.)

Page 4 of 53

1. Planning In setting out to build a sizing guide, it is important to think about the goals for the Sizing Guide Solution and to understand the inherent limitations of the tools and the process. In establishing sizing goals, the application provider will need to make some very basic decisions, including ironically, the decision of “how accurate or precise the estimation needs to be.” The IBM eServer Sizing Guide Estimator is not a faultlessly precise tool. Its sizing precision and accuracy will depend on how much effort is put into testing and building the Sizing Guide Solution itself. In other words, it is important to know — before jumping into the mechanics of setup, testing, and analysis — that the effort can vary from huge to small, with a corresponding relationship of accuracy. Yet, it is also imperative to understand that there is a limit on the accuracy of the Estimator. Attempting to build more accuracy into the Sizing Guide Solution than the overall Estimator has the capability to provide is a waste of effort. Additionally, incorporating “micro” levels of accuracy will essentially contribute “macro” levels of complexity to the Sizing Guide Description through an increased volume of questions that are more difficult for the end Estimator tool user to answer, and may not significantly change the resulting Estimator server recommendation. Planning is extremely important to the process of developing a Sizing Guide Solution. This is because this effort involves scrutiny of a large number of details, such as collecting and analyzing the correct data and managing the controlled environment within which the testing takes place. Common pitfalls that must be avoided include the tendency to take too many measurements; the opposite tendency to take too few measurements, or the potential to take the wrong measurements. Even the complexity of the questions that must be answered can muddy the waters. The experience level of the audience (sales representatives versus customers) will have to be taken into consideration. Server configurations need to be properly balanced and system parameters need to be set rationally to fit the mission of the application. Another mistake that will almost certainly skew the findings of a sizing benchmark is attempting to run the tests on non-dedicated servers. Take time to form an idea of the breadth of the estimations for which the tool should provide support. Here are a few examples of the range of precision that can be considered: “How many users will there be (in orders of magnitude)?", “What range of server models are being considered?”, “Is the Business Partner thinking mainly of small, 1-way servers, or imagining that the application will consume the full posture of a 32-way server?” Consider if there are other goal/objective types of considerations prior to getting into the detailed steps of how to do it. While the steps that follow are simply stated, they can represent a significant work effort. Throughout the process, it will be important to remember that the mission of the Estimator is to recommend a suitable system configuration, not to predict the exact system utilization that the customer will ultimately experience. A goal of being within about 10% of actual utilization is, in the large majority of cases, quite sufficient. Along the way, it will be important to keep this in mind as tradeoffs become necessary. One last thought involves the need to form a general expectation of how much time, effort, and server resource will be required to “pull off” this venture of building the Sizing Guide Solution and verifying it. It is best if this is evaluated now and scheduled for it. At the completion of the planning phase, this total effort can be more accurately estimated and plans can be adjusted.

Page 5 of 53

For example, by reducing variations and being willing to accept less accuracy in estimations, etc.; it will be possible to reduce the total effort/cost. At a high level, the process of building a sizing guide consists of: Preparation

– Education. Careful preparation is imperative to avoid pitfalls and to ensure that resources are identified and allocated up front. One of the first steps, therefore, involves education. Become familiar with the concepts and tools involved in building and performing sizings (including the IBM eServer Sizing Guide Development Kit and IBM eServer Sizing Guide Estimator).

– Define questions. Define questions that will be presented to the end user of WSizing Guide Solution. This should be a succinct set of questions that will broadly quantify the volume and types of activities the application will perform.

Define the test plan – Determine application workload. Describe the application workload. Identify the type

and volume of work that represents the expected uses of the application. This workload information will be used to replicate and measure the resources required by typical users of the application.

– Establish Hardware environment. Establish the hardware environment, by defining the systems and peripherals requirements and obtaining the necessary equipment.

– List of performance tests. Define the combinations of workload variations and server configurations that will be tested. The goal of these tests is to generate data which represents the resource requirements of the application in the various applicable settings.

– Write scripts. Develop a method of driving the workload (defined in a previous step). Quite often this involves using a workload simulation tool to create scripts which represent end-user activity on the server.

– Build the database. Build the database, or acquire a base of data the application can use and against which, testing can occur.

Test execution and analysis of data Next, it is time to actually run the scripts against the test database on one or more iSeries servers and gather performance statistics. Prior to doing this, create setup and run procedures for consistent, repeatable results. And, design a plan for collecting the necessary performance data.

Analysis of data – Determine CPU/memory/disk requirements. This is the heart of the process—as the

performance data is analyzed to determine the relationship between key application or system variables and system resource requirements. The results will yield information that will allow decisions such as the determination of the CPU, memory, and disk requirements for application workloads. Results of the analyses can sometimes point to further testing. The process can be iterative.

Generation of Sizing Guide Solution Use the IBM eServer Sizing Guide Developer to create the sizing definition/solution.

– Create “Help” documentation. It is important at this point to also make Sizing Guide Solution user-friendly, so be sure to build “Help” and “Comments” text for the Estimator to reference as “results output” from the Sizing Guide Solution.

Validation Verify the results of the Sizing Guide Solution against one or more production servers.

Publication to Web For the Business Partner, the newly created Sizing Guide Solution will need to be distributed to the customer set, via the Web.

Page 6 of 53

Each of these steps will be discussed in detail in the remaining sections of this paper. However, one critical benefit of the Estimator merits highlighting at this point in time in the following note... While the upfront investment in gathering and analyzing the application’s performance data can be large for Business Partners; the Estimator is designed, built, and maintained to minimize the need for changes to the Sizing Guide Solutions as subsequent changes occur to hardware and the iSeries server. The message included in this last sentence cannot be overemphasized. What it means is that follow-on efforts in maintaining the sizing guide for a particular application is dramatically reduced since the “feeds and speeds” of future systems and software will automatically flow into the Estimator. There are only two ongoing considerations for the Business Partner in regard to revisiting the customization effort of the Sizing Guide Solution: If a new release level of the operating system includes significant modification or

enhancement of functions that are key to the application, the Sizing Guide Solution may need to be reexamined to embrace the enhancements offered by the newer OS version.

Also, if the application itself comes under significant modification or enhancement, then the Sizing Guide Solution will obviously need to be tweaked.

Page 7 of 53

2. Preparation Now that the “30,000 foot” view of using this Estimator tool is understood, let’s lower the perspective a bit to get a better “hands on” look at each of the steps.

2.1 Education An extremely important planning task is to become familiar with the Sizing Guide Estimator and the IBM eServer Sizing Guide Developer. The first step is to invoke the IBM Sizing Guide Estimator (found at: www.developer.ibm.com/sizing/sg ). At this Web site, it is easy to become familiar with its general flow and the level of questions for some of the IBM-supplied Sizing Guide Solutions. It is also beneficial to run through both the generalized and detailed tutorials, and to take the benchmark tutorials (which are accessible on-line within the Estimator product). This is also a good time to work with the Sizing Guide Estimator’s sample workloads. The need to understand the sizing product is very important, because, as will be seen in the discussions that follow, the construction of Sizing Guide Descriptions will come from application parameters and other application and environment variations that will be collected. If the uses of that data are not well understood, the resulting Sizing Guide Descriptions may be ineffectual. Next, take a look at the IBM eServer Sizing Guide Developer Users Guide (which is located in the toolkit) since it is important to have at least a general comprehension of the topics in this document—even before fully appreciating the scope of its contents. In other words, a detailed understanding of this document will ultimately be needed, but at this point, a cursory knowledge of its general topics will still be greatly beneficial. [NOTE: Later (after the analysis), reread this document in more detail as part of the ‘Build Sizing Guide Solution’ step.”] This extremely informative paper includes many screen captures and charts to extensively cover the fundamentals of the IBM eServer Sizing Guide Developer. It discusses, at length, the following components: workload, page, calculations, comments, ConstantGroup and Constant, persistent data, partition minimums, system criteria, and bibliography. It also discusses version upgrade logic, debugging considerations, sample question definitions, sample system criteria logic, as well as sample “comments” logic. Beyond reading the Developing Sizing Guide Solutions... paper, it may also be valuable for the Business Partner to visit the iSeries Benchmark Web site (www-1.ibm.com/servers/eserver/iseries/benchmark/cbc/). This web-site has a large amount of information related to preparing for an application sizing experience.

2.2 Questions After having become familiar with the tools just mentioned and feeling comfortable with the application sizing experience, the second task gets into the meat of the project. It will be necessary to decide on the end-user questions required to size the application. Ultimately, these questions will be used to determine such things as how large a processor and how much disk space is required for the application. When defining the questions, one good source of information is the Business Partner’s sales representatives. They are most likely already using some volume-related questions to get a feel for a customer’s processing requirements. They are closest to the “heart” of the customer set. They will also provide information as to what application features the customers usually are

Page 8 of 53

interested in. Knowing which features are most heavily used will enable the proper Sizing Guide Description. Also, by talking with the representatives, two additional benefits can be realized. The first being a general feeling of the level of knowledge the end user will reasonably be able to provide. The questions should be structured in a manner such that the user will either “know the answer right away,” or “can get the answer quickly.” If the question requires the user to do a lengthy investigation or have complete detailed knowledge of the inner workings of the application, it may not be the most ideal question. The second benefit of seeking input from the sales force is the potential for their “buy-in” to the development of the sizing guide. This is important since they will be one of the major users of the sizing information. Most often, three types of information will be needed: Processing. The first type of question has to do with processing (i.e., CPU)

requirements. The business transaction volumes will be important for sizing CPU requirements. In some situations, this can be reduced to merely asking for “number of users.” In other cases, not all users put the same load on the server, so it may be necessary to ask for the “total number of business transactions processed.” Other questions include: How many orders are entered in a typical 8-hour day? What is the average number of items per order? How many loans are processed? How many hits-per-second are typical for a Web application?

Memory. The second type of question has to do with memory requirements. Often, applications are designed so that separate jobs and/or storage are used for each user. These questions will take the form of: How many order entry users are there? How many accounts payable users are there? How many concurrent users/transactions are there? ...etc.

Disk. The third type of question will have to do with DASD requirements. How many documents and records are there? What is their average size? How many SKUs are maintained? How many customer accounts? How long is historical data kept on-line?

[NOTE: Each of these three categories for information gathering may benefit from additional questions to help determine the complexity or “largeness” of that item. Questions such as “Are users accessing the application through the Web or a native interface?” might affect the amount of CPU processing. Another question, “What proportion of application users are doing Web browsing versus Web purchasing?” may affect disk activity for updates as compared to the total memory demand for caching frequently referenced Web pages.] Some care must be taken to organize the questions, phrasing them in a logical and comprehensible manner so the end user does not become overwhelmed. Also, keep in mind that code can be provided in the Sizing Guide Solution to offer some reasonable defaults in the event that not all of the information is readily known by the user. Customers that are adding new applications may not have many details regarding the use of the application and, therefore, must rely on the Business Partner to guide them as to common/average usage numbers. Customers that are replacing or upgrading applications may have many more details and can input or override defaults provided by the Business Partner. In some cases, the customer will only have “impressions” of how the server will be used. This will lead to using a large number of “fuzzy” terms to describe the work. Terms such as “light,” “heavy,” “moderate,” “casual,” and “complex” would be used extensively as answers to the sizing questions. This can make the effort of quantifying the default values more critical to achieving reasonable sizing results. In the experience of IBM, it is best for the Business Partner to include default values for essentially every sizing question that is asked — at least in all those cases where it is remotely possible to “guesstimate.”

Page 9 of 53

An additional point should be made about the design of the questions that involves the importance of limiting the total number of questions presented to the user. In general, a target of six to ten questions is most reasonable, if possible. One reason for this is logistical — about six to ten questions can visually appear on a Web browser screen at a time. This gives the impression to the end user that this a quick and easy process to be undertaken. However, keeping down the total number of questions does not mean that broad volumes of sizing information cannot be collected. It simply means that it is important to ensure that the questions are well-structured and well-selected. For example, the parameters for RAID and for DBCS are on all workloads, but most customers set them at the tool preference level and do not consider them, even though they are on each individual workload. Let’s belabor the topic of “quick and easy” for a moment. There are multiple uses of the Estimator. In some cases, the audience will include users who are experienced and very knowledgeable of the details related to the application being sized, while in other cases, the user will be more similar to a “lightweight” customer or prospect who is simply considering the application and is not as knowledgeable about the day-to-day workings of the application. Some Business Partners may want to write their Sizing Guide Descriptions to serve the less knowledgeable user as well as the somewhat expert sales and user staff. So, while it is important, as mentioned in the last paragraph, to keep the questions short, sweet, and simple. It will be incumbent on the Business Partner to provide “Help” text behind the questions that describe, with much more detail, how the user would pick one answer over another. Some Business Partners have found that even the sales force often is not knowledgeable about applications well enough to answer very specific questions. So, having more general answers, augmented by “Help” support that describes at length the conditions constituting the optimal answer, will usually assist every level of user in providing more thorough and accurate sizing information to the Estimator. Another point to consider when designing questions for the benchmarking process is whether or not there will be any predictable spikes in application usage. Known “resource hog” functions, however infrequently executed, will cause severe server degradation. Therefore, some means of quantifying this potentiality should be designed into the characterization process and included in the resource estimation formulas, and perhaps even in the question/information gathering process.

2.3 Assumptions In any performance measurement exercise, it is important that the application has been adequately tested and debugged. It will be difficult to get through the performance tests in a consistent manner and gather the necessary sizing data if the application is still undergoing changes and fixes. It is not a good use of resources to spend valuable performance measurement time debugging applications or databases that are still under development.

Page 10 of 53

3. Define the test plan Now that the Application Provider has been educated and has decided which sizing questions to present to its customer, it is time to define the measurements and the testing processes that will be followed to size the application.

3.1 Determine application workload First of all, an understanding must be formed of the various ways the application is utilized. For example, the same Shipping and Receiving package might be used differently in a manufacturing setting than in a distribution/warehouse setting. The functions executed, the volume of transactions, and the number of line items per transaction could all be different. Even the demand put on the application by different departments within the same organization and from individual users can change the processing that is required and thus the definition of the workload. Where the application is being utilized in larger enterprises, it is not uncommon to see high numbers of users and a wide range of purposes between the users. The larger the number of users and the greater the range of usages, often, the more important it is to distinguish between the usages in estimating the resource needs. Where the application is being installed in smaller businesses, it is not uncommon to see less users and fewer types or groups of users. With smaller numbers of users and narrower ranges of uses, often a more simple approach to sizing can be invoked. Consider this a bit like “sizing to average usage” rather than “sizing to each distinct usage.” Just as a side note, it may be possible to notice that, during the development of the application workload(s), it becomes necessary to tweak the questions that were created for the sizing effort. In other words, as one gets more intimately into the functions and user types for the application, it may become apparent that there is a need for more questions or that the wording of the questions should be more precise to capture how the application is used by a particular group of users. The Business Partner should also be aware of the importance of balancing the requirements for detail and accuracy with the need for simplicity and ease of completing the effort to build a Workload Solution. For several IBM-developed Workload Solutions, it was valuable to get a rough sizing available to (in the hands of) the end-user organization rather than take additional time trying to understand and build a more detailed and precise Workload Solution. For these situations, a multistage approach allowed for an initial (simplistic) Workload Solution to be created. Then, based on feedback from the users of the Sizing Guide Estimator Workload Solution, updates were incorporated into later versions, either to add more detail and precision or to allow for new usages. The Business Partner should consider the pros and cons of a multistage deployment of the Workload Solution. As a Web-based tool, there is minimal effort in the actual replacement of older versions of the solution with newer ones. The process for defining the application workload is as follows: Review the potential customer set for the application(s): As mentioned earlier, the sales team will be able to provide valuable input here. Not all customers will use the same set of functions within an application. For instance, in an insurance environment, the application may be utilized differently depending on the type of agency: home/auto insurance versus life

Page 11 of 53

insurance. For a healthcare application, there may be small, medium, and large HMOs with 100,000, or 1,000,000, or even 5,000,000 members, respectively. The database required to represent the HMOs would, therefore, vary. It is possible that there are differences in the processing requirements, as well, based on HMO size. The goal is: (1) to capture the important distinctions where application functions or database volumes will be significantly different, but at the same time (2) to keep the variables for the sizing guide reasonable. Not every variation will be measured. Between one and three categories is typical. Identify the user groups (within each customer set): For example, a life insurance agency may have two groups of users: claims management and underwriters. An on-line catalog application may include order management and manufacturing/fulfillment components. In banking, there could be on-line banking, traditional teller transactions, and perhaps a sales component. These workloads may need to be measured separately so that each segment can be sized at different levels in the Sizing Guide Estimator. Identify the business transactions (within each user group): List the kinds of business transactions a typical user performs and the frequency that each is executed. The goal is to derive a short list of core functions. This is not a functional test where complete coverage is important. A transaction should only be included if it occurs very frequently, or if it occurs infrequently but consumes a significant amount of system resource when it does occurs. An example matrix that takes some of these points into consideration is shown next. Please note that this matrix has been put together to demonstrate just one example of a way that a Sizing Guide Description might be framed. Certainly not every application has multiple user groups or even multiple business transactions. But this is the thought process that should be helpful in pulling together a thorough and complete Sizing Guide Description.

Page 12 of 53

Actual business transactional volumes Cus- tomer

set User

Group Business

transactions Hourly peak

Number of active users

Batch processing required?

DB description

Order entry 30 Order update 20

Small —

100,000 acc’ts

Order Mgmt

Order inquiry 25 2

Order post

15% historical data

3% hold status

Order entry 300 Order update 200 Order

Mgmt Order inquiry 250

20

Backorder release 50

Medium —

1,000,000 acc’ts

Inventory Mgmt Ship confirm 300

4

Order post Receiving post

Shipping post

20% historical data

8% hold status 5% backorder status

Order entry 3,000 Order update 2,000 Order inquiry 2,500 Order

Mgmt Order from

history 300

200

Backorder release 500

Ship confirm 3,000

Large —

5,000,000 acc’ts Inventory

Mgmt Ship tracking 3,000

25

Order post Receiving post

Shipping post

Backorder post

Track/ship post

35% historical data

15% hold status

5% backorder status

6% tracking data

A script could be created for each of the User Groups. In the example above, two scripts would be required, one for order management and one for inventory management. The number and complexity of the scripts should be kept to a minimum. To get a feel for relative priority, it may be useful to look back at the initial list of questions. For each of the business transaction questions, there is a script(s) to measure the resources required. This Sizing Guide Description will help to drive the list of measurement runs or tests that will be required to build the Workload Solution. It should be noted in the above chart that the business transaction volumes are given as a peak hourly rate. When capturing data for use in sizing a system, it is important to measure peak workload. The intent is to size the system so that it can adequately handle the peak. If the server performs well during the peak, then it will perform well all of the time.

3.2 Establish the hardware environment The accuracy of the Sizing Guide Estimator is usually better when the formulas are based on the results of multiple tests on each of several different hardware configurations. Of course, there are tradeoffs to be considered by the Business Partner in regard to the amount of effort versus cost versus accuracy. Though some who have investments in on-site iSeries servers will prefer to do their sizing efforts inhouse, there are others—those with no inhouse systems or without the luxury of testing multiple configurations in a dedicated mode—who will opt to travel to one of the two iSeries Benchmark Centers (Rochester, Minnesota or Montpellier, France) to

Page 13 of 53

achieve a more ideal sizing environment. Another option for gaining access to iSeries servers are the IBM Solution Partner Centers. In selecting the hardware against which the workload will be run, the following should be considered: 1. Two or more systems are recommended. Three or four may be ideal if a broad range of

systems is to be modeled. 2. Choose the latest iSeries server hardware models that are available. 3. If a specific configuration is targeted for the application, or a configuration is believed to be

popular with potential customers, be sure to include this as one of the configurations. 4. Choose configurations that span the range of possible systems for this application. A

Business Partner with an application targeted for small and medium businesses may want to benchmark on the smallest available uni-processor (which will have a lower processor rating). Or, it may be more advantageous to choose a midrange processor (a 2-way or 4-way) that has a higher processor rating than the uni-processor. Of course, other applications will need to be tested on high-end processors (e.g., near max n-way machines). As the reader may know, there can be internal hardware differences between the various uni-processor models as well as differences between uni-processors and n-way processor models. For example, there may be differences in the way memory is packaged, differences in internal bus speeds and differences in the size of processor cache. This is one reason it can be advantageous to run tests on a variety of configurations.

5. In every case, choose hardware that exemplifies a balanced configuration. When the tests are executed, no resources should be excessively underutilized or overutilized. For example, if the disk arms are only 1% busy, it will be difficult to project their behavior under loaded conditions (more typical of end-user environments). Conversely, if the server does not have enough memory and is, therefore, experiencing a high rate of page faults, this not only affects response times (for increased I/O), but also puts a strain on other resources, e.g., CPU, such that the CPU requirements and disk access time will be inflated. The result may be that it is necessary to change the configuration from test to test until a good balance is achieved. This will be illustrated further under the discussion of “Test Execution.”

6. A final note regarding the selection of hardware for benchmarks specifically concerns memory and disk. Be sure the configuration(s) on which the testing is being executed has enough memory and disk resources to support any “what-if” testing that is desired. It is best to plan for more disk and memory than is actually anticipated.

3.3 List of Performance Tests At some point during the planning and preparation, it will be important to run some exploratory tests to get a feel for how the application performs under the defined workloads and on the planned hardware configurations. This is referred to as “exploratory testing” because it is important in determining which tests and what hardware will be most beneficial to yield the most information. The purpose of exploratory tests is not to acquire a precise good/bad or pass/fail result. Instead, an exploratory test is used to gather general behavior characteristics, such as: Whether the CPU increases linearly as do the number of users Verifying whether an application will spread out to take advantage of all processors or

will only effectively use one, two, or ‘n’ processors on an n-way processor An appropriate starting point for the number of disk drives or amount of memory required

by the application.

Page 14 of 53

Exploratory tests are not yet the strictly run and repeatable measurements that will be used for analysis and sizing. Instead, they provide general information for defining the conditions for the formal measurements and help determine upon which servers the measurements should be made. After some exploratory testing has been done, the list of questions for the end user has been established, the workload(s) that will best represent the users has been defined, and the hardware upon which the tests will be run has been determined; the next logical task in preparation will be to determine the list of tests that will be run and analyzed to provide the data that is the basis for creating the Workload Solution. It all comes down to defining formulas for just a few metrics: NICPW – the non-interactive CPU required by the application processing workload ICPW – the interactive CPU required by the workload CIW – the compute-intensive CPU required by the workload MEM – the memory required by the workload STORAGE – the amount of disk storage required by the workload ARMS – the number of disk arms needed to adequately handle the I/O for the workload The document “Example Sizing Guide Documentation” at http://www.developer.ibm.com/welcome/eserver/e3/CSFServlet?mvcid=main&packageid=3000 explains more about how to actually define these variables within the Business Partner’s customized Workload Solution and how they are used by the Sizing Guide Estimator. The performance tests fall into one of three categories. The first type of test is designed to provide the data needed to derive formulas for NICPW, ICPW, and CIW—the three metrics having to do with CPU. The second type of test involves memory sizing and will allow for the definition of the metric MEM. The third type of test is intended to refine the disk requirements of the application, i.e., to define the STORAGE and ARMS metrics. For the first type of test, refer to the matrix below. Notice that the processor varies from test to test for each of the workloads. The purpose of these tests is to find the CPU requirements of the various pieces of the workload. Testing the same workload on multiple sizes of servers will help determine how well the application scales with respect to the IBM metrics of CPW and CIW. [NOTE: CPW and CIW are two ways of classifying iSeries CPU requirements. They will be explained in more detail later in the analysis part of this paper.] Memory and Disk requirements For these tests, assume memory and DASD are not a limiting factor. There will need to be some specialized memory sizing tests and, there may be the need to have some variations to test out different numbers of disk arms. See section 5.2 “Determine Memory Requirements” and section 5.3 “To Size Disk Requirements” for more about test cases and methods for understanding memory and disk requirements.

Page 15 of 53

Customer Set

User Group

Hardware configuration

Number of users

810 1-way 50 100 150 810 2-way 100 200 300

Small — 100,000

accounts

Order Management

825 6-way 500 1,000 1,500 810 1-way 50 100 150 810 2-way 100 200 300 Order

Management 825 6-way 500 1,000 1,500 810 1-way 50 100 150 810 2-way 100 200 300

Medium— 1,000,000 accounts Inventory

Management 825 6-way 500 1,000 1,500 810 1-way 50 100 150 810 2-way 100 200 300 Order

Management 825 6-way 500 1,000 1,500 810 1-way 50 100 150 810 2-way 100 200 300

Large — 5,000,000 accounts Inventory

Management 825 6-way 500 1,000 1,500

This seems a daunting list, but it is not imperative to measure all workloads on every hardware configuration. One approach to reduce this matrix to a reasonable subset of performance tests is: 1. Start simple and then expand the understanding of the process through additional testing.

For example, begin with one customer set and user group (e.g., a small customer and order management as the user group). Use the simplest configuration, in this case, the uni-processor with the fewest disk drives and the smallest amount of memory—an iSeries Model 810. It will be beneficial to run the full range of numbers of users; 50, 100, and 150; on this configuration and with this workload.

2. Next, test the other customer sets with the same user group (e.g., a medium-sized enterprise customer set and order management as the user group, and a large customer set along with order management as the user group).

3. Now, expand the results by varying the hardware configuration, keeping the customer set and user group constant (e.g., a medium-sized customer set and order management as the user group, along with an iSeries Model 810 2-way and an iSeries Model 825 6-way).

4. Similarly, select additional tests for the other customer set and user group combinations. 5. It is also important to select a couple of the largest workload points to validate the formulas

derived from the analyses (e.g., a large customer set and order management and the maximum number of users, as well as a large customer set with inventory management as the user group and the maximum number of users).

Thus, one possible set of tests to be measured is demonstrated in the chart that follows. Notice that there are points that will NOT be measured. Instead, these points will be estimated by the formulas within the Workload Solution.

Page 16 of 53

Customer

set User

group Hardware

configuration Number of users

810 1-way 50 100 150 810 2-way 100 200 300

Small — 100,000

accounts Order mgmt

825 6-way 500 1,000 1,500 810 1-way 50 100 150 810 2-way estimate estimate estimate Order mgmt 825 6-way estimate estimate estimate 810 1-way 50 100 150 810 2-way 100 200 300

Medium— 1,000,000 accounts Inventory

mgmt 825 6-way 500 1,000 1,500 810 1-way 50 100 150 810 2-way estimate estimate estimate Order mgmt 825 6-way estimate estimate check 810 1-way 50 100 150 810 2-way estimate estimate estimate

Large — 5,000,000 accounts Inventory

mgmt 825 6-way estimate estimate check

We should make a note here that the number of users can sometimes be increased without starting a completely new test run. For example, the Small Company/Order Management workload at 50, 100, and 150 users could be executed as one test, starting with ten users and adding new users in fixed increments. The performance data can be analyzed at the various steps on the curve so that several sets of data are gathered with nearly the same effort as a single test. In addition to the tests suggested by this matrix, the application provider will need to include specific tests to size memory and disk drive requirements. These two topics are addressed in their respective sections of the paper. [NOTE: It has already been mentioned in this paper, but this is a good point to remind the reader that the good old 80/20 rule applies (80% of the result will come from the first 20% of the effort). Most of the information will be gleaned from just a couple of runs. After that, each successive test adds accuracy, but not on the same scale as the results of the initial testing. However, if there is the need to determine, with extreme precision, the resource needs of a particular application under specific loads, additional runs and comparisons of the results will be required.]

3.4 Write the scripts In order to gather multiple sets of consistent data to be used for sizing purposes, a repeatable method of putting work on the system is required. Several automated testing tools can be used (for example, LoadRunner® from Mercury, Inc.). Alternatively, some development shops have written their own load generators or have purchased other tools for functional or load testing that can also be used for sizing measurements. Some workload generators use APIs to send transactions to an iSeries server. For example, a program on one iSeries server can submit jobs which send MQ transactions to the iSeries server under test. And, finally, the iSeries Benchmark Center can be a source for developing scripts for sizing purposes. When developing scripts or a workload generator, the following should be considered:

Page 17 of 53

Pace the transactions – Build in “think” time to the scripts so that transactions arrive at the server in a realistic manner.

Stagger the transactions – Incorporate a staggered start and some variability into the scripts so users are not all executing the same functions simultaneously—possibly creating unrealistic spikes in resource demands.

Wait to begin analysis – Allow time for the script to reach a steady state before analyzing the performance data from the test.

Vary the data – Use variable data to drive the scripts so that objects do not inadvertently become resident in memory.

Refer back to the chart shown on page 14 of this paper. The scripts could be written along the lines as illustrated — either “User Groups” or “Business transactions” whichever seems to more logically fit the various needs or the application and the customer set. Using the divisions structured as shown, the application provider might be able to create fewer numbers of scripts and simply use the scripts in differing ways. In other words, even though the chart may illustrate two different types of transactions (e.g., “shipping confirmations” and “shipping tracking”), it may be possible to use the same script for both (since it is possible that each transaction type references the same files, the same number of times, in a similarly random manner). In essence, if “shipping confirmations” and “shipping tracking” are similarly loaded transactions, the Sizing Guide Estimator does not really care if it is delivering information related to a shipping confirmation versus a shipping status. The application would “see” both transaction types as being essentially identical in the load they put on the system.

3.5 Build the database In most cases, the workload will need to run against data—so a database will be required. In the ideal case, this database will be realistic in size and structure to the environment that is being replicated. In the example discussed on page 14, for instance three different sizes of databases would be required: one with 100,000 accounts; one with 1,000,000 accounts; and one with 5,000,000 accounts. There is no simple way to generate data of this nature. Typically, it involves using data build tools written by the application provider, lots of disk space, and a significant amount of time to generate large-scale databases. Sometimes, a “seed” database is used as a starting point and then is replicated many times, varying the key components of the data. In other instances, an application provider can use the database of an actual customer. The advantages to this approach is that it requires no extra effort to create and it is realistic, having come from an actual production site. However, a production level database can have inconsistent data that can make scripting more challenging. For example, if a script refers to a customer account that has a “delinquent” status, the application may post an error message that the script has not been programmed to accept. The question that must be asked at this point is... Does size matter in database performance? The answer depends on the application. If, for example, the application does full table scans of the data table, then having a realistic database will give a much better picture of performance. In other instances, when the application always accesses data via an index, it may not be as important. The person responsible for defining the test plan needs to decide what is necessary for accuracy and what is practical. Keeping in mind that the Sizing Guide Estimator sizing tool is accurate to +/- 10%, the effort of creating or acquiring several different sizes of large databases may not be worthwhile.

Page 18 of 53

It should also be pointed out that some applications do not depend on a DB2® UDB-like (relational) database, but instead are more dependent on interacting with information found on Web pages, or within Integrated File System (IFS) flat files. Acquiring or building data for this type of load testing may require different considerations than are needed for relational databases. Data Reset: The database may need to be reset between tests if the performance tests alter the database and if the modified content of the database will affect the performance of the application. Resetting the database so that every test has the same starting point is, again, the ideal. This ensures repeatability and consistency between tests. It also allows results to be compared, knowing that there are no differences between tests other than the ones dictated by the test plan. The most common and simplest way to reset data is to restore the database from a savefile or tape. If a full restore is not deemed necessary, sometimes files can be cleared or some other aspect of the execution environment can be changed to allow the iterative tests to be run. Alternatively, an altered range of input values can be provided to the scripts so that different database records will be updated with each test execution. These latter methods might be acceptable if it is believed that the performance is comparable with or without a full restore between tests, or if a full restore is not practical due to time constraints. Final preparation comments: The various components of the test plan (workload definition, hardware requirements, matrix of tests, scripts, and database) are all closely related. The process for defining these can, in fact, be cyclical. As scripts are created, it may be necessary to tweak the workload definition and even to go back to refine the list of questions discussed in the previous section. As the database is defined, it will be possible that the scripts are impacted or that the list of tests included in the test plan should be refined. Throughout the process of planning and preparation, and even during execution, there may need to be revisions and contingencies.

Page 19 of 53

4. Test Execution Once the test plan with its various components has been fully defined, the next step is to set up the environment and run the tests. Before executing the list of formal tests that have been identified, it will be important to run some preliminary baseline tests. The purpose of these types of tests is to allow for: (1) tuning of the system, (2) a shake down of the test and run procedures, (3) verification that the workload measured during the test meets the test specifications, and (4) determining the right configuration for a balanced system. The baseline test is the first place that all the pieces of the test environment come together. The defined workload is executed using the test scripts on the prescribed hardware and software environment. The run and test procedures should be followed, i.e., performance data is collected, initialization, setup, and reset procedures are followed. During this test, the Business Partner will have the opportunity to see the system in action. Application and system settings can be reviewed, and performance can be monitored real-time to ensure that the system is performing in an optimal fashion. Following the test, it will be necessary to verify that the desired transactions occurred on the server. Load generation tools will usually provide reporting mechanisms. On the application side, it may be possible to check logs or the sizes of files to verify that the desired throughput was achieved. If the numbers are not satisfactory, it may be necessary to adjust “think” or “delay” times in the workload generator and then run the baseline test again. Similarly, it may be important to make some adjustments to the server or application and then rerun the test. The goal is to run the tests on balanced systems where resources are not under or overutilized. Basic resource utilization guidelines: So, what is meant by a “balanced system?" How is it possible to tell if the server is performing optimally? Briefly, the following information is an attempt to give the reader some guidance. The server under test should be configured with sufficient disk arms, I/O processors, and memory so that these resources do not become a bottleneck. This was mentioned in the earlier discussion on the selection of hardware. In the planning phase, the application provider must strive to select configurations that are balanced. Now, in the test execution phase, it is necessary to monitor the server in order to verify that resources are performing “within guidelines.” The baseline exploratory test is an important initial step. This is the first test with the prescribed workload running on the proposed test configuration and will be a prime opportunity to determine whether a balanced configuration has been selected—one where the various resources are performing within some basic utilization guidelines. In subsequent tests, as the configuration or the workload is varied, it will be important to monitor the server to see that guideline boundaries are not breached. It will be impossible to get an accurate measurement of the resources required by the application if one or more of the system resources is overutilized. During the execution of a test, if a resource is overworked, then it will be necessary either to reduce the workload and run the test again, or to somehow alter the configuration (add memory, disk drives, or a communications IOP, for example) and then run the test again.

Page 20 of 53

With this in mind, here are some basic resource utilization guidelines. Disk arm utilization ideally should be 20-25% busy or less, IOP utilizations should also be 25% busy or less, and CPU utilization should not be more than 70% for high priority work. As an aside, the reason for keeping utilization figures in these ranges is that, in regard to general server performance, it is important to keep the impact of slower resources (such as disk drives and IOPs) in balance with higher speed resources (such as memory and CPU). Utilization not only reflects the amount of work a particular resource is doing, it may also affect how long a new request for work (by that resource) has to wait due to the resource executing other work. For resources such as disk and IOPs, it has been found that utilizations in the ranges noted here represent a reasonable level—given the relative speeds of the processors. There is no specific guideline for “memory pool faulting rate.” Generally, though, it is important to have enough memory so that faulting does not make up a significant portion of system response time—but not so much that a majority of the data referenced in the test measurements becomes resident in memory (which would result in little or no disk paging activity). Altering the memory available for the application is easily done. That is, it does not require modifications to the hardware configuration as would be the case when changing CPU models or the number of disk drives. [NOTE: Look at “Appendix B: Page Faults” where a method is provided for determining the impact of faulting on response time.] Next, let’s discuss the test environment. What is important is keeping the environment the same for every test. Dedicated server: To make accurate and repeatable measurements, it will be necessary to run on a dedicated server where the only jobs executing are a part of the defined workload. It may be necessary to vary off communication lines, disable user profiles, or otherwise control access. It is important that the only variances between the defined tests are the ones prescribed in the test plan. Extraneous users and applications running during a performance test can cause the response times of the application under test to vary and, more importantly, will make it difficult to isolate the resources required by the test application. Even though it may be more like the “real world” to have other applications and users on the server, for sizing measurements, the goal is to characterize the resource requirements of the application and not to provide a realistic stress test of an actual production system. Run procedures: The environment should remain consistent between performance runs, so a checklist of run procedures should be created. This checklist consists of the tasks that are accomplished before, during, and after each test to help ensure a comparable starting point is executed in a consistent fashion. These tasks may pertain to the iSeries server, to other hardware involved in the test, and to client workstations used to drive the workload. For example, before every test: The database may need to be reset. Memory pools can be cleared using the iSeries command, CLRPOOL. Performance monitoring tools should be turned on with the desired attributes. System tuning parameters (memory pool sizes, activity levels, and system values)

should be reviewed and adjusted as needed. The library list and any environment or application variables should be set. The server should be checked to ensure no unaccounted for jobs are running. (A more complete example of a run procedures checklist is found in Appendix A.)

Page 21 of 53

Performance data collection: One of the key tasks on the run procedures checklist will be the monitoring and collection of performance data. After all, the ultimate goal for conducting the tests is to come away with relevant information that can be used to establish the formulas for the Sizing Guide Solution. In addition, it is also necessary to monitor performance during the test to ensure that the test is valid and that none of the system resources are over or underutilized. So, to reiterate, there are two requirements for performance monitoring: real-time monitoring and data collection for use in building the Sizing Guide Solution. Management Central Monitors, a part of iSeries Navigator, can be used to monitor many of the application processing workload metrics, such as CPU utilization, disk arm utilization, and memory pool faulting. Management Central Monitors provides a means of real-time monitoring, i.e., graphs are displayed depicting utilization of various resources while the test is taking place. After the test, the data that has been collected can be viewed using the Graph History function. From Graph History, the data can be exported to spreadsheets for further analysis or presentation. In addition to the Management Central Monitors, Collection Services can also be used to collect a more complete set of performance data. Collection Services stores the data it gathers in the same Management Collection Object as Management Central Monitors. From the Collection Services interface, file members can be created in the Performance Data library, usually named QPFRDATA or QMPGDATA, and the traditional Performance Tools System and Component reports can be run against the collection to give in-depth resource utilization statistics. (See Appendix C for information on how to start up Collection Services. Another excellent reference is the Redbook “Managing OS/400 with Operations Navigator V5R1 Volume 5: Performance Management,” SG24-6565-00.) Here is a summary, then, of recommended performance data collection on the iSeries server. “Graph History” (discussed at length in “Appendix C: Collection Services”) – After the

fact, to graph the data that was gathered, and pull that history into spreadsheets for analysis.

Collection Services – Gathers more comprehensive system performance data. Performance Tools System, Component, and other reports can be created to provide resource utilization metrics. See: publib.boulder.ibm.com/iseries/v5r2/ic2924/books/c4153401.pdf for more information on the Performance Tools.

Management Central Monitors (which is found under “iSeries Navigator,” also formerly known as “Operations Navigator”) – Monitors the systems during the tests. This is illustrated in the screen capture shown below, where a group of resource utilization and performance-related charts are presented.

Page 22 of 53

In summary, the test execution phase is the point within the process where a great deal of performance data has been collected in an orderly and documented fashion. Because of the preparation work prior to the testing phase and because of the scrutiny that has surely been employed during the testing, the application provider now is more than adequately prepared to begin the analysis portion of this process—which is what will be discussed next.

Page 23 of 53

5. Analysis While the benchmark analysis process can get complicated, at its heart, it is very simple. There is the need to size three distinct resources: CPU, DASD, and memory. Taken one at a time, the analysis of the performance of each of these components is easier to accomplish.

5.1 Determine CPU Requirements In determining CPU requirements, the goal is to find the relationship between the number of transactions processed (or the number of active users) and the CPU required. In order to do this, the same workload should be executed on more than one configuration and CPU utilization measured on each. By analyzing the CPU statistics for these various tests, it will be possible to determine the necessary formulas to express CPU requirements. The Sizing Guide Estimator variables are as follows: NICPW: non-interactive CPU (expressed as CPW) required by the application

processing workload ICPW: interactive CPU (expressed as CPW) required by the workload CIW: compute-intensive CPU (expressed as CIW) required by the workload As can be noted from the above, the definition of the variables for CPU power on the iSeries server is represented in terms of CPW ratings and/or CIW ratings. Before going any further, though, it is necessary to determine whether the application workload more closely scales with the CPW CPU rating or the CIW CPU rating. If CPW is a closer fit, then it is possible to further break this down into non-interactive CPW and interactive CPW requirements.

CPW versus CIW: Commercial Processing Workload (CPW) is used to compare the processing capacity of various iSeries models. It measures the processing capacity of a server within a commercial arena. In these environments, a majority of the CPU is spent in the operating system and database layers, while relatively little is spent in the application layer. Not all workloads on the iSeries platform fit this pattern. So, in OS/400 V5R1, a new metric was defined—CIW (Compute Intensive Workload). Unlike commercial applications, compute-intense applications spend the majority of their processing time in the application layers. They may scale differently than traditional commercial workloads since they are typically more sensitive to megahertz speeds rather than level 2 cache, for example. The CIW metric was created to more closely represent the scaling characteristics of these compute-intensive workloads.

In the example used earlier in this document, the order management / small account customer workload could be executed on the iSeries Model 810 uni-processor and 2-way processor and the iSeries Model 825 6-way processor. The data can then be plotted for best analysis of whether the application is more closely aligns with CPW or CIW, or whether it falls somewhere in-between. Here is how to determine the points for graphing: In this example, the system under test is an 810-2465 with a CPW rating of 750 and a CIW rating of 250. Let’s look at the test consisting of 100 users processing a total of 1,000 orders per hour. Using Management Central Monitors, the average CPU utilization during the test is 56%, with the CPU utilization of the interactive feature equal to 0%. CPU required for the workload is calculated at 420 CPW (only 56% of 750 available CPW was utilized) or 140 CIW (56% of 250). The CPW required per user is 4.2 (420 CPW divided by 100 users). The CPW required per

Page 24 of 53

order processed can be expressed as .42 * x where “x” is the number of orders processed per hour. Similarly, the CIW required per user is 1.4 and the CIW required per order is .14. The same calculations are performed with the numbers from the 50-user test and 150-user test. The resulting three points are graphed along with similar points taken from the other two configurations, systems with different CPW and CIW ratings. Plotting these points should help in determining how closely the application scales to CPW or CIW. It is even possible that the processor requirement of the application is best expressed as a function of both CIW and CPW.

CPW Requirem ents

02000400060008000

1000012000140001600018000

0 2000 4000 6000CPW Consumed

Ord

ers

Proc

esse

d pe

r ho

ur

810 2- way810 1- way825 6- way

Page 25 of 53

CIW Requirements

02000400060008000

1000012000140001600018000

0 500 1000 1500 2000 2500CIW Consumed

Ord

ers

Pro

cess

ed

per h

our

810 2-way810 1-way825 6-way

The following is the raw data behind the CPW and CIW graphs.

System CPW Rat ing CI W Rat ing CPU% CPW Used CI W Used Orders/ Hr # Users CPW / Order CI W / Order8 1 0 - 2 4 6 5 750 250 30% 225 75 500 50 0.450 0.1501 - w ay 750 250 56% 420 140 1000 100 0.420 0.140

750 250 82% 615 205 1450 150 0.424 0.1418 1 0 - 2 4 6 9 2700 950 16% 432 152 1000 100 0.432 0.1522 - w ay 2700 950 22% 594 209 1480 150 0.401 0.141

2700 950 78% 2106 741 5023 500 0.419 0.1488 2 5 - 2 4 7 3 6600 2890 8% 528 231.2 1530 150 0.345 0.1516 - w ay 6600 2890 26% 1716 751.4 5100 500 0.336 0.147

6600 2890 48% 3168 1387.2 10200 1000 0.311 0.1366600 2890 77% 5082 2225.3 15200 1500 0.334 0.146

From the above graphs, it is possible to see that the slopes of the three lines match up more closely on the CIW graph. The conclusion drawn is that while the graphs are similar, the application scales more closely to CIW than to CPW. Thus, the two CPW metrics would be set to “0” and a formula would be derived to quantify the CIW requirements. Looking a little more closely at the CIW/order column in this worksheet, it is necessary to decide which test results best represent the CPU requirements per transaction of this application. This is where the “art,” rather than the science, of performance analysis comes into play. The thought process might go like this: More weight should be put on test results where the CPU is moderately utilized, i.e., neither high nor low. It would be better to have numbers from larger rather than smaller configurations since there is generally more confidence scaling downward rather than upward. Scaling up can tend to magnify any measurement error. So assuming that 0.148 and 0.146 are chosen as the most interesting points on the chart, the resulting formula could be:

Page 26 of 53

CIW = .147 * order_rate NICPW = 0 ICPW = 0 ...where order_rate is expressed in number of orders processed per hour.

The previous example involved no Interactive CPU processing. For 5250 type or traditional “green screen” applications, the CPU utilization - Interactive Feature can represent a significant part of the CPU resource. Management Central Monitors can be used to see CPU utilization for Interactive Feature. In the figure below, we see a CPU utilization ranging from about 35 % to 0%. This is on the graph labeled CPU Utilization (Average). The CPU Utilization (Interactive Feature), as shown in the graph, is at or near 0%. CPU for Interactive would be analyzed similarly to the CPU analysis shown in the previous example, instead using the “CPU utilization - Interactive Feature” monitor with the system CPU ratings (CPW and CIW). In this way, the formula for ICPW would be derived. Then, to calculate the non-interactive CPU, subtract the “CPU Utilization - Interactive Feature” from the total system CPU Utilization. (See the screen capture below.) The difference is the CPU that is used by non-interactive work. This utilization number can then be used as was the total system CPU utilization in the previous example to produce a formula for either CIW or CPW (NICPW).

Page 27 of 53

5.2 Determine Memory Requirements The goal in defining memory requirements is to find a formula for the metric MEM. Memory is typically sized on a per-session or per-user basis. However, when sizing memory for a Web application, it may be appropriate to size on a per-thread, per-server, or sometimes even a per transaction basis. There are a couple of fundamental approaches to the problem of determining memory requirements. The simplest method is to use a rule-of-thumb number that is based on experience. When memory is not a significant cost factor to merit the time and resource required for more extensive memory sizing tests, this approach may be adequate. The other approach is based on running tests and gathering data. Here is the general approach. For a fixed number of users (or a fixed transaction rate for a Web-based application), start with a memory pool that is known to be too small. Faulting will be high and response times will most likely be impacted. (See the sample chart below.)

Then, gradually add more memory to the pool, observing the faulting rate each time memory is added. When faulting no longer decreases, the adequate amount of memory required by the application workload has been observed. It is recommended that this step be executed a

Page 28 of 53

second time with a different number of users (or transaction rate) to see how it influences the behavior of the page faulting. Now, having two sets of numbers, the standard formula for the slope of a line (y = mx + b) can be used to solve two equations for two unknowns. In this formula, the known variables are...

- “y” which indicates the size of the pool when there is just enough memory, and - “x” which indicates the number of users.

The unknown variables are... - “m” which represents the per-user memory requirement, and - “b” which represents the amount of memory used by the application but shared across multiple users.

Again, be aware that if Web users will represent a significant part of the user mix, “m” may need to represent a per-thread, per-server, or per-transaction formula, as mentioned earlier in this paper.

Page 29 of 53

Example of a memory-sizing exercise In this example, three tests are run on three volumes of active users (150, 100, and 50) with the objective of determining the memory required at each of these three volume levels. From these three data points, it is possible to determine the memory sizing formula. Test 1: 150 users 150 simulated users are running in a memory pool configured initially with 225 megabytes of memory. After this has run long enough for performance to stabilize, it is observed that faulting is about 125 faults per second. 50 megabytes is added to the pool, once again the system is allowed to stabilize and then it is observed that faulting has dropped to less than 50 faults per second. This process is repeated until adding memory does not reduce faulting. The graph indicates that this happens at 325 megabytes. So, the first data point for this exercise is that 150 users required 325 megabytes memory. Some of the 325 megabytes may be shared between, and some of it is related to the 150 individual jobs. This can be expressed as 325= m*150 + b where “m” is the amount of storage required by each user and “b” is the shared memory requirement.

150 users

0

20

40

60

80

100

120

140

100 200 300 400 500

Mem ory ( MB)

Fau

lts

per

Se

cond

Page 30 of 53

Test 2: 100 users Following the same discussion that was applied to the test with 150 users, similar logic applies here, with the conclusion being that for 100 users, the formula would be: 225 = 100 * m + b .

100 users

0

10

20

30

40

50

60

70

100 150 200 250 300

Me m ory ( MB)

Fau

lts

per

Sec

ond

Solving the two equations (for the 150 users and for the 100 users)...

325 = 150m + b (-) 225 = 100m + b --------------------- 100 = 50m m = 2 megabytes of memory Now we substitute 2 for m in one of the original equations and solve for b. 325 = 150*2 + b 325 = 300 + b b = 25 megabytes of memory ... the per user requirement is two megabytes and the shared memory requirement is 25 megabytes. So, the formula to include in the workload solution is: MEM = 2 megabytes * users + 25 megabytes where “users” is the number of concurrent active users. This can be checked by running the third test for 50 users.

Page 31 of 53

50 users

0102030405060708090

100

50 100 150 200 250

Mem ory ( MB)

Faul

ts p

er S

eco

nd

From the graph, it can be seen that 50 users require approximately 125 to 150 megabytes of memory. The formula can be validated by calculating the memory requirement for 50 users.

50 users * 2mb + 25 mb = 125 mb In this example, the assumption is that the application is running out of just one memory pool. Some applications run in more than one subsystem and/or pool. For example, an order entry application might have an interactive component running in one pool which places new orders on a data queue for an order fulfillment job that is running in another pool. The size of all application pools needs to be considered in this exercise. However, there is no need to include the machine pool or memory required by base system functions that occur in the base pool. The Sizing Guide Estimator will take care of sizing this requirement.

5.3 To Size Disk Requirements Disk-sizing goal: There are two goals for the disk-sizing portion of the tests: to analyze the total amount of storage the application requires, and to determine the number of disk drives necessary for good performance. Ultimately, equations must be derived that define two values:

STORAGE—The amount of disk storage required by the application ARMS—The number of disk arms required to adequately handle the I/O for the application processing workload. Here, the term “disk arms” or “arms” is synonymous with “disk drives.”

In regard to determining the total amount of disk storage, it is necessary to think about all of the objects, programs, data, and files required in some way by the application. These could be objects that are either created or used by the application. Some objects will be fixed in size, such as application programs, an application-defined lookup table, or some other static object. The size of other objects will be dependent on application usage. Data size also varies,

Page 32 of 53

including information such as the number of items in an inventory database, the amount of historical data kept on-line, and details maintained about users while they are active. Do not forget to include any temporary space needed by the application. Some database functions, for example, must utilize temporary space to build indices or to sort data records. Once a complete set of all objects required by the application has been defined, this set is analyzed to determine a formula to quantify the storage requirements. The formula will most likely have a constant portion representing the fixed size requirements. It will also have one or more expressions to account for the usage-dependent data. The formula might look something like this example:

Total Storage = fixed storage (size of programs and static objects) + # of customers in customer master file * typical size of customer information for one customer + # of items in inventory file * size of information kept about each inventory item + # active users * storage requirement for a single user + # orders processed per day * average order size * # days retained on-line

The data for this formula comes from the questions previously defined in Section 2.2. That is, the number of customers, number of items in inventory, number of active users, number of orders processed per day, average order size, and retention period could all be determined by questions posed to the end user. This example defines storage for an order entry application but the concepts should be similar for most applications. Generally speaking, the formula for specifying total disk space requirements will have the form:

STORAGE = fs + (n1 * s1) + (n2 * s2) + .... + (nn * sn),

Where “fs” is the fixed storage requirement, n1, n2, .... is the number(s) of customers, accounts, users, etc., that have associated storage and which vary from one use of the application to another, and s1, s2, .... is the corresponding storage requirement for each n1, n2....

Remember when developing this formula, it will be better to err on the “high side,” while still being as realistic as possible. This disk storage is spread across one or more disk drives, which brings the discussion to the next formula that must be defined—the number of disk drives required. Number of disk drives: When sizing the disk requirements, it is important not only to determine the total amount of storage needed, but also to ascertain an adequate number of disk drives over which the storage is spread. If the data is spread across too small a number of drives, application performance will suffer—because data requests are issued faster than they can be satisfied and the application (and end-user) has to wait for the drives to catch up. Thus, the number of disk drives required will be driven by how much work the application is doing, that is, it is determined by the application transaction volumes.

Page 33 of 53

Just as with the approach for sizing memory, there are different approaches to determining a formula for the number of disk drives needed. A simple “rule-of-thumb” approach would be to observe how the application performs at a couple of customer locations, and then simply deduce a relationship between transaction rate and number of disk drives required—based on the transactions believed to be processed at the customer locations and the number of drives in the customer configurations. Or, in very simple situations and perhaps for applications which will only be marketed on smaller configurations, the number of disk drives could be “hard-coded” to be a constant number which is “known” to be adequate for the application. Apart from these “rule-of-thumb” or experience-based approaches, it is possible, once again, to measure and analyze data—somewhat similar to the memory-sizing exercise. More than one test may be required. In this case, rather than changing memory pool sizes and observing faulting rates, the number of configured disk drives could even be changed in the system to observe what happens to arm utilization. However, reconfiguring the number of drives on a system is a time-consuming process!! So an effort should be made to make as few changes as possible. The approach described here is to 1) determine an initial disk configuration, 2) use the exploratory tests to refine this initial configuration, if necessary, and 3) run a few tests with varying numbers of users or transactions against a fixed disk configuration. The goal is to find the relationship between users (or transactions processed) and the number of arms required. To calculate an initial number of arms for the DASD tests: Start with twice as many disk drives as are needed to satisfy the total disk storage

(determined above.) This allows some room for growth, ensures that disk performance is not negatively impacted because the drives are too full, and also allows enough room if RAID protection is turned on.

During an exploratory test, monitor the “% busy” of the disk drives. (See the Management Central monitor “Disk Arm Utilization.”) In a test environment, a good target is 10-25% busy. Less than 10% busy may not be realistic and more than 25% busy in a test environment doesn’t allow for occasional peaks in I/O activity which normally occur in a less pristine production environment. (In order for this to be a balanced configuration, a disk % busy of 10-25% would support a CPU utilization of 60-70% busy.)

If the arm utilization is significantly below 10% busy, while the CPU is 60-70% busy, it may be necessary to remove some disk arms and try the test again. Similarly, if the disk arm utilization is greater than 25% busy, additional arms should be added and the workload should be run again. It may be appropriate to use the CLRPOOL command to clear memory pools between tests to ensure that all of the active data for the test isn’t left in memory from test to test.

The screen capture shown below is the same iSeries Navigator screen as shown earlier. It graphically represents a number of performance measurements. At this point in the discussion, however, it is important to focus on the graphs that demonstrate the demand being placed on the disk(s)—“Disk Arm Utilization (Average)” and “Disk Arm Utilization (Maximum).”

Page 34 of 53

Page 35 of 53

Page 35 of 53

The following chart and table is an example illustrating how a workload utilizes two different disk configurations: 10 versus 20 drives. As transaction volumes increase, disk I/O typically increases and the number of required disk drives increases. When there are 10 disk drives configured, the disk drive % busy exceeds guidelines at higher order rates. With 20 drives, disk drive % busy stays below 40% busy.

Disk Drive Ut ilizat ion

0%

20%

40%

60%

80%

100%

0 200 400 600 800

Orders Processed

% B

usy

10 Disk Drives 20 Disk Drives

When doing tests to determine disk arm requirements, it is seldom necessary to complete an entire set of measurement points as illustrated in the data. Instead, one or two points can be measured. From this data, a calculation would be performed to determine a proposed general formula for disk drive requirements for the application. Once the formula is established, it is possible to measure a couple of additional points to validate or update the formula, or it may have been deemed necessary to add or subtract disk drives from the configuration to keep disk % busy in the 10-25% range.

System # of drives % busy CPU % busy Orders/ hour Drives at 25%820- 2435 10 7% 13% 50 2.8820- 2435 10 12% 21% 100 4.8820- 2435 10 33% 47% 300 13.2820- 2435 10 45% 60% 400 18820- 2435 10 57% 73% 500 22.8820- 2435 10 88% 88% 600 35.2820- 2435 20 7% 21% 100 5.6820- 2435 20 19% 47% 300 15.2820- 2435 20 25% 60% 400 20820- 2435 20 31% 73% 500 24.8820- 2435 20 40% 88% 600 32

Page 36 of 53

To come up with the general formula for disk arm requirements, the relationship between the number of transactions processed and the number of disk drives must be ascertained. To do this, the first step is to calculate the drives required at a target utilization. For this example, 25% busy is the target utilization (see the chart above). (However, the particular % busy number used in a live benchmarking exercise will be higher or lower, depending on what is deemed most appropriate.) The “drives required at 25% busy” number will be used when finding the relationship between the orders processed and the number of drives required. For this example, two measurement points (see shaded rows in the table above) make it possible to solve two equations in two unknowns. #drives = orders/hour * drives/order + drives_required_ for_ base_functions 13.2 drives = 300 * d + b 4.8 drvies = 100 * d + b 8.4 drives = 200 * d d = .042 and it follows that b = .6 The resulting formula for drives required or arms required is: ARMS = .042 * orders/hr + .6 The following graph shows how the derived formula compares with the measured data.

Num ber of Disk Drives Required

0

5

10

15

20

25

30

35

40

0 100 200 300 400 500 600 700

Transact ions per Hour Rat e

Drives ( 2 5 % busy)

Page 37 of 53

5.4 A Word About Scalability Scalability refers to the performance characteristics of (1) an application as the workload increases, and (2) an the application as it runs on larger and larger server configurations. For example, an application with a characteristic of linear scalability requires twice the server resources to perform twice the workload. Another vertical scalability characteristic involves whether an application can make full use of system resources (e.g., all processors as n-way increases). Independent of the sizing work (addressed in this document), the application itself should be “tuned” to make optimal use of CPU, memory, and disk resources — that is, to make the application scale in a predictable and efficient manner. Examples of tuning techniques include: analyzing the application’s efficiency, adjusting the application design and implementation to perform as much parallel work in multiple threads (or jobs), and minimizing locking contention between application users (or between various parts of the application). Given enough disk and memory resources, and with enough workload, a highly scalable application can push the processor to 90% utilization or higher. Scalability can be much less simple than as just broadly stated in the previous paragraph. An application may have been designed for a certain kind of hardware configuration, which effectively means that buying larger, faster hardware may not yield the expected, linear improvement in throughput. For instance, if the application is written to take advantage of 16 threads, then running that application on a 32-way processor will not show throughput that is anywhere near twice as fast. Herein lies the real benefit of testing an application on multiple server platforms. It is important to understand how the application scales, and in turn include those characteristics in the Sizing Guide Solution. In this way, it is possible to recognize and validate whether the application will scale as might ideally be expected. It is not at all uncommon for today’s legacy applications to be written for optimal performance within a bounded set of users and server speeds.

Page 38 of 53

Let’s look at the example below. CPU Requirements

100 200 300 400 500 600Users

0

100

200

300

400

500

600

CP

W

Sys 1

Sys 2

Sys 3

From this set of hypothetical CPU data, one can make a few observations: There appear to be two differing groups of systems. This should be investigated to see if

the machines were set up as the plan spelled out. For demonstration purposes, it is assumed that Sys 1 is a “cacheless” system, while both Sys2 and Sys3 have L2 caches. Thus, when the Sizing Guide Solution is developed, one set of calculations will be made for “cache” systems and another for “cacheless.”

Sys2 and Sys3 seem to hit a “knee.” Again, when developing the Sizing Guide Solution, this is handled by having one equation for the amount of CPU needed when the number of users is less than 400 (for example), and another equation when the number of users is more than 400.

Page 39 of 53

6. Generation of Sizing Guide Solution At this point, the hard part of generating the data values has already come to pass. All the numbers have been gathered and calculated. Now, it is time to use the Sizing Guide Developer tool to create the Sizing Guide Solution. For a description of how to do this, refer to “Developing Sizing Guide Descriptions for the IBM Sizing Guide Estimator,” authored by Joseph Puhl and found at: www.developer.ibm.com/welcome/e3.pl6.1 General Considerations The result from the Sizing Guide Estimator is the determination of the system resources needed (e.g., CPU rating, memory size, number of disk drives, and total disk storage) to support the workloads or combination of workloads chosen. The Sizing Guide Estimator goes even further by identifying a system model that is configured to contain those resources. Some general aspects commonly associated with Business Partner applications include: The allowance for extra resources for certain future capabilities of the application. The

Sizing Guide Estimator provides several ways for a user to specify additional resources or to allow for future growth. (See the Sizing Guide Estimator Help and tutorials for more information.)

A restriction on server model recommendations. Capabilities exist both within the Sizing Guide Estimator and during the creation of the Sizing Guide Solution to limit which server models are recommended.

Grouping disk and/or memory resource recommendations into configurable (full) multiples (e.g., number of disk drives in increments that fill disk drive cabinets). Configuration support, per se, is not a role of the Sizing Guide Estimator and is left to be determined by other tools or sales personnel who are specialized in configuration.

The recommendation of “standardized” model configurations (e.g., CPU, memory, disk packages). This also is specialized configuration support that is left for other tools or sales personnel who are specialized in configuration.

6.2 Create “Help” Documentation Creating a good Sizing Guide Solution involves more than performance analysis of the application, defining the questions, and producing the formulas to turn question answers into resource requirements. Equally important is the consideration of usability and understandability. If the customer or sales representative is unable to use or understand the questions in the Sizing Guide Solution, gross errors specifying the answers to the questions will lead to incorrect sizings and poor customer satisfaction with the server configuration that has been sized. To facilitate usability and understandability, the Sizing Guide Estimator encourages the customer or sales representative to link to additional descriptive help information. It also contains messages or comments specific to the solution that has been sized in the final results. Sizing Guide Solutions provided by IBM include context sensitive (question or terminology level) descriptive help; overall Sizing Guide Solution level descriptive help; additional reference level links to more complete descriptive help, technical papers; performance studies; and error, caution, and/or informational messages. These same help capabilities should be developed for every Sizing Guide Solution created for the Sizing Guide Estimator. The Sizing Guide Estimator is meant to be useable by less-

Page 40 of 53

experienced people as well as by people who are more expert in sizing the specific applications. Include plenty of time and effort for generating error and guidance messages and “Help” text that will assist these folks in successfully using the Sizing Guide Solution. A less-experienced person should review the entire process and the related documentation to ensure that, from an end-user perspective, the sizing process that has been assembled makes sense and is intuitive to use.

Page 41 of 53

7. Validation Now that the Sizing Guide Solution has been generated, it will be necessary to verify the results of the new Sizing Guide by comparing its results against one, or ideally, several production servers. Do the recommendations made by the Sizing Guide Solution fall in line with the day-to-day utilization that customers are actually experiencing? This is the last opportunity, before publication of the sizing product, to ensure its reasonability in real-world environments. This is an excellent opportunity to put the Sizing Guide Solution in front of other people within the Business Partner organization, perhaps some salespeople, for instance. Let them work through the tool using some of the parameters that are relevant to customer situations where they already “know” the answer. Do the recommendations made by the Sizing Guide Solution coincide with what the customer is actually experiencing in a real-world environment? Do the salespeople get a sense that they can trust the results of the sizing effort? Do they find the Sizing Guide Solution easy to use? Answers to these questions will yield some final feedback that can allow for tweaking the Sizing Guide Solution so it is even more useful once it is “rolled out.”

Page 42 of 53

8. Publication to Web Once the customized Sizing Guide has been created, tested, and validated; it is time to deliver it to the Business Partner’s customer set. This is probably most easily done via the Web—providing for convenient update while assuring that the “public” will instantly have the latest version of the Sizing Guide Solution at their disposal.

Page 43 of 53

9. Final Words There are many challenges that Business Partners face in the very early phases of developing new client relationships, as well as in the course of meeting the continuing needs of their longstanding customers. One of these challenges—that of recommending an appropriately sized server system—would seem rather simple. But, in fact, the complexity and breadth of today’s applications, coupled with the rather unpredictable spikes from Web applications, cause the process of determining loads and optimal resource utilization to be inordinately mathematical and even theoretical. This is the reason IBM has worked to construct the IBM eServer Sizing Guide Estimator. This tool (as this paper has communicated) is designed to assist Business Partners in providing a “server sizing guide solution” to their customers... a tool that is customized to predict how the provider’s application(s) would work in a given server environment, even when other applications are running. A customized estimator provides a competitive advantage by: Extending the perception of the Business Partner’s expertise among the client and

prospect base. Delivering to the Business Partner sales force a vehicle for more credible proposal

generation. Offering Business Partners’ existing customers a self-sufficient vehicle for near and long-

term planning purposes. Establishing for internal purposes, an efficient method for assessing workload changes

that occur because of added functionality in a new release of the provider’s software. IBM encourages its Business Partners to look at this powerful tool as a strong addition to their collection of marketing and support tools.

Page 44 of 53

Appendix A: Run Procedures 1. Run sheet: Log the run on a run sheet (or spreadsheet) with date/time, name of

QPFRDATA member, Management Collection object name, LoadRunner results file name, run description, key performance metrics such as CPU %, response time, etc.

2. iSeries checklists: a. Memory Allocation

Review pool sizes, activity levels, paging characteristics (*CALC versus *FIXED). Consider using CLRPOOL to clear memory pools between runs.

b. Batch Check job queues, are any applicable batch jobs submitted and on hold? Check output queues. Clear output queues if necessary. Put output queues on

hold so any spool files created during the run do not print. c. Journaling

Is Journaling/Mimix turned on? If a user ASP is used, is it big enough? Do any journal receivers need to be

deleted? If not using a user ASP for journal receivers, consider it for performance.

d. Communications Check communication lines, if applicable. Remote dial-up lines should be varied

off so that remote users cannot impact the run. Check communications between any local systems in the benchmark (e.g., Mimix connection, other servers).

OptiConnect (if applicable): Is OptiConnect active? Check for QSOC subsystem, be prepared to monitor.

e. Web servers, other subsystems Are applicable subsystems and servers started? HTTP server, WebSphere, etc. Are the appropriate server-specific tuning parameters set correctly for the run?

Garbage collection, timeout values, queue size, etc. f. Interactive Jobs

Check for disconnected jobs from a previous run. Check controllers and devices used by Mercury interactive jobs. Make sure all

are in the correct state (i.e., none are varied off). g. Collection Services

Verify that Collection Services has a status of “started.” Typically, data is collected at 1-minute or 5-minute intervals, depending on the overall length of the measurement period.

h. Database Reset Procedure List specific steps here, these may include: clearing files, resetting a date,

restoring a subset of files, etc. 3. Simulation Tool checklists:

a. May need to restart PCs between runs. b. Check/change variable data feeding the scripts. c. Are processes in place to monitor Simulation Tool performance for ensuring the

Simulation Tools do not become the bottleneck?

Page 45 of 53

Appendix B: Page Faults The following steps will help determine the impact of page faults on response time so that either the workload can be reduced or more memory can be added to ensure a better sizing result. 1. Determine the faults per second in the pool(s) where the jobs are running by using IBM

Management Central Monitors or the performance tools System Report. 2. Find the transactions per second, either through Management Central Monitors or from

statistics provided by the workload generator. 3. Calculate the faults per transaction. 4. Find the disk response time from a system report. 5. Find the portion of response time delay that is due to faulting by multiplying the faults per

transaction by the disk response time. If the portion of response time due to faulting is 25% or more, memory should either be added or workload should be reduced. This is a rough rule of thumb. Exceptions are possible.

Page 46 of 53

Appendix C: Collection Services As noted in this document, in addition to Management Central Realtime Monitors, the use of Collection Services to gather more comprehensive system performance data is recommended. Collection Services takes the place of the iSeries Performance Monitor with which many Business Partners may be familiar. The screen capture that follows shows the interface for Collections Services on the iSeries Navigator window.

Page 47 of 53

This next screen capture shows the interface for gathering Collections Services performance data. There are options on the Performance Tools green screen menus to start and stop Collection Services. However, the recommendation is to use Management Central/iSeries Navigator to start and stop collection and create performance database members for use with the Performance Tools. There are a few recommended settings for the collection that are illustrated here: During test execution, the collection retention period for detailed data should be

“permanent” to avoid having data inadvertently removed from the system, and The graph data should cover seven days or, with the use of Performance

Management/400, up to 31 days.

Page 48 of 53

When collecting performance data, it is important to specify which data is to be collected by Collection Services. For analysis of performance data using the Performance Tools, specify that the “collection profile to use” be set to “Standard plus protocol” as shown on the following screen capture. Notice that the “Refresh” tab is missing on this window. This is because this is a “start collecting” window. The previous chart (which does have a “Refresh” tab) was a “properties” window.

Page 49 of 53

The screen capture below shows collection objects under Collection Services. In this example, Collection Services was continually collecting performance data, cycling every night at midnight. A collection object can be selected for analysis by creating database files that can be analyzed by the Performance Tools LPP, or “Graph History” can be chosen, as shown here. “Graph History” provides access to performance data for system analysis from completed benchmark runs.

Page 50 of 53

The realtime data that was available via Management Central Monitors and the Collection Services data can also be analyzed, after the fact, with the “Graph History” function. The pull down (illustrated in the screen capture below) shows the statistics that are available through “Graph History.” In this example, “LAN Utilization” was selected. By looking at the primary chart shown in this screen capture, a sharp increase in utilization at 3:35 can be observed.

Page 51 of 53

“Graph History” provides both a drill-down analysis of specific time intervals and a comparison of multiple resources, as can be seen in the following two screen captures.

Page 52 of 53

Trademarks and Disclaimers ®Copyright IBM Corporation, 1994-2003. All rights reserved. References in this document to IBM products or services do not imply that IBM intends to make them available in every country. The following terms are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both: IBM(logo) iSeries e (logo) business eServer IBM Lotus Domino WebSphere DB2 OS/400 Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. Loadrunner is a registered trademark of Mercury Interactive. Information is provided "AS IS" without warranty of any kind. All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer. Information in this presentation concerning non-IBM products does not constitute an endorsement of such products by IBM. IBM cannot confirm the accuracy of performance, capability, or any other claims related to non-IBM products. Questions on the capability of non-IBM products should be addressed to the supplier of those products. All statements regarding IBM future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. Contact your local IBM office or IBM authorized reseller for the full text of the specific Statement of Direction. Some information in this presentation addresses anticipated future capabilities. Such information is not intended as a definitive statement of a commitment to specific levels of performance, function or delivery schedules with respect to any future products. Such commitments are only made in IBM product announcements. The information is presented here to communicate IBM's current investment and development activities as a good faith effort to help with our customers' future planning. Performance is based on measurements and projections using benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput or performance improvements equivalent to the ratios stated here.

Page 53 of 53