IBM Tivoli Identity Manager Performance Tuning Guidepublib.boulder.ibm.com/tividd/td/ITIM/SC32-1496-02/en_US/PDF/ITIM… · IBM Tivoli Identity Manager Performance Tuning Guide Page

IBM Tivoli Identity Manager Performance Tuning Guide Version 4.5.1

Issue Date: 2005 December 15 – Fifth Edition

Publication Number: SC32-1496-04

Copyright Notice

Copyright IBM Corporation 2005. All rights reserved. May only be used pursuant to a Tivoli Systems Software License Agreement, an IBM Software License Agreement, or Addendum for Tivoli Products to IBM Customer or License Agreement. No part of this publication may be reproduced, transmitted, transcribed, stored in a retrieval system, or translated into any computer language, in any form or by any means, electronic, mechanical, magnetic, optical, chemical, manual, or otherwise, without prior written permission of IBM Corporation. IBM Corporation grants you limited permission to make hardcopy or other reproductions of any machine-readable documentation for your own use, provided that each such reproduction shall carry the IBM Corporation copyright notice. No other rights under copyright are granted without prior written permission of IBM Corporation. The document is not intended for production and is furnished “as is” without warranty of any kind. All warranties on this document are hereby disclaimed, including the warranties of merchantability and fitness for a particular purpose.

U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corporation.

Trademarks

IBM, the IBM logo, Tivoli, the Tivoli logo, AIX, IBM DB2, IBM Tivoli Identity Manager and WebSphere Application Server are trademarks or registered trademarks of International Business Machines Corporation or Tivoli Systems Inc. in the United States, other countries, or both.

Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both.

UNIX is a registered trademark of The Open Group in the United States and other countries.

Java and all Java-based trademarks and logos are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States and other countries.

Other company, product, and service names may be trademarks or service marks of others.

Notices

References in this publication to Tivoli Systems or IBM products, programs, or services do not imply that they will be available in all countries in which Tivoli Systems or IBM operates. Any reference to these products, programs, or services is not intended to imply that only Tivoli Systems or IBM products, programs, or services can be used. Subject to valid intellectual property or other legally protectable right of Tivoli Systems or IBM, any functionally equivalent product, program, or service can be used instead of the referenced product, program, or service. The evaluation and verification of operation in conjunction with other products, except those expressly designated by Tivoli Systems or IBM, are the responsibility of the user. Tivoli Systems or IBM may have patents or pending patent applications covering subject matter in this document. The furnishing of this document does not give you any license to these patents. You can send license inquiries, in writing, to the IBM Director of Licensing, IBM Corporation, North Castle Drive, Armonk, New York 10504-1785, U.S.A.

IBM Tivoli Identity Manager Performance Tuning Guide Page 1

Table of contents Table of contents ......................................................................................................................................1 About this guide ........................................................................................................................................3

Who should use this guide ...................................................................................................................3 What’s new ...........................................................................................................................................3

Fifth edition.......................................................................................................................................3 Fourth edition ...................................................................................................................................3 Third edition .....................................................................................................................................3 Second edition .................................................................................................................................4

1 Introduction...........................................................................................................................................5 1.1 Vital tunings .................................................................................................................................5 1.2 Initial tunings................................................................................................................................5 1.3 Resource allocation .....................................................................................................................6

1.3.1 Memory....................................................................................................................................6 1.3.2 CPU .........................................................................................................................................6 1.3.3 Disk space ...............................................................................................................................7

2 J2EE application servers ......................................................................................................................8 2.1 IBM WebSphere Application Server ............................................................................................8

2.1.1 Java virtual machine (JVM) size..............................................................................................8 2.1.2 Java virtual machine (JVM) options – Sun Solaris..................................................................9 2.1.3 JDBC pool sizes and timeouts ..............................................................................................10

2.1.3.1 Pool sizes .....................................................................................................................11 2.1.3.2 Timeouts .......................................................................................................................11 2.1.3.3 Statement cache size ...................................................................................................12 2.1.3.4 Setting the JDBC values...............................................................................................12

2.1.4 JMS pool size and timeouts ..................................................................................................12 2.1.4.1 Pool size .......................................................................................................................12 2.1.4.2 Timeouts .......................................................................................................................13 2.1.4.3 Setting the JMS values.................................................................................................13

2.1.5 Client inactivity timeouts ........................................................................................................13 2.1.6 IBM HTTP Server ..................................................................................................................14

2.1.6.4 Connections..................................................................................................................14 2.1.7 IBM WebSphere MQ .............................................................................................................14

2.2 BEA WebLogic Server...............................................................................................................15 3 IBM Tivoli Identity Manager application..............................................................................................16

3.1 Threads......................................................................................................................................16 3.2 Intel multi-processor systems ....................................................................................................17

4 Database servers ...............................................................................................................................18 4.1 IBM DB2 ....................................................................................................................................18

4.1.1 Quick guide for setting the IBM DB2 tuning parameters .......................................................18 4.1.2 Bufferpools ............................................................................................................................18 4.1.3 Connections...........................................................................................................................19 4.1.4 Tablespaces ..........................................................................................................................19 4.1.5 Transaction logs ....................................................................................................................20 4.1.6 Database application heaps ..................................................................................................20 4.1.7 Indexing .................................................................................................................................20 4.1.8 Runstats ................................................................................................................................21 4.1.9 Maximum open files...............................................................................................................21 4.1.10 Lock tuning ............................................................................................................................22

4.1.10.5 Lock type ..................................................................................................................22 4.1.10.6 Lock list and maximum locks....................................................................................22 4.1.10.7 Lock timeout .............................................................................................................22

4.2 Microsoft SQL Server ................................................................................................................22 4.3 Oracle Database........................................................................................................................23

4.3.1 Server configuration ..............................................................................................................23

Page 2 IBM Tivoli Identity Manager Performance Tuning Guide

4.3.2 Spreading database data ......................................................................................................23 4.3.3 Indexing .................................................................................................................................23 4.3.4 Updating statistics .................................................................................................................23

5 Directory servers ................................................................................................................................25 5.1 IBM Tivoli Directory Server........................................................................................................25

5.1.1 Quick guide for setting the IBM Tivoli Directory Server tuning parameters ..........................25 5.1.2 Cache sizes ...........................................................................................................................25 5.1.3 Bufferpools ............................................................................................................................26 5.1.4 Connections...........................................................................................................................27 5.1.5 Transaction logs ....................................................................................................................28 5.1.6 Database statement heap .....................................................................................................28 5.1.7 Database application heaps ..................................................................................................29 5.1.8 Package Cache Size .............................................................................................................29 5.1.9 Database sort heaps .............................................................................................................30 5.1.10 Indexing .................................................................................................................................30 5.1.11 Runstats ................................................................................................................................31

5.2 Sun ONE Directory Server ........................................................................................................31 5.2.1 All IDs Threshold Value.........................................................................................................32 5.2.2 Indexing .................................................................................................................................33 5.2.3 Database cache size .............................................................................................................33

6 Best practices .....................................................................................................................................35 7 Troubleshooting..................................................................................................................................36

7.1 Log files .....................................................................................................................................36 7.2 Identifying bottlenecks ...............................................................................................................36 7.3 Specific problems ......................................................................................................................37

7.3.1 IBM Tivoli Identity Manager application.................................................................................37 7.3.2 IBM WebSphere MQ .............................................................................................................37 7.3.3 IBM Tivoli Directory Server....................................................................................................38 7.3.4 Sun ONE Directory Server ....................................................................................................39

8 IBM DB2 performance monitoring ......................................................................................................41 8.1 Enable monitoring......................................................................................................................41 8.2 Snapshots..................................................................................................................................41 8.3 Bufferpool hit ratio......................................................................................................................41

9 Other resources..................................................................................................................................42 10 Appendix A – scripts and files ............................................................................................................43

10.1 perftune_runstats.sh..................................................................................................................43 10.2 ids_indexes.ldif ..........................................................................................................................43


About this guide This guide identifies some ways to tune your IBM® Tivoli Identity Manager™ system to improve performance.

Who should use this guide Use this guide if you are responsible for installing or maintaining an IBM Tivoli Identity Manager system. The following competencies are recommended:

• Familiarity with basic database and directory design principles.

• General knowledge of system resource management (memory, disk, and processor) including your operating system’s memory model. Without knowledge of other resource requirements on the system it is possible to negatively impact performance by over tuning.

• Understanding of how to configure and administer your directory and database servers. You may need to have your local database administrator or directory administrator perform these tunings for you.

What’s new Fifth edition

New tuning: Added Oracle indexes and tablespace distributions.

Updated: Updated mechanism for adding IBM Tivoli Directory Server™ indexes. Updated IBM Tivoli Directory Server cache size recommendations. Updated the best practices section to include recommended provisioning policy sizes.

Fourth edition

New tuning: Added IBM WebSphere Application Server client inactivity timeout values. Added IBM Tivoli Directory Server pckcachesz parameter tuning. Added IBM HTTP Server ThreadsPerChild (Windows) MaxClients (Unix) setting.

Updated: Updated IBM Tivoli Directory Server indexing to include the O attribute. Updated IBM Tivoli Identity Manager database tablespace size and transaction tunings. Updated JVM minimum and maximum size recommendations. Updated IBM Tivoli Directory Server ACL and filter cache settings. Updated IBM Tivoli Directory Server STMTHEAP setting. Updated Sun ONE Directory Server All IDs Threshold value. Added v3.user.at method of adding IBM Tivoli Directory Server indexes.

New feature: Oracle database tuning.

Third edition

New tuning: Added additional IBM DB2® lock tuning information. Added tuning for IBM DB2 maximum number of open files. Added tuning for IBM WebSphere® Application Server on Sun Solaris™. Added tuning for IBM WebSphere MQ.

Updated: Changed the IBM Tivoli Directory Server cache tunings.

New feature: Best Practices section.


Troubleshooting section.

Second edition

New tuning: Added tunings for IBM HTTP Server included with the IBM WebSphere Application Server. Added tunings for Sun ONE Directory Server. Added additional information on the IBM Tivoli Identity Manager application threading. Added IBM DB2 runstats tuning information for the IBM Tivoli Directory Server database.

Updated: The perftune_runstats.sh script includes the new index tuning for the IBM Tivoli Directory Server database.

Organization: Reorganized the IBM WebSphere Application Server sections to make applying tunings easier. Changed the Quick guide sections to make them less verbose.


1 Introduction

The IBM Tivoli Identity Manager product solves the complex problem of identity management. Due to the complexity of the problem, Tivoli Identity Manager can be challenging to maximize the use of resources – or in other words, to tune. This tuning guide provides a system administrator with the information needed to tune the application for your environment. Other individuals (such as IBM DB2 or IBM Tivoli Directory Server administrators) in your organization might offer differing advice. It has been our experience that system administrators know your environment better and their advice might be more accurate for your environment than this tuning document can be.

The Tivoli Identity Manager product can be divided into four major pieces: Java™ 2 Platform, Enterprise Edition (J2EE) application servers, the Tivoli Identity Manager application, database servers, and directory servers. Some of these pieces can be divided further such as workflow (WF) and user interface (UI) components for the Tivoli Identity Manager servers. We will address each of these separately in this document.

The Tivoli Identity Manager server can be installed in three different modes: single server, clustered servers, and functional cluster servers. For performance reasons single servers and clustered servers are the same – each machine has both a WF and UI component on one machine. A clustered environment can be considered a group of single servers with regard to tuning. A functional cluster server is different than a regular cluster because in a functional cluster each component (WF/UI) is on a separate machine. In a functional cluster environment you might have any multiple of machines.

This document is a working document, as more information is gathered settings may be added, removed or changed in future releases. It is recommended that you check the IBM web site for the most recent version. To find the most recent version, go to http://www.ibm.com/support/us. Select “Search technical support”, type “ITIM Tuning Guide” in the box, and click submit.

Tivoli Identity Manager configurations vary significantly by platform, operating system, middleware applications, and hardware being used. You must tailor your configuration settings for your deployment. Configurations and settings used in this document include AIX®, Windows®, and Solaris in single server and cluster configurations.

1.1 Vital tunings There are several thousand different parameters that you can tune for J2EE application servers, Tivoli Identity Manager product, directory servers, and database servers. This tuning guide discusses a small subset of these parameters.

If you are setting up an acceptance or production environment, read each section and perform the applicable tunings for your systems. If you are setting up a test environment and want to get started as quickly as possible, focus on these areas:

• IBM DB2 - Runstats

• IBM DB2 - Bufferpools

• IBM WebSphere Application Server - Java virtual machine (JVM) size

The IBM DB2 - Runstats tunings is a vital part of the Tivoli Identity Manager product performance when DB2 is used as the database server. See the Initial tunings section to tune this parameter in a newly deployed environment.

1.2 Initial tunings Most of these tunings can be implemented in a newly deployed environment or an environment that is already deployed. When tuning IBM DB2 in a newly deployed environment it is important to prime your database statistics using the DB2 runstats command.

http://www.ibm.com/support/us


The runstats command creates information in the DB2 statistics tables based on information currently in the data tables. In a pristine environment, those tables are empty and the statistics generated from runstats on these tables are useless to DB2. Without adequate information on the distribution of data within these tables, DB2 makes poor choices on how to fulfill the queries used by the Tivoli Identity Manager application.

The solution to this problem is to prime your database by doing a small number of transactions, running the runstats command, and then running the full transaction. The IBM DB2 - Runstats section discusses how to use the runstats command. Failing to prime the database in this manner can result in poor performance or transaction rollbacks.

Example: If the goal is to load 20,000 users into the Tivoli Identity Manager product using a DSML feed, load the first 50 to 100 people using the DSML file first. After the load completes, run the DB2 runstats command and load the remaining users with the DSML file.

1.3 Resource allocation The tuning values are difficult to manage when more than one piece of middleware is on a given system, for example having the Tivoli Identity Manager server, DB2, and IBM Tivoli Directory Server all on the same machine. Regardless of configuration, be sure that resources are not over-allocated.

1.3.1 Memory All middleware components allow you to adjust how much memory they will use. When calculating how to allocate memory to middleware components, keep these considerations in mind:

• Configuring middleware memory settings too high can result in the operating system swapping memory out to disk if the physical memory is exceeded. This will result in extremely poor performance and should be avoided. After setting up or changing the memory values for the middleware, monitor the memory and swap space used to ensure that nothing is being swapped out to disk. If it is, adjust your memory settings to compensate.

• 32-bit processes can only allocate 2GB of RAM. If a 32-bit process is configured to allocate more than 2GB of memory, the application (such as IBM Tivoli Directory Server) could halt or unexpectedly fail.

• IBM Tivoli Directory Server has internal caches in addition to the IBM Tivoli Directory Server process. Ensure that the size of your caches plus the size of your IBM Tivoli Directory Server process does not exceed 2GB. The entry cache size limit determines the number of entries in the cache, not the size of the cache. The size of each cache entry will vary based on the Tivoli Identity Manager product configuration and any extensions to the base IBM Tivoli Directory Server schema. See the IBM Tivoli Directory Server – Cache sizes section.

• Although the bufferpools account for a large amount of the memory used by DB2, the application control heaps, the sort heaps, and the statement heaps also use memory. Do not overlook these sizes when computing how much memory to allocate to DB2. See the IBM DB2 section.

• A large part of the WebSphere Application Server’s memory usage is the JVM size. However, the size of the JVM does not set an upper bound on the amount of memory that the WebSphere Application Server may use. See the J2EE application servers - IBM WebSphere Application Server section.

1.3.2 CPU All the components of the Tivoli Identity Manager product (Tivoli Identity Manager application, J2EE application server, database server, and directory server) are CPU intensive. This document groups the J2EE application server and the Tivoli Identity Manager application together because it is difficult to isolate their CPU usage. Keep in mind the following considerations:


• Tivoli Identity Manager product Version 4.4 and Version 4.5 workflow processes are limited to a single thread for performance reasons. These versions are best suited for machines with fast processors rather than machines with multiple, slower processors. The Tivoli Identity Manager product Version 4.5.1 can take better advantage of multi-processor machines.

• Some Tivoli Identity Manager server processes (such as some provisioning calculations) are single threaded and will be isolated to a single processor regardless of what version of Tivoli Identity Manager server you are using.

• Both IBM Tivoli Directory Server and DB2 are multi-threaded applications and will perform best on a multi-processor machine.

• Even in a well tuned environment the system, bottleneck may vary between being CPU bound on the Tivoli Identity Manager server, on the directory server, and on the database server. For this reason, the Tivoli Identity Manager server, the directory server, and the database server should all be on separate machines. If that is not possible, put the database server and the directory server on the same machine instead of putting one of them on a machine with the Tivoli Identity Manager server.

1.3.3 Disk space Each of the middleware components use different amounts of disk space for various purposes.

• The J2EE application server and the Tivoli Identity Manager application use disk space beyond their installation size because of log files (such as the itim.log file) and WebSphere MQ queues. Adjust the number of archives and size of the itim.log file in the enRoleLogging.properties file. Make sure that WebSphere MQ has enough disk space for its processing logs (not error logs) to grow. The Tivoli Identity Manager server pushes many entries onto the queues during large provisioning changes, causing the queues to grow.

• IBM Tivoli Directory Server uses disk space both from the IBM Tivoli Directory Server process (log files like ibmslapd.log) and the DB2 database. IBM Tivoli Directory Server uses system managed space (SMS) tablespaces that allow the system to manage the amount of disk space used. SMS tablespaces do not allow the specification of an upper bound so it is important to monitor the amount of disk space used to ensure that the drive does not become full.

• In addition to the tablespaces for the database data, DB2 uses disk space for the transaction logs. When configuring the transaction logs (discussed in the IBM Tivoli Directory Server – Transaction logs and IBM DB2 – Transaction logs sections) ensure that you have enough disk space for the log files.

• The Tivoli Identity Manager DB2 database uses directory managed space (DMS) tablespaces that require pre-allocating disk space for the database to use. If the tablespace becomes full, DB2 will be unable to continue and will return an error in the itim.log file. If this occurs, add more tablespace containers. As a general rule, for every 200,000 transactions that occur, 1Gb of tablespace is needed to archive the historical transaction information. A policy change that makes modifications to 5,000 users is 5,000 transactions from a IBM Tivoli Identity Manager perspective. See the IBM DB2 – Tablespaces section for additional information.


2 J2EE application servers

Regardless of the installation type (single, regular cluster, or functional cluster), the Tivoli Identity Manager server can be thought of as two components: the J2EE application server running the application (WebSphere Application Server or BEA WebLogic® Server) and the Tivoli Identity Manager application itself. Both components need to be tuned for your environment.

2.1 IBM WebSphere Application Server WebSphere Application Server allows you to tune a variety of settings for your environment. This document discusses the Java virtual machine (JVM) size, the minimum and maximum Java database connectivity (JDBC) and Java Messaging Service (JMS) pool sizes, and the various timeouts.

2.1.1 Java virtual machine (JVM) size By default, WebSphere Application Server sets the maximum JVM size to 256MB. This value is too small for the Tivoli Identity Manager product to run beyond a basic concept test and should be increased to 512MB. If your machine has adequate available RAM (such as an unused 1.5GB or more of physical RAM), consider increasing it to as much as 1.5GB. For large reconciliations and policy evaluations 512MB may not be enough memory to complete these tasks and this increase may be a requirement.

Setting the maximum memory value too high can cause large pauses in the system during a full garbage collection. Do not set the maximum memory value higher than 1.5GB even if your system has the available memory.

Do not set the JVM heap size to be larger than the physical RAM. The WebSphere Application Server suffers significant performance degradation if the operating system swaps out the JVM to swap space.

The WebSphere Application Server sets the initial JVM size to 0. To decrease the number of full garbage collections, speed up start time, and prevent the JVM using virtual memory (disk), increase this to the size of your maximum heap size.

Determining the values

initial_jvm_heap_size = The initial size of the JVM heap in megabytes. Recommended value: size of the max_jvm_heap_size.

max_jvm_heap_size = The maximum size of the JVM heap in megabytes. Recommended value: 512 or higher.

Setting the values

1) Open the WebSphere Administration Console.

2) Expand the Servers list in the navigation pane.

3) Select Application Servers in the navigation pane.

4) Select the server to manage.

5) Select Process Definition from the Additional Properties pane at the bottom.

6) Select Java Virtual Machine from the Additional Properties pane at the bottom.

7) Set the Initial Heap Size to initial_jvm_heap_size.

8) Set the Maximum Heap Size to max_jvm_heap_size.

9) Repeat this procedure for each Tivoli Identity Manager server.

2.1.2 Java virtual machine (JVM) options – Sun Solaris Note: This section applies specifically to WebSphere Application Server V5.0 on Sun Solaris, including later fixpack releases, but not later versions. For example, it applies to WebSphere Application Server V5.0.2 but not Version 5.1. This section applies to WebSphere Application Server for Sun Solaris, not AIX or Windows. WebSphere Application Server V5.0 for AIX ships with IBM’s implementation of Java 1.3.1 whereas WebSphere Application Server V5.0 for Sun Solaris ships with Sun’s implementation of Java 1.3.1. IBM’s Java and Sun’s Java have two different garbage collection algorithms and need to be tuned differently. If you are using WebSphere Application Server on AIX or Windows, skip this section.

Sun’s 1.3.1 JVM on Sun Solaris supports two different JVM types: client and server. The client JVM is designed to have a faster startup time and is tuned for shorter-running applications than the server JVM. The server JVM is designed for longer-running applications and therefore sets different initial values to some JVM tunings. By default the WebSphere Application Server uses the client JVM. Tivoli Identity Manager runs better under the server JVM.

Sun’s 1.3.1 JVM uses a generational garbage collection algorithm. How this algorithm works is outside the scope of this document, but we will give a short summary. See the Other resources section at the end of this document for links to more information.

The following is a simplified view of how the generational garbage collection algorithm works. The JVM divides the Java heap size into two major components: the young generation, which includes Eden and two Survivor pools, and the old generation (sometimes called the tenured generation). When objects are allocated inside Tivoli Identity Manager, they are created inside Eden. After Eden becomes full, a partial garbage collection occurs and any objects that are still being used (live objects) are moved to the first Survivor pool. Eden is cleared and object allocation begins again inside Eden. After Eden becomes full again, any live objects inside Eden and the first Survivor pool are moved to the second Survivor pool. Eden is cleared and object allocation begins again inside Eden. This process repeats with the objects being switched between the two survivor pools until objects die or are eventually tenured to the old generation. When there is no more space inside Eden or the Survivor pools, all remaining live objects are moved to the old generation. Eden is cleared and object allocation begins again. When the old generation becomes full, a full garbage collection occurs. Every object within the heap is analyzed and heap compaction occurs within the old generation.

When garbage collections, both partial and full, occur everything within the JVM stops. This causes small pauses within Tivoli Identity Manager during garbage collections. Partial garbage collections finish very quickly, usually in less than half a second. Full garbage collections take longer to compact the entire heap and can take 10 seconds or more to finish, depending on the size of your heap. Because of this, it is important to minimize the number of garbage collections, particularly full garbage collections. We do this by adjusting the size of the Java heap, the new generation, and the survivor pools.

A diagram illustrating the JVM heap layout:

Eden Tenured GenerationSurvivor pools

max_new_sizemax_jvm_size

new_ra tiosurvivor_ratio


jvm_type = The type of JVM to use, either -client or -server. Recommended value: -server

max_new_size = The maximum size of the new generation within the JVM in megabytes. Set this


to 1/3 of your maximum heap size.

new_ratio = The ratio of the new generation to the old generation. Set this to 2, thereby setting the size of your new generation to 1/3 of your heap size.

survivor_ratio = The ratio of a single survivor pool to Eden. Set this to 3, thereby setting the size of each survivor pool to 1/5 of your new generation (note this is not 1/4 because there are two survivor pools).

target_survivor_ratio = A target for the amount of survivor space to be used after a garbage collection. This value is used by the JVM to calculate for how many iterations an object will stay in the survivor pools before being tenured to the old generation. Set this to 75 to allow more time for objects to die in the survivor pool before being tenured.

With a total JVM heap size of 512MB, the recommendations above would yield the following heap configuration:

Eden Tenured GenerationSurvivorpools

170MB

512MB

102MB 34MB

342MB

Calculating the values

argument_string = jvm_type -XX:MaxNewSize=max_new_sizem -XX:NewRatio=new_ratio -XX:SurvivorRatio=survivor_ratio -XX:TargetSurvivorRatio=target_survivor_ratio

Note: Ensure the MaxNewSize value has a trailing ‘m’ specifying that the value is in megabytes.

Example: For a JVM size of 512MB, use the following argument string:

argument_string = -server -XX:MaxNewSize=170m -XX:NewRatio=2 -XX:SurvivorRatio=3 -XX:TargetSurvivorRatio=75

Setting the values





5) Select Process Definition from the Additional Properties pane at the bottom.

6) Select Java Virtual Machine from the Additional Properties pane at the bottom.

7) Append argument_string to Generic JVM arguments


2.1.3 JDBC pool sizes and timeouts The Tivoli Identity Manager application uses JDBC connections to connect to the database backend during the processing of workflow items (such as changing a password, creating a new account, or loading a DSML feed). The JDBC pool size and timeouts need to be updated for your system.



2.1.3.1 Pool sizes

The minimum and maximum values for the JDBC connections might need to be increased depending on the function of the Tivoli Identity Manager installation.


Find the value in the following table that matches your Tivoli Identity Manager installation configuration.

Functional Cluster (per each component) Parameter Single Server or

Regular Cluster UI Component WF Component

min_pool_size 15 5 15

max_pool_size 60 10 50

The difference in the UI and WF component values is because most transactions in the UI (such as password changes) are delegated to a WF component, which makes the actual JDBC connection. If the Tivoli Identity Manager server is unable to keep your database server busy under heavy load, consider increasing the maximum pool size.

Your database needs to be able to accept the total number of connections plus 5 more for maintenance purposes.

Example: You have a functional cluster with 2 workflows (WF) and 2 user interfaces (UI). Your WF component will need 50 * 2 = 100 connections and your UI component will need 10 * 2 = 20 possible connections. You also need to include the additional 5 connections for administrative purposes, which means you need to set your database connections to 100 + 20 + 5 = 125.

Example: If you have a regular cluster with 3 nodes (each Tivoli Identity Manager node includes one WF and one UI) you need (50 + 10) * 3 + 5 = 185 database connections.

Refer to the database connections section for the appropriate database for more information on setting the maximum number of connections.

Setting the values

See the Setting the JDBC values section to set the pool size, timeouts, and statement cache size together.

2.1.3.2 Timeouts

The timeouts for the Tivoli Identity Manager application need to be adjusted from their initial settings. The following values work well for most environments. If you begin receiving a large number of timeouts, consider increasing the appropriate timeout value.


jdbc_connection_timeout = The number of seconds before the JDBC connection is closed due to inactivity. Recommended value: 60.

jdbc_idle_timeout = The number of seconds an allocated JDBC connection can remain in the pool before being freed. Recommended value: 90.

jdbc_orphan_timeout = The number of seconds an application can hold an unused JDBC connection before being freed. Recommended value: 60.

Setting the values



2.1.3.3 Statement cache size

The Tivoli Identity Manager application uses prepared statements when communicating with the database server. Adjust this value to yield a better tradeoff between performance and resource utilization.


jdbc_statement_cache_size = The maximum number of prepared statements to cache. Recommended value: 60.

Setting the values


2.1.3.4 Setting the JDBC values


2) Expand the Resources list in the navigation pane.

3) Select JDBC Providers in the navigation pane.

4) Single server: Select Server and click Apply. Cluster server: Select the Node and Server (using Browse Nodes and Browse Servers) and click Apply.

5) Select ITIM JDBC Provider (XA).

6) Select Data Sources (Version 4) from the Additional Properties pane at the bottom.

7) Select ITIM Data Source.

8) Select Connection Pool from the Additional Properties pane at the bottom.

1) Set the Minimum Pool Size to min_pool_size.

9) Set the Maximum Pool Size to max_pool_size.

10) Set the Connection Timeout to jdbc_connection_timeout (recommended value: 60).

11) Set the Idle Timeout to jdbc_idle_timeout (recommended value: 90).

12) Set the Orphan Timeout to jdbc_orphan_timeout (recommended value: 60).

13) Set the Statement Cache Size to jdbc_statement_cache_size (recommended value: 60).


2.1.4 JMS pool size and timeouts The Tivoli Identity Manager application uses JMS queues to manage workflow items. The JMS pool size and timeouts need to be updated for your system.

2.1.4.1 Pool size

The connection pool size for the Tivoli Identity Manager application may need to be adjusted for your system if you increased the maximum number of threads for the Tivoli Identity Manager messaging queues. See the IBM Tivoli Identity Manager application - Threads section for more information.


max_pool_size = The maximum number of connections available in the queue pool. Default value: 50.


Setting the values

See the Setting the JMS values section to set the pool size and timeouts together.

2.1.4.2 Timeouts

The timeouts for the Tivoli Identity Manager application need to be adjusted from their initial settings. The following values work well for most environments. If you begin receiving a large number of timeouts, consider increasing the appropriate timeout value.


queue_connection_timeout = The number of seconds before the queue connection is closed due to inactivity. Recommended value: 300.

queue_reap_time = The number of seconds between runs of the queue pool maintenance thread. Recommended value: 300.

Setting the values

See the Setting the JMS values section to set the pool size and timeouts together.

2.1.4.3 Setting the JMS values


2) Expand the Resources list in the navigation pane.

3) Select WebSphere JMS Providers in the navigation pane.

4) Single server: Select Server and click Apply. Cluster server: Select the Node and Server (using Browse Nodes and Browse Servers) and click Apply.

5) Select WebSphere Queue Connection Factories.

6) Select ITIM Queue Connection Factory.

7) Select Connection Pool from the Additional Properties pane at the bottom.

8) Set the Max Connections to max_pool_size.

9) Set the Connection Timeout to queue_connection_timeout.

10) Set the Reap Time to queue_reap_time.


2.1.5 Client inactivity timeouts IBM Tivoli Identity Manager activities that run for long periods can be timed out inside WebSphere and abort running operations. A primary example of this is a long running reconciliation process. For customers that have long running reconciliations and experience transaction rollback messages in the itim.log, increasing this value will allow these requests to complete.


inactivity_timeout = The number of seconds before a client transaction request is rolled back due to inactivity. Recommended value: 3600.

Setting the values






5) Select Transaction Service from the Additional Properties pane at the bottom.

6) Set the Client Inactivity Timeout to inactivity_timeout.


2.1.6 IBM HTTP Server The WebSphere Application Server uses the IBM HTTP Server as a front-end server in a single-server installation and as a load balancer between nodes in a cluster installation. The default configuration parameters for the IBM HTTP Server work well for small and medium configurations but may need to be increased in environments with a large number of concurrent users.

2.1.6.4 Connections

The IBM HTTP Server can be configured to limit the number of connections that it will accept at one time. This value may be too small if the servers experience a large number of concurrent users. Connections are maintained to the server until a user selects logout (which doesn’t always happen) or until after 10 minutes of inactivity when the connection is closed.

The IBM HTTP Server supports the HTTP/1.1 KeepAlive request that allows a client to make multiple HTTP requests through a single persistent connection. There is a limit on the number of KeepAlive requests that a single connection will accept. After this limit is reached, the connection is closed and another connection must be established. The default value may be too small for some external provisioning processes such as a JNDI feed.


ibmhttp_home = The home directory of the IBM HTTP Server, such as /usr/IBMHttpServer.

max_connections = The maximum number of connections that can be made to the HTTP server at one time. This should be set to the maximum number of concurrent users you expect on your system. Recommended value: 200. This may be increased to as high as 600 for customers with a large number of concurrent users.

max_keepalive = The maximum number of requests allowed during a persistent connection. Recommended value: 0 (unlimited).

Setting the values

Note: The MaxClients parameter on Windows is called ThreadsPerChild.

Edit the ibmhttp_home/conf/httpd.conf file and update the following entries: MaxClients max_connections MaxKeepAliveRequests max_keepalive

Stop and restart the IBM HTTP Server for these changes to take effect.

2.1.7 IBM WebSphere MQ The WebSphere Application Server uses an embedded copy of the IBM MQSeries messaging service to manage Java Message Service (JMS) messages. The Tivoli Identity Manager server configures several persistent queues (the exact name and number can vary slightly from version to version): WQ_itim_adhocSync, WQ_itim_ms, WQ_itim_rs, WQ_itim_wf, WQ_itim_wf_abort, WQ_itim_wf_pending. The Tivoli Identity Manager server uses these queues during various provisioning actions. During large


provisioning policies it is possible to reach the limit on the number of uncommitted messages, so this value should be increased.


queue_manager_name = The name of the queue manager, such as WAS_vanexel_jmsserver. You can find a list of valid queue manager names by running dspmq.

max_uncommitted_messages = The maximum number of uncommitted messages. Default value: 10000. Recommended value: 50000.

Setting the values

1) Start the MQ Script interpreter: runmqsc queue_manager_name

2) Alter the maximum uncommitted messages size: alter qmgr maxumsgs(max_uncommitted_messages)

3) Confirm the change: display qmgr maxumsgs

4) Exit the MQ Script interpreter: end

2.2 BEA WebLogic Server Steps to tune BEA WebLogic Server will be included here when those settings and their values are determined.


3 IBM Tivoli Identity Manager application

The IBM Tivoli Identity Manager application includes several configuration files that provide an area for tuning various parts of the application’s performance. These are in the data/ directory under the Tivoli Identity Manager product home directory.

3.1 Threads The IBM Tivoli Identity Manager product is a multi-threaded application and in most cases is able to take advantage of multiple processors. In the Tivoli Identity Manager product Version 4.4 and Version 4.5, the threads that process workflow items were set to one for performance reasons. The Tivoli Identity Manager product Version 4.5.1 is better able to take advantage of multiple processors and the number of workflow threads should be set to more than one.

In most cases the default number of threads are adequate. If during a large provisioning policy or other large workload run, you find your database server and the Tivoli Identity Manager server are not fully utilized, you might try increasing the maximum number of threads.

There are seven thread pools: adhocSyncQueue, workflowQueue, workflowPendingQueue, workflowAbortQueue, remoteServicesQueue, remotePendingQueue, and mailServicesQueue. Only two of these, workflowQueue and workflowPendingQueue, should need adjustment. The configuration file allows adjustments to both the maximum and the minimum number of threads. However, the minimum number of threads should not be changed as the system quickly increases the number of threads beyond this value.

The workflowQueue is used to process workflow items and should have the largest allocation of threads. The workflowPendingQueue is used to start certain provisioning activities and should have fewer threads than the workflowQueue. Use a 2:1 ratio between the number of workflowQueue threads and workflowPendingQueue threads.


itim_home = The Tivoli Identity Manager product home directory, such as /itim45.

max_workflow_threads = The maximum number of threads to use processing workflow items. Default value: 10.

max_workflow_pending_threads = The maximum number of threads to use when processing pending workflow items. Default value: 5.

Note: When increasing the maximum number of threads, you might need to increase the size of the JMS connection pool within your J2EE application server. The size of the JMS connection pool should be greater than the sum of the maximum values for all messaging threads. Following this guideline ensures an adequate number of JMS connections in the event that the maximum number of threads for all Tivoli Identity Manager messaging queues are active at one time.

Setting the values

Note: The lines below are split for readability purposes; they should be all in one line in the enRole.properties file.

Edit the itim_home/data/enRole.properties file and update the following lines: enrole.messaging.workflowQueue.attributes=TRANSACTED=true RECEIVE_TIMEOUT=60 MAX_THREADS=max_workflow_threads MIN_THREADS=5

enrole.messaging.workflowPendingQueue.attributes=TRANSACTED=true RECEIVE_TIMEOUT=60 WAIT_TIME=0 OVERCAPACITY_WAIT_TIME=10 MAX_THREADS=max_workflow_pending_threads MIN_THREADS=2


Restart the Tivoli Identity Manager server for the changes to take effect.

3.2 Intel multi-processor systems For multi-processor Intel systems it is necessary to update the enrole.messaging parameters in the enRole.properties file to eliminate Java context switching.


itim_home = The Tivoli Identity Manager product home directory, such as /itim45.

Setting the values

Note: The lines below are split for readability purposes, they should be all in one line in the enRole.properties file.

Edit the itim_home/data/enRole.properties file and append PRIORITY=5 to the following lines as shown: enrole.messaging.adhocSyncQueue.attributes=TRANSACTED=true RECEIVE_TIMEOUT=60 MAX_THREADS=5 MIN_THREADS=1 PRIORITY=5

enrole.messaging.workflowQueue.attributes=TRANSACTED=true RECEIVE_TIMEOUT=60 MAX_THREADS=10 MIN_THREADS=5 PRIORITY=5

enrole.messaging.workflowPendingQueue.attributes=TRANSACTED=true RECEIVE_TIMEOUT=60 WAIT_TIME=0 OVERCAPACITY_WAIT_TIME=10 MAX_THREADS=5 MIN_THREADS=2 PRIORITY=5

enrole.messaging.workflowAbortQueue.attributes=TRANSACTED=true RECEIVE_TIMEOUT=60 WAIT_TIME=0 MAX_THREADS=5 MIN_THREADS=1 PRIORITY=5

enrole.messaging.remoteServicesQueue.attributes=TRANSACTED=false RECEIVE_TIMEOUT=60 WAIT_TIME=0 MAX_THREADS=7 MIN_THREADS=2 PRIORITY=5

enrole.messaging.mailServicesQueue.attributes=TRANSACTED=false RECEIVE_TIMEOUT=60 WAIT_TIME=0 MAX_THREADS=3 MIN_THREADS=1 PRIORITY=5

Restart the Tivoli Identity Manager server for the changes to take effect.


4 Database servers

The Tivoli Identity Manager product supports three different database systems: IBM DB2, Microsoft® SQL Server™, and Oracle® Database. Each database requires slightly different tunings.

Each database machine should have at least one processor and 1GB of RAM. It can reside on a single processor machine by itself or share a multi-processor machine with other applications, but the requirements of 1GB of RAM per one CPU is the minimum for the database server. Tuning the database is one of the most important tunings for the Tivoli Identity Manager product.

4.1 IBM DB2 Tuning IBM DB2 to run with the Tivoli Identity Manager product involves adjusting the bufferpools, modifying the number of connections, modifying internal database values, adding tablespace, adjusting logs, indexing, and running runstats.

4.1.1 Quick guide for setting the IBM DB2 tuning parameters This section includes the commands needed to tune the IBM DB2 parameters. This section assumes you have been through the tunings before, know what the desired values are, and just need a reminder of how to set them.

As the database administrator, connect to the database and run the following commands: db2 alter bufferpool ibmdefaultbp size ibmdefaultbp_npages db2 alter bufferpool enrolebp size enrolebp_npages db2 update db cfg for itim_database_name using maxappls num_connections db2 "ALTER TABLESPACE ENROLE_DATA ADD ( \ FILE '/database_home/database_name/NODE0000/SQL00001/container_name' num_pages) \ PREFETCHSIZE 16 OVERHEAD 24.1 TRANSFERRATE 0.9 BUFFERPOOL ENROLEBP" db2 update db cfg for itim_database using logsecond logs_secondary db2 update db cfg for itim_database using logfilsiz logs_size db2 update db cfg for itim_database using applheapsz applheap_size db2 update db cfg for itim_database using app_ctl_heap_sz appctl_size db2 update db cfg for itim_database using maxfilop max_files_open db2set DB2_RR_TO_RS=YES

4.1.2 Bufferpools The Tivoli Identity Manager database has two bufferpools: IBMDEFAULTBP and ENROLEBP. IBMDEFAULTBP is used as a buffer for tablespaces with small extent sizes (4k) and ENROLEBP is used as a buffer for tablespaces with large extent sizes (32k). Most of the tables within the Tivoli Identity Manager database use the tablespace with a large extent size and thus will use the ENROLEBP bufferpool. Use a 1:3 memory ratio between IBMDEFAULTBP and ENROLEBP.


First, determine the following values for your system:

total_mem_avail = The total amount of physical memory in bytes.

mem_for_itimdb = The amount of memory in bytes to allocate to the Tivoli Identity Manager database. This value should be small enough that it will reside in physical memory and not be swapped out to disk. Recommended value: 500000000 (500MB) or greater.

Next, calculate the following values:

ibmdefaultbp_npages = (mem_for_itimdb / 4096) * 0.25 enrolebp_npages = (mem_for_itimdb / 32768) * 0.75


Setting the values

As the database administrator, connect to the database and run the following commands: db2 alter bufferpool ibmdefaultbp size ibmdefaultbp_npages db2 alter bufferpool enrolebp size enrolebp_npages

4.1.3 Connections The Tivoli Identity Manager server uses JDBC connections from the J2EE application server to communicate with the database. The number of connections from the application server to the database depends on the needs of the application. Both application servers allow you to set maximum connection values. The DB2 variable MAXAPPLS needs to be set to a value greater than the maximum possible connections. We recommend setting MAXAPPLS to five more than the total maximum possible connections.


itim_database_name = The name of your Tivoli Identity Manager database, such as itim.

num_connections = Maximum number of connections. See the J2EE application servers section to calculate the maximum number of connections and add five to that value.

Setting the values

As the database administrator, connect to the database and run the following command: db2 update db cfg for itim_database_name using maxappls num_connections

4.1.4 Tablespaces The Tivoli Identity Manager product uses a Database Managed Space (DMS) tablespace to store its data. This type of tablespace gives better performance than System Managed Space (SMS) tablespaces but requires you to set aside disk space for the database to use. The default size of the Tivoli Identity Manager tablespace is 512MB. This is frequently not large enough and should be increased by adding more containers to the tablespace.

Add more tablespaces by using the DB2 alter tablespace command. If possible, add files that reside on another physical drive because DB2 performs better if one tablespace has multiple containers on multiple drives.


database_home = The home directory of your database administrator, such as /home/db2inst1.

database_name = The name of the database such as db2inst1. This is also a subdirectory under database_home.

container_name = The name of the file to create to hold the tablespace container, such as enrole_data2.

num_pages = The number of 32k pages to add to the tablespace. To calculate the number of pages from the amount of disk space, divide the size in megabytes by 0.032768. A 512MB tablespace would be 15625 pages.

Setting the values

As the database administrator, connect to the database and run the following command: db2 "ALTER TABLESPACE ENROLE_DATA ADD ( \ FILE '/database_home/database_name/NODE0000/SQL00001/container_name' num_pages) \ PREFETCHSIZE 16 OVERHEAD 24.1 TRANSFERRATE 0.9 BUFFERPOOL ENROLEBP"


4.1.5 Transaction logs DB2 keeps logs during transaction processing. During large transactions the default log number and sizes might be too small and cause transaction rollbacks. This is solved by increasing the size and number of log files available to DB2.

DB2 has two types of log files: primary and secondary. Primary logs are allocated when the database is started and remain allocated until the database is stopped. Secondary logs are allocated as needed after the primary logs are full and are released after they are no longer needed.


Increase the number of secondary logs to be prepared when large transactions occur. The default size of the log files is 1000 4k pages, or 4MB. Increase this to 10000 4k pages, or 40MB. Note that this will increase the size of both your primary and secondary log files.

itim_database = The name of the Tivoli Identity Manager database, such as itim.

logs_secondary = The number of secondary logs. Recommended value: 12.

logs_size = The size of the primary and secondary logs in 4k pages. Recommended value: 10000.

Setting the values

As the database administrator, connect to the database and run the following commands: db2 update db cfg for itim_database using logsecond logs_secondary db2 update db cfg for itim_database using logfilsiz logs_size

4.1.6 Database application heaps Some of the queries that the Tivoli Identity Manager application submits to the DB2 server result in complex SQL statements. The default sizes of the application heaps are not large enough to allow DB2 to fulfill the queries. Failing to increase these will result in errors in the itim.log file and might cause transaction rollback error messages. If you receive errors after increasing these values, continue to increase the values in increments of 256.


itim_database = The name of the Tivoli Identity Manager database, such as itim.

applheap_size = The value of applheapsz in 4k pages. Default value: 256. Recommended value: 2048.

appctl_size = The value of app_ctl_heap_sz in 4k pages. Default value: 128. Recommended value: 1024.

Setting the values

As the database administrator, connect to the database and run the following commands: db2 update db cfg for itim_database using applheapsz applheap_size db2 update db cfg for itim_database using app_ctl_heap_sz appctl_size

Stop and restart the database instance for these changes to take effect.

4.1.7 Indexing Adding an index to a heavily used table can greatly increase performance. Without indexes, DB2 must scan every row of the table until it finds the specified data. With an index, it uses a more efficient search method. The following indexes have been identified as increasing database performance.

Note: It is very important to run runstats on the system after adding new indexes. See the IBM DB2 - Runstats section for more information.



There are no values to be determined for this section.

Setting the values - Tivoli Identity Manager product Version 4.4 and Version 4.5

As the database administrator, connect to the database and run the following commands:

Note: Some of these indexes have already been added to the Tivoli Identity Manager installation. If there are "unable to create index" errors, they can be ignored as duplicates.

db2 'CREATE UNIQUE INDEX ACTIVITY_ID_X ON "ENROLE.ACTIVITY" \ ("ID" ASC) INCLUDE ("SUBTYPE")' db2 'CREATE INDEX ACTIVITY_PID_X ON "ENROLE.ACTIVITY" ("PROCESS_ID" DESC)' db2 'CREATE INDEX ACTIVITY_STATE_X ON "ENROLE.ACTIVITY" ("STATE" DESC)' db2 'CREATE UNIQUE INDEX PROCESS_ID_X ON "ENROLE.PROCESS" ("ID" ASC)' db2 'CREATE INDEX PROCESSDATA_PID_X ON "ENROLE.PROCESSDATA" \ ("PROCESS_ID" DESC, "DEF_ID" DESC)' db2 'CREATE INDEX WORKITEM_PID_X ON "ENROLE.WORKITEM" ("PROCESS_ID" DESC)' db2 'CREATE INDEX PROCESS_SUB_X ON "ENROLE.PROCESS" ("SUBMITTED" DESC, "PARENT_ID" ASC)'

Setting the values - Tivoli Identity Manager product Version 4.5.1

No additional indexes are required for tuning the Tivoli Identity Manager product Version 4.5.1.

4.1.8 Runstats The number of rows in the tables and what indexes are available are required for DB2 to efficiently fulfill queries. Run the runstats command on all tables in the database before and after large DSML loads and reconciliations. If you experience high CPU usage or poor DB2 performance, run the runstats command on all the tables in the database. DB2 reorgchk does not update index statistics and is not a replacement for runstats. To update index statistics, run the runstats command on each table individually. A script (perftune_runstats.sh) that detects the version of DB2 and runs the runstats command against all tables for a given schema in a database is provided in Appendix A – scripts and files.

If the runstats command is run in a working environment, allow the connected applications to continue to write to the database. Use the "shrlevel change" option on DB2 V7 and "allow write access" option on DB2 V8 to allow users to write to a database while runstats is running.

Use the runstats command on an idle or lightly-used database because it requires update locking on system statistics table to update the database statistics. A database with a heavy load might experience transaction rollbacks due to the system acquiring locks on the tables that are used by the database optimizer to fulfill queries.


Use the runstats command on every table in the ENROLE schema. As the database administrator, connect to the database and use the following command to get a listing of all tables in the schema:

db2 list tables for all | grep ENROLE

Setting the values – DB2 V7

For each table in the ENROLE schema, run the following DB2 runstats command: db2 runstats on table ENROLE.table_name with distribution and \ detailed indexes all shrlevel change


For each table in the ENROLE schema, run the following DB2 runstats command: db2 runstats on table ENROLE.table_name with distribution and \ detailed indexes all allow write access

4.1.9 Maximum open files In an attempt to work well with other applications running on the system, DB2 sets a limit on the number of files it keeps open through the maxfilop setting. If it needs to open more files this value, DB2 will close


a currently open file to open the new one. This can cause a performance loss on systems that do not need this restriction on the number of open files.

Increase this value in increments of 64 if a database snapshot shows that database files have closed.


max_files_open = The maximum number of files DB2 will have open at any one time. Default value: 64. Recommended value: 256.

Setting the values

As the database administrator, connect to the database and run the following command: db2 update db cfg for itim_database using maxfilop max_files_open

4.1.10 Lock tuning

4.1.10.5 Lock type

The Tivoli Identity Manager application requires that RR_TO_RS locking be enabled. This setting allows select statements that involve rows that are deleted but not yet committed to be fulfilled without waiting for the transaction to complete.

More information on this environment setting is available at:

http://www.ibm.com/software/data/db2/udb/ad/db2irfp7/db2ir169.htm



Setting the values

Run the following command as the Tivoli Identity Manager database administrator: db2set DB2_RR_TO_RS=YES

Stop and restart the database instance for this change to take effect.

4.1.10.6 Lock list and maximum locks

The performance team has found that the default settings for the DB2 lock list (locklist) and maximum locks (maxlocks) are adequate for most installations. These may be increased if the local DB2 administrator recommends so for your environment.

4.1.10.7 Lock timeout

The Tivoli Identity Manager application can generally be classified an online transaction processing (OLTP) application. As such it is normally recommended that the lock timeout (locktimeout) value within DB2 be decreased from the default value of -1, which represents infinity. However, it is strongly encouraged that you do not modify this from the -1 value. Not all components of the application recover from a lock timeout and making such a change can result in transaction rollback errors.

4.2 Microsoft SQL Server Steps to tune Microsoft SQL Server will be included here when those settings and their values are determined.

http://www.ibm.com/software/data/db2/udb/ad/db2irfp7/db2ir169.htm


4.3 Oracle Database

4.3.1 Server configuration The default Oracle configuration uses a low performance setting, increasing the configuration to the middle performance setting provides faster Oracle performance. Consult with an Oracle DBA, Oracle administrator, or Oracle documentation for more information on tuning of the Oracle server.

4.3.2 Spreading database data Spreading database data among multiple hard drives is important for production environments. Significant Oracle I/O contentions have been observed which have led to poor Tivoli Identity Manager performance. The spreading is highly environment specific. When creating the Oracle database, as documented in Server Installation and Configuration Guide - Chapter 2, consider spreading the tablespace files across the disks. The following SQL script illustrates the spreading for a Unix system that has 4 hard drives (disk1 through disk4 with Oracle on disk1):

Note: The script is for illustrative purposes only. It is environment specific – not only the file systems, but also the sizes that you can allocate to each tablespace file. Please consult your Oracle DBA and tailor them for your environment before you apply them.


disk# = The disk to spread the tablespaces onto (in our example, disks 1 through 4).

tablespace_size = The size to make each tablespace (in our example between 1GB and 3GB).

Setting the values create tablespace RBS datafile '/disk2/oradata/rbs01.dbf' size 1000m; create temporary tablespace TEMP tempfile '/disk2/oradata/temp01.dbf' size 1000m reuse; create tablespace ENROLE_DATA datafile '/disk3/oradata/enrole_data_01.dbf' size 3000m; create tablespace ENROLE_INDEXES datafile '/disk4/oradata/enrole_indexes_01.dbf' size 1500m;

Note: The itim_home/config/rdbms/oracle/create_rollbackSegment.sql script puts the roll back segment tablespace on ORACLE_HOME (because it only specifies the file name). Consider putting it to a different disk if your environment allows, as illustrated in the above script.

4.3.3 Indexing Adding an index to a heavily used table can greatly increase performance. Without indexes, Oracle must scan every row of the table until it finds the specified data. With an index, it uses a more efficient search method. The following indexes have been identified as increasing database performance.



Setting the values

As the database administrator, connect to the database and run the following SQL commands: create index PROCESS_SUB_X on ENROLE.PROCESS ("SUBMITTED" DESC, "PARENT_ID" ASC); create index PROCESS_ID_ST on ENROLE.PROCESS ("ID" ASC, "STATE" ASC); create index PROCESSDATA_ID_DEF on ENROLE.PROCESSDATA ("PROCESS_ID" ASC, "DEF_ID" ASC, "VALUE_LAST_MODIFIED" ASC); create index SCH_MSG_SVR on ENROLE.SCHEDULED_MESSAGE ("SERVER" ASC);

4.3.4 Updating statistics At regular intervals from 1 week to 1 month on a production IBM Tivoli Identity Manager system, or after a


large amount of processing has been completed, database statistics should be gathered and updated. These statistics are used by Oracle to make query decisions on locating information that have a dramatic impact on how fast Oracle can return requests. The DBMS_STAT package must be installed to issue the following DBMS_STAT commands.


database_instance = The name of the database instance such as enrole.

Setting the values

Create a file named Oracle_dbms.stat_cmds.txt containing the following lines: exec dbms_stats.gather_schema_stats(ownname => 'database_instance', cascade => true);

As the system user, execute sqlplus and run the following command: @ Oracle_dbms.stat_cmds.txt

Generating statistics may take from several minutes to several hours for a large database. Execution should be issued during off-peak times.


5 Directory servers

The Tivoli Identity Manager product supports two different directory servers: IBM Tivoli Directory Server and Sun ONE Directory Server (formerly known as Netscape iPlanet Directory Server).

5.1 IBM Tivoli Directory Server With respect to tuning, IBM Tivoli Directory Server can be divided into two parts: the IBM Tivoli Directory Server process and IBM DB2.

In a well-tuned system, the IBM Tivoli Directory Server process and the DB2 processes consume approximately the same amount of CPU. In a poorly tuned system, DB2 can max out the CPU usage trying to fulfill queries in a poor manner.

Both IBM Tivoli Directory Server and DB2 have caches that speed up data retrieval. Optimizing your available memory is the key to tuning IBM Tivoli Directory Server. When a read request comes in to IBM Tivoli Directory Server, it checks the filter cache to see if has seen that search filter before. If it has, the results are pulled from the cache, otherwise the query goes to DB2. After the search filter is evaluated, IBM Tivoli Directory Server pulls the entries that match the search filter from the entry cache. If the values are not in the entry cache it queries DB2. For each request to DB2, DB2 checks to see if the data resides in a bufferpool. If not, it reads the value from the disk. Ideally, all requests to the directory server register an IBM Tivoli Directory Server cache hit or a DB2 bufferpool hit for the quickest response. Queries that require disk access can be very slow.

5.1.1 Quick guide for setting the IBM Tivoli Directory Server tuning parameters

This section includes the commands needed to tune the IBM Tivoli Directory Server parameters. This section assumes you have been through the tunings before, know what the desired values are, and just need a reminder of how to set them.

As the database administrator, connect to the database and run the following commands: db2 alter bufferpool ibmdefaultbp size ibmdefaultbp_npages db2 alter bufferpool ldapbp size ldapbp_npages db2 update db cfg for ldap_database using logsecond logs_secondary db2 update db cfg for ldap_database using logfilsiz logs_size db2 update db cfg for ldap_database using maxappls max_db_connections db2 update db cfg for ldap_database using stmtheap stmtheap_size db2 update db cfg for ldap_database using applheapsz applheap_size db2 update db cfg for ldap_database using app_ctl_heap_sz appctl_size db2 update db cfg for ldap_database using pckcachesz pckcache_size db2 update db cfg for ldap_database using sortheap sortheap_size db2 update dbm cfg using sheapthres sortheapthres_size

Stop IBM Tivoli Directory Server, edit /etc/slapd32.conf (IBM Tivoli Directory Server V4.1) or /etc/ibmslapd.conf (IBM Tivoli Directory Server V5.x) and update the following configuration options:

ibm-slapdDbConnections: ldap_db_connections ibm-slapdACLCache: acl_cache ibm-slapdACLCacheSize: acl_cache_size ibm-slapdFilterCacheSize: filter_cache_size ibm-slapdEntryCacheSize: entry_cache_size

Stop and restart DB2 and the IBM Tivoli Directory Server server for these changes to take effect.

5.1.2 Cache sizes IBM Tivoli Directory Server has three caches: the ACL cache, the filter cache, and the entry cache. Because the Tivoli Identity Manager server binds as an authoritative user, the ACL cache is used only for internal processes and can be small to allow the memory to be used elsewhere. It has been found that a small value in ACL cache is beneficial for IBM Tivoli Directory Server performance.


The filter cache is most helpful for programs that issue more read requests than write or update requests because the entire filter cache is invalidated at every write. The Tivoli Identity Manager application frequently updates the directory server so it is not beneficial to allocate a large filter cache, in most situations. However, the policy and role evaluation steps conducted by IBM Tivoli Identity Manager make use of the filter cache and to improve performance in these areas, IBM recommends leaving the cache setting at its default value.

IBM Tivoli Directory Server allows you to control how many entries the entry cache can store but does not restrict the size of the cache. The size of each entry in the cache is based on the number and the size of attributes that a given LDAP entry has. Typically, many entries are users and their accounts, which have a fairly constant size. When setting the value for the entry cache, calculate the size of the average entry and divide that into the amount of memory used by the IBM Tivoli Directory Server process. Users with few attributes can generate entry sizes that are approximately 4k where users with more attributes can generate entry sizes around 9k. The IBM Tivoli Directory Server Performance Tuning Guide discusses the procedure for determining the size of the average entry.


acl_cache = Specifies if the ACL cache is used. Recommended value: TRUE (enabled).

acl_cache_size = The size of the ACL cache. Recommended value: 100.

filter_cache_size = The size of the filter cache. Recommended value: 100.

entry_cache_size = The size of the entry cache. Recommended value: as large as the number of Tivoli Identity Manager users times the average number of accounts plus one per user. So if you have 25,000 users with two accounts each, you would want 25,000 * (2+1) = 75,000. This value is bounded by the amount of memory allocated to the IBM Tivoli Directory Server process minus the size of the process itself (about 128MB).

Note: Do not set the entry cache size larger than you have available physical memory. If the IBM Tivoli Directory Server process size exceeds the amount of available memory, significant performance degradation will occur because of page swapping. Care should be taken when increasing the size of the caches so the amount of memory required does not exceed the maximum amount a process can allocate such as the 2GB for most 32-bit processes.

Note: Performance metrics suggest that the Attribute cache available in the IBM Tivoli Directory Server version 5 and later does not provide a significant performance boost and the memory would be better allocated elsewhere.

Example: Setting the entry cache size to 75,000 and given a size of 9k for an entry in the entry cache would require 675MB (75,000 * 9KB = 675,000KB = 675MB) of physical RAM for the entry cache alone, not including the 128MB needed for the server process itself.

Setting the values

Stop IBM Tivoli Directory Server, edit /etc/slapd32.conf (IBM Tivoli Directory Server V4.1) or /etc/ibmslapd.conf (IBM Tivoli Directory Server V5.x) and update the following configuration options:

ibm-slapdACLCache: acl_cache ibm-slapdACLCacheSize: acl_cache_size ibm-slapdFilterCacheSize: filter_cache_size ibm-slapdEntryCacheSize: entry_cache_size

Restart IBM Tivoli Directory Server for these changes to take effect.

5.1.3 Bufferpools DB2 bufferpools are the secondary buffer for IBM Tivoli Directory Server. These bufferpools should be large enough that most table searches can be read directly from memory instead of using the disk. This value can be measured by looking at the hit ratio for the bufferpools. See the IBM DB2 performance monitoring section for information about calculating the bufferpool hit ratio.

The IBM Tivoli Directory Server database (ldapdb2 by default) has two bufferpools: IBMDEFAULTBP and LDAPBP. IBMDEFAULTBP is used as a buffer for tablespaces with small extent sizes (4k) and LDAPBP


is used as a buffer for tablespaces with large extent sizes (32k). Most of the tables in the ldapdb2 database use the tablespace with a small extent size and use IBMDEFAULTBP. Use a 3:1 memory ratio between IBMDEFAULTBP and LDAPBP.


Allocate enough memory to the DB2 bufferpools so the bufferpool hit ratio is greater than 95% and allocate the rest of the memory to the IBM Tivoli Directory Server process and the caches. The following formula can help in determining the bufferpool values for your machine.

First, determine the values for your system:

ldap_database = The name of the IBM Tivoli Directory Server database, such as ldapdb2.

mem_for_ldapdb2_bps = The amount of memory in bytes to allocate to the ldapdb2 bufferpools. This value should be small enough that it will reside in physical memory and not be swapped out to disk. Recommended value: 500000000 (500MB) or greater.

Next, calculate the following values:

ibmdefaultbp_npages = (mem_for_ldapdb2_bps / 4096) * 0.75 ldapbp_npages = (mem_for_ldapdb2_bps / 32768) * 0.25

Setting the values

As the database administrator, connect to the database and run the following commands: db2 alter bufferpool ibmdefaultbp size ibmdefaultbp_npages db2 alter bufferpool ldapbp size ldapbp_npages

5.1.4 Connections There are three connection values to modify for IBM Tivoli Directory Server. The first value is the number of connections that IBM Tivoli Directory Server should make to the DB2 backend. Set this to 30.

The second value is the number of connections that the DB2 server should allow to connect to it. This value should be greater than the number of connections that IBM Tivoli Directory Server will make to it. Set this value to 10 more than the number specified for the IBM Tivoli Directory Server server. These extra connections allow for administrative functions.

The third value only applies to Windows systems. Windows allows up to 64 sockets in a select call for each thread. This limits the number of inbound connections to 64. To overcome this problem set the SLAPD_OCHANDLERS environment variable to specify the number of connection threads, each one supporting 64 connections.


ldap_database_name = The name of your IBM Tivoli Directory Server database, such as ldapdb2.

num_ldap_connections = The number of connections from IBM Tivoli Directory Server to the DB2 backend. Recommended value: 30 (from the IBM Tivoli Directory Server Tuning Guide).

num_db_connections = The number of connections the DB2 backend server will accept. Recommended value: 40.

For Windows systems, also determine the following values:

max_num_LDAP_connections = The maximum number of IBM Tivoli Directory Server connections (enrole.connectionpool.maxpoolsize) from the enRole.properties file. The Tivoli Identity Manager application default is 100.

itim_nodes = The number of Tivoli Identity Manager nodes in your cluster. This is the sum of the WF and the UI components.

oc_handlers = (max_num_LDAP_connections * itim_nodes) / 64


Setting the values

As the database administrator, connect to the database and run the following commands: db2 update db cfg for ldap_database_name using maxappls num_db_connections

To set the number of connections the IBM Tivoli Directory Server server will make to the DB2 server, stop IBM Tivoli Directory Server, edit /etc/slapd32.conf (IBM Tivoli Directory Server V4.1) or /etc/ibmslapd.conf (IBM Tivoli Directory Server V5.x), and find the stanza:

dn: cn=Directory, cn=RDBM Backends, cn=IBM Directory, cn=Schemas, cn=Configuration

Change the following attribute: ibm-slapdDbConnections: num_ldap_connections

For Windows systems, also set the SLAPD_OCHANDLERS values. Stop IBM Tivoli Directory Server, edit slapd32.conf (IBM Tivoli Directory Server 4.1) or ibmslapd.conf (IBM Tivoli Directory Server 5.x) and find the stanza:

dn: cn=Front End, cn=Configuration

Add the following configuration option at the end of the stanza before the blank line: ibm-slapdsetenv: SLAPD_OCHANDLERS=oc_handlers

Stop and start the IBM Tivoli Directory Server database and restart IBM Tivoli Directory Server for these changes to take effect.

5.1.5 Transaction logs DB2 keeps logs during transaction processing. During large transactions the default log number and sizes might be too small and cause transaction rollbacks. This is solved by increasing the size and number of log files available to DB2.

DB2 has two types of log files: primary and secondary. Primary logs are allocated when the database is started and remain allocated until the database is stopped. Secondary logs are allocated as needed after the primary logs are full and are released after they are no longer needed.


Increase the number of secondary logs to be prepared when large transactions occur. The default size of the log files is 1000 4k pages, or 4MB. Increase this to 10000 4k pages, or 40MB. Note that this will increase the size of both your primary and secondary log files.


logs_secondary = The number of secondary logs. Recommended value: 12.

logs_size = The size of the primary and secondary logs in 4k pages. Recommended value: 10000.

Setting the values

As the database administrator, connect to the database and run the following commands: db2 update db cfg for ldap_database using logsecond logs_secondary db2 update db cfg for ldap_database using logfilsiz logs_size

5.1.6 Database statement heap The Tivoli Identity Manager application can submit very long LDAP queries to the IBM Tivoli Directory Server. If the query is particularly long, the query will not fit in the DB2 statement heap (stmtheap) and IBM Tivoli Directory Server will return an error to the Tivoli Identity Manager application. IBM is recommending a value of 4096. If there is an evaluation of password policies of more than 12 services, increase the value as follows: 17 services = 8192 stmtheap, 24 services = 16384 stmtheap. Warning: The increase of this value can dramatically increase memory used by the DB2 server with each


agent(connection) having an increased memory size, insure you have enough available memory for this change.



stmtheap_size = The value of stmtheap in 4k pages. Default value: 2048. Recommended value: 4096.

Setting the values

As the database administrator, connect to the database and run the following commands: db2 update db cfg for ldap_database using stmtheap stmtheap_size

Stop and restart the IBM Tivoli Directory Server database and IBM Tivoli Directory Server for these changes to take effect.

5.1.7 Database application heaps Complex queries that the Tivoli Identity Manager application submits to IBM Tivoli Directory Server result in complex SQL statements to the DB2 database. The default sizes of the application heaps are not large enough to allow DB2 to fulfill these queries from IBM Tivoli Directory Server. Failing to increase these heaps might cause JNDI errors in the itim.log file. If you receive errors after increasing these values, continue to increase these values in increments of 256.



applheap_size = The value of applheapsz in 4k pages. Default value: 256. Recommended value: 2048.

appctl_size = The value of app_ctl_heap_sz in 4k pages. Default value: 128. Recommended value: 1024.

Setting the values

As the database administrator, connect to the database and run the following commands: db2 update db cfg for ldap_database using applheapsz applheap_size db2 update db cfg for ldap_database using app_ctl_heap_sz appctl_size


5.1.8 Package Cache Size Increasing the pckcachesz value size has been shown to decrease DB2 preparation time resulting in less CPU utilization when evaluating provisioning policies with many unique roles. The package cache stores evaluations of these complex queries and increasing this value prevents the DB2 server from repeatedly executing the same costly evaluations of the roles.



pckcache_size = The value of pckcachesz in 4k pages. Default value: 360. Recommended value: 3600.

Setting the values

As the database administrator, connect to the database and run the following commands: db2 update db cfg for ldap_database using pckcachesz pckcache_size



5.1.9 Database sort heaps Several of the SQL queries from IBM Tivoli Directory Server require sorts in the database. To increase performance for these queries, increase the size of the sort heaps in DB2.

There are two sort heap parameters inside DB2: sortheap and sheapthres. The sortheap parameter is a database-level parameter and is allocated as needed to perform sorts. Each sort is allocated its own sortheap. The sheapthres parameter is a database-manager level parameter and provides an upper bound on the total amount of sort heap available to sorts. When increasing the size of the sortheap, ensure that sheapthres is large enough for multiple sorts to allocate sort heap.

If you experience excessive sort overflows, as identified from a database snapshot, increase the sortheap parameter. You might need to increase the sheapthres parameter as well.



sortheap_size = The value of sortheap in 4k pages. Default value: 256. Recommended value: 1024.

sortheapthres_size = The value of sheapthres in 4k pages. Default value: 20,000 pages on UNIX® systems and 10,000 pages on Windows systems. Ideally this value would be sortheap * number_of_database_connections however this value may be too high for your system. The sortheapthres_size should be at least twice as large as sortheap_size.

Setting the values

As the database administrator, connect to the database and run the following commands: db2 update db cfg for ldap_database using sortheap sortheap_size db2 update dbm cfg using sheapthres sortheapthres_size


5.1.10 Indexing Indexing the attributes that applications search on increases IBM Tivoli Directory Server performance. IBM Tivoli Directory Server indexes are automatically translated into IBM DB2 indexes by updating the IBM Tivoli Directory Server schema for those attributes.

If you extend the LDAP schema in IBM Tivoli Directory Server to include additional attributes, index those attributes that you will search for. Any filter in the Tivoli Identity Manager application (such as with dynamic roles) is translated into a search string for the IBM Tivoli Directory Server.

The Tivoli Identity Manager application frequently searches against the o, owner and ou attributes. Index these attributes either through the IBM Tivoli Directory Server Web Administration tool or by importing the ids_indexes.ldif file using the ldapmodify command.

After updating the LDAP schema, run DB2 runstats on the database to update the statistics for the newly created indexes.


root_dn = The root DN of the IBM Tivoli Directory Server server.

root_password = The password for the root DN.

Setting the values

1) To use the LDIF, create the ids_indexes.ldif file from Appendix A – scripts and files and place it on the computer.


2) Run the following command to import the LDIF into IBM Tivoli Directory Server: ldapmodify –D root_dn –w root_password –f ids_indexes.ldif

5.1.11 Runstats The number of rows in the tables and what indexes are available are required for DB2 to efficiently fulfill queries. Run the runstats command on all tables in the database before and after large DSML loads and reconciliations. If you experience high CPU usage or poor DB2 performance, run the runstats command on all the tables in the database. DB2 reorgchk does not update index statistics and is not a replacement for runstats. To update index statistics, run the runstats command on each table individually. A script (perftune_runstats.sh) that detects the version of DB2 and runs the runstats command against all tables for a given schema in a database is provided in Appendix A – scripts and files.

If the runstats command is run in a working environment, allow the connected applications to continue to write to the database. Use the "shrlevel change" option on DB2 V7 and "allow write access" option on DB2 V8 to allow users to write to a database while runstats is running.

Use the runstats command on an idle or lightly-used database because it requires update locking on system statistics table to update the database statistics. A database with a heavy load might experience transaction rollbacks due to the system acquiring locks on the tables that are used by the database optimizer to fulfill queries.

In addition to running runstats on all tables within the database, we also manually update the statistics table for the LDAP_DESC and LDAP_ENTRY tables. The choices DB2 makes about when to uses these tables for fulfilling queries for the IBM Tivoli Directory Server are not idea for Tivoli Identity Manager. By manually adjusting the statistics table, we can help DB2 make better choices and use these tables at the end of the access plan instead of the beginning.


As the database administrator, connect to the database and use the following command to get a listing of all tables in the schema:

db2 list tables for all | grep LDAPDB2


For each table in the LDAPDB2 schema, run the following DB2 runstats command: db2 runstats on table ENROLE.table_name with distribution and \ detailed indexes all shrlevel change

Manually update the database statistics table for the LDAP_DESC and LDAP_ENTRY tables. db2 update sysstat.tables set card = 9E18 where tabname = 'LDAP_DESC' db2 update sysstat.tables set card = 9E18 where tabname = 'LDAP_ENTRY'


For each table in the LDAPDB2 schema, run the following DB2 runstats command: db2 runstats on table ENROLE.table_name with distribution and \ detailed indexes all allow write access

Manually update the database statistics table for the LDAP_DESC and LDAP_ENTRY tables. db2 update sysstat.tables set card = 9E18 where tabname = 'LDAP_DESC' db2 update sysstat.tables set card = 9E18 where tabname = 'LDAP_ENTRY'

5.2 Sun ONE Directory Server The Sun ONE Directory Server (formerly known as Netscape iPlanet Directory Server) does not provide many parameters to adjust for better performance. The three areas that provide the best performance boost are indexing, the All IDs value, and the database cache size.


5.2.1 All IDs Threshold Value The Sun ONE Directory Server sets an upper bound on how many index entries should be allowed for each attribute indexed. If this value is exceeded, the index is invalidated and is not used during searches. This behavior prevents the server from including all entries in an index and consuming excessive amounts of memory.

The Sun ONE Directory Server Installation and Tuning Guide recommends setting this value to be 5% of the size of your directory. The IBM Tivoli Identity Manager product uses the directory server in a dramatically different manner the writers of SunOne Tuning Guide had in mind and this value should be increased to the largest number of users, groups, services, or accounts the ITIM server has stored in the directory.

Because the method to change this parameter is non-trivial, it is recommended that you set the value based on the estimated size of your directory in its final form. If the current size of your directory is 20,000 but you expect the final size of your directory to be 100,000 entries, use the larger value to calculate the All IDs Threshold value.

Please refer to the Sun ONE Directory Server Installation and Tuning Guide for more information on adjusting the All IDs Threshold value.


directory_home = The Sun ONE Directory Server home directory, such as /usr/iplanet/servers/slapd-gso-2DS.

number_of_entries = The number of entries in your directory. A good way to estimate this number is to multiply the number of Tivoli Identity Manager users by the average number of accounts for each user.

suffix_name = The name of the root suffix, such as dc=com.

database_name = The name of the directory server database, such as itim5145.

temp_directory = The name of a temporary directory, such as /tmp. This directory should have enough free space to store an LDIF of your entire directory. Allow 1KB per Tivoli Identity Manager user.

Setting the values

Note: Setting this value requires the database to be rebuilt (the data exported and re-imported). If done incorrectly, this process has the potential to destroy information within your directory. It is highly recommended that you back up the information in your directory prior to setting this value.

Note: Increasing the size of the All IDs Threashold will cause the size of the indexes to grow. Because the indexes are stored in the database cache, the cache size might need to be increased as well. See the See the Sun ONE Directory Server - Database cache size section for information about increasing this value.

Edit directory_home/config/dse.ldif file and find the stanza: dn: cn=config,cn=ldbm database,cn=plugins,cn=config

Change the following attribute: nsslapd-allidsthreshold: number_of_entries

You will need to export and re-import all of your data for this parameter to take effect. This can be done with the db2ldif and ldif2db commands.

First, export your data: db2ldif -n database_name -a temp_directory/export.ldif -s "suffix_name"

Next, stop the directory server.

Finally, import your data: ldif2db -n database_name -i temp_directory/export.ldif


Start the directory server.

5.2.2 Indexing Indexing the attributes that applications search on increases Sun ONE Directory Server performance.

If you extend the LDAP schema to include additional attributes, index those attributes that you will search for. Any filter in the Tivoli Identity Manager application (such as with dynamic roles) is translated into a search string for the LDAP server.

The Tivoli Identity Manager application frequently searches against the O, Owner and OU. Owner is indexed by default in Sun ONE Directory Server but OU and O is not. Index the OU and O attribute from the directory server configuration console.



Setting the values

1) Open the Directory Server Console.

2) Select the Configuration tab.

3) Expand the Data node in the navigation pane.

4) Expand the suffix of your database in the navigation pane and select the database.

5) Select the Indexes tab.

6) Select the Add attribute… button.

7) Find OU in the list box and click OK.

8) If the Equality and Presence options are not already selected for the OU attribute, select them.

9) Click Save.

10) A progress dialog will appear while the index is added. Click Close after the index creation completes.

5.2.3 Database cache size The Sun ONE Directory Server uses a database cache to store index information, filter requests, and entry requests. The directory server uses a database cache of 10MB by default. This value is too small for any but the smallest directory.

Setting this value greater than the amount of physical RAM available will cause performance degradation as the system swaps the pages out to disk.


cache_size = The size of the database cache. Recommended value: 256000000 (256MB). If the entry cache hit ratio on the Status tab for the database is below 95%, increase this value in increments of 128MB until the hit ratio is above 95% or your system is out of physical memory.

Setting the values

1) Open the Directory Server Console.

2) Select the Configuration tab.

3) Expand the Data node in the navigation pane.

4) Expand the suffix of your database in the navigation pane and select the database.

5) Select the Database Settings tab.


6) Set the Memory available for cache to cache_size.

7) Click Save.


6 Best practices

The IBM Tivoli Identity Manager product can be set up and configured in many ways. The following are some best practice suggestions to help guide you in setting up your environment.

• Provisioning policies that touch a large number of users result in a single query to the directory server that returns a large number of users. Because the directory servers only process a query on a single thread, you will obtain faster provisioning with fewer, faster processors than more, slower processors. For example, a 2 x 1.4GHz machine is better suited than a 4 x 700MHz machine.

• In general, network latency is not a major performance bottleneck, but it is still a good idea to have as few hops as possible between components. This includes the Tivoli Identity Manager server components (WF/UI), the directory server, database server, agents, and agent endpoints. Ideally all components would be on the same subnet or no more than one hop away and on 100Mbit or faster network.

• To minimize disk bottlenecks, it is frequently better to have several hard drives in the system rather than one large drive. DB2 can make use of multiple drives in the system if it is configured to do so. The Tivoli Identity Manager product itself can benefit from having the log files on a different drive than the messaging queues. Keep in mind that high-end I/O backplanes or other advanced storage systems can overcome this by balancing the load across multiple drives automatically.

• Database and directory activity can be very CPU and memory intensive. It is recommended that each application have at least one processor and 2GB of RAM each. More processors are generally the better. To obtain optimal performance, it is not recommended to have all Tivoli Identity Manager components on the same system unless it is a very powerful unit such as an 8 x 1.4GHz machine with 12GB RAM.

• Individual provisioning policies should not affect more than 20,000 users. In Tivoli Identity Manager version 4.5 and version 4.5.1 when an automatic provisioning policy is modified, the complete list of affected accounts is created in memory and can result in out-of-memory errors with more than 20,000 people. Tivoli Identity Manager version 4.6 does not have this limitation.


7 Troubleshooting

Finding performance problems with IBM Tivoli Identity Manager can be complicated by the large number of middleware dependencies. A problem such as a slow DSML feed with account provisioning could be the result of a slow directory server, database locking issues, or an inadequate number of Tivoli Identity Manager workflow threads. This section is designed to assist you in identifying problem areas and some pointers on fixing them.

Information in this section is provided with the assumption that you have read and applied the tunings recommended in the previous sections. If this is not the case, stop and review those sections now.

7.1 Log files When something seems to stop working, the first place to check is the itim.log file. It often contains useful information about what is currently being processed and any errors that may have occurred. If you suspect something isn’t working correctly and the itim.log files doesn’t contain enough information, increase the logging in the enRoleLogging.properties file. Edit the file and either uncomment or add one or more of the following lines: log4j.category.com.ibm.itim.workflow=DEBUG log4j.category.com.ibm.itim.workflowextensions=DEBUG log4j.category.com.ibm.itim.remoteservices=DEBUG

The workflow and workflowextensions classes are used during all workflow processes. Setting these to DEBUG can help track down workflow problems. The remoteservices class communicates with external agents (such as a Windows NT service and a Tivoli Access Manager service but not the ITIM Service) during provisioning. Set this to DEBUG to help diagnose problems with agents. Changes to the enRoleLogging.properties file are picked up every 10 minutes or whenever the Tivoli Identity Manager application is restarted.

Also, be sure to check the log file for the appropriate middleware components:

• IBM WebSphere Application Server - /usr/WebSphere/AppServer/logs/server_name/

• BEA WebLogic – weblogic_home/user_projects/itim/server_name/server_name.log

• IBM Tivoli Directory Server - /var/ldap/ibmslapd.log

• Sun ONE Directory Server – sun_one_home/logs/access

• IBM DB2 – instance_home/sqllib/db2dump/db2diag.log

• Oracle Database - *.trc files

• Microsoft SQL Server – mssql_home\LOG\

7.2 Identifying bottlenecks The Tivoli Identity Manager application makes intense usage of both the database and the directory server it is configured into. The database is used as an information staging area and audit trail for provisioning actions. The directory server is used as a permanent storage location and can be heavily queried when evaluating provisioning policies. Because of this usage it is possible for either the database or the directory server to be a bottleneck during heavy usage or large provisioning changes.

During an action that causes a large provisioning policy to be evaluated, such as adding a new policy or modifying an existing policy that impacts a large number of users, the affected users must be queried from the directory server. The directory server evaluates the query and returns the matching users. If the


directory server is not tuned correctly it can become the bottleneck as the Tivoli Identity Manager server waits for the result set before starting the required provisioning action. Ensure the directory server is fulfilling the requested queries as quickly and efficiently as possible to minimize this behavior. Refer to the appropriate directory server tuning section for more information.

After the result set is returned, the Tivoli Identity Manager server will begin to enforce the provisioning policy for each user. This process is multi-threaded and benefits from multiple processors (the workflow cluster member in a functional cluster). If it appears that the processors on the server are not being fully utilized and no other server appears to be the bottleneck, consider increasing the maximum number of threads being used. See the IBM Tivoli Identity Manager application - Threads section for more information.

Any enforcement action required for a user (account addition, modification, or deletion) is stored as a workflow item in the database. When a thread becomes available, it queries the database for the next workflow item that needs processing and proceeds to act upon that item. This usage can result in access contention and thus locking in the database. Appropriate indexes and access-plan statistics will help minimize the number of locks needed to fulfill these requests. Refer to the appropriate database tuning section for more information.

7.3 Specific problems

7.3.1 IBM Tivoli Identity Manager application Problem: Transaction rollback errors in the itim.log file. Solution: Transaction rollbacks can occur for several different reasons, most of them database-related. There should be an error message within the itim.log file giving more information about what went wrong. Some areas to check when you get a transaction rollback:

• Database storage space. If the database runs out of storage space for the tablespaces, a transaction rollback error can occur. Increase the amount of disk space allocated to the tablespaces to resolve this problem.

• Database locking issues. If the database encounters extreme locking issues it is possible to get a transaction rollback error. Confirm that the locks are tuned appropriately and update the table statistics.

• Database memory issues. If there is not enough memory available to database structures to fulfill the requested query, a transaction rollback error may occur. The JNDI error in the itim.log file should indicate which database heap should be increased. Increase the appropriate heap using the recommendations provided in the section for that middleware.

Problem: Out of Memory errors in the itim.log file. Solution: The message is from the J2EE Application Server. The Java virtual machine (JVM) ran out of heap size. Increase the maximum heap size and restart the application server.

7.3.2 IBM WebSphere MQ Problem: Nothing seems to be going through workflow and new items are piling up on the queue. Solution: A message on the message queue may be corrupt and blocking new transactions. Clearing the queues may help.

Note: Clearing the queues can result in an inconsistent system state. It is recommended that you only resort to clearing your queues when instructed to do so by a support technician.

1. Stop the Tivoli Identity Manager application. You can’t clear the queues while the Tivoli Identity Manager application is running.


2. Find the name of the queue manager by listing all queues on the system: dspmq

3. Start the MQ Script interpreter: runmqsc WAS_machinename_jmsserver

4. List Tivoli Identity Manager queues: display queue('WQ_itim*')

5. For each queue, clear it: clear qlocal('WQ_itim_name')

Note: If you receive a message that the queue is in use, stop the Tivoli Identity Manager server and try again.

6. Exit the MQ Script interpreter: end

7.3.3 IBM Tivoli Directory Server Problem: Poor search performance. Solution: Slow queries or high CPU utilization with the IBM Tivoli Directory Server can be caused by several factors. Some areas to consider:

1. Check for long-running queries that need indexes. The IBM Tivoli Directory Server uses IBM DB2 to process LDAP queries. By checking DB2 for long-running queries it is possible to find what attributes may need indexing. To find how long each query takes, turn on statement cache monitoring within DB2: db2 update dbm cfg using DFT_MON_STMT ON Stop the directory server, restart the database, and restart the directory server for this to take effect. After the monitoring is turned on, duplicate the suspect action within Tivoli Identity Manager. After the action has completed, get a snapshot of the statement cache: db2 get snapshot for dynamic sql on database_name The output will contain stanzas like the following: Number of executions = 12 Number of compilations = 1 Worst preparation time (ms) = 3 Best preparation time (ms) = 3 Internal rows deleted = 0 Internal rows inserted = 0 Rows read = 10024 Internal rows updated = 0 Rows written = 0 Statement sorts = 0 Total execution time (sec.ms) = 136.000663 Total user cpu time (sec.ms) = 62.010000 Total system cpu time (sec.ms) = 10.000000 Statement text = SELECT distinct E.EID FROM LDAPDB2.LDAP_ENTRY AS E, LDAPDB2.LDAP_ENTRY as pchild WHERE E.EID=pchild.EID AND pchild.PEID=? AND E.EID IN (SELECT EID FROM LDAPDB2.OU WHERE OU = ?)

Calculate the average execution time per query by dividing the total execution time by the number of executions: total execution time / number of executions. In the example above: 136 / 12 = 11.33 seconds per execution. Most queries should take a second or less to complete. Queries that take longer than that may be searching on columns that are not indexed, and hence are not indexed within the IBM Tivoli Directory Server. In the above example the column OU is likely not indexed. Correct this by indexing the OU attribute within the IBM Tivoli Directory Server. See the IBM Tivoli Directory Server - Indexing


Indexing the attributes that applications search on increases IBM Tivoli Directory Server performance. IBM Tivoli Directory Server indexes are automatically translated into IBM DB2 indexes by updating the IBM Tivoli Directory Server schema for those attributes.

If you extend the LDAP schema in IBM Tivoli Directory Server to include additional attributes, index those attributes that you will search for. Any filter in the Tivoli Identity Manager application (such as with dynamic roles) is translated into a search string for the IBM Tivoli Directory Server.

The Tivoli Identity Manager application frequently searches against the o, owner and ou attributes. Index these attributes either through the IBM Tivoli Directory Server Web Administration tool or by importing the ids_indexes.ldif file using the ldapmodify command.

After updating the LDAP schema, run DB2 runstats on the database to update the statistics for the newly created indexes.


root_dn = The root DN of the IBM Tivoli Directory Server server.

root_password = The password for the root DN.

Setting the values

3) To use the LDIF, create the ids_indexes.ldif file from Appendix A – scripts and files and place it on the computer.

4) Run the following command to import the LDIF into IBM Tivoli Directory Server:

2. ldapmodify –D root_dn –w root_password –f ids_indexes.ldif section for more information. The Tivoli Identity Manager tuning scripts contain a script (perfanalyze_dynamicsql.pl) that will calculate the time per execution for all stanzas and sort the results. See Appendix A – scripts and files for information on how to get the tuning scripts.

3. Check for low bufferpool hit ratio. The IBM Tivoli Directory Server uses IBM DB2 to process LDAP queries. The IBM Tivoli Directory Server database should have a high (>95%) hit ratio. If the bufferpools are not large enough IBM DB2 will need to read more information from the disk resulting in high I/O wait. See the IBM DB2 performance monitoring - Bufferpool hit ratio section for information on calculating the bufferpool hit ratio. The Tivoli Identity Manager tuning scripts contain a script (perfanalyze_bufferpools.pl) that will calculate the hit ratio for all bufferpools for you. See Appendix A – scripts and files for information on how to get the tuning scripts.

Problem: The directory server crashes for no apparent reason. Solution: Check the size of your cache sizes. If your cache sizes cause the IBM Tivoli Directory Server process to grow beyond what is supported by your operating system’s memory model (in general around 2GB on 32-bit operating systems) it can crash. See the IBM Tivoli Directory Server - Cache sizes section for more information.

7.3.4 Sun ONE Directory Server Problem: Poor search performance. Solution: Slow queries or high CPU utilization with the Sun ONE Directory Server can be caused by several factors. Some areas to consider:

1. Check for long-running queries that need indexes. Most queries should take less than one second to complete. Queries taking longer than a second are generally searching on attributes that are not indexed. To find queries taking longer than a second, search the Sun ONE Directory server access logs for etime=X where X is an integer


greater than 1, X representing the number of seconds the query took to complete. We are searching for queries running 2 seconds or longer. For example: [26/Mar/2004:12:54:25] conn=4236 op=1 msgId=2 - RESULT err=0 tag=101 nentries=1 etime=2 Now search for what query the search time is associated with by searching the same file for the matching connection and operation number, which in the above example is conn=4236 op=1. The query for the above example was: [26/Mar/2004:12:54:24] conn=4236 op=1 msgId=2 – SRCH base="ou=ibm,dc=com" scope=2 filter="(&(!(erIsDeleted=y))(&(erEnabled=true)(erPolicyMembership=2;*))(objectClass=erProvisioningPolicy))" attrs=ALL

Examine the LDAP query string and identify the attributes being searched on. In the above example they are erIsDeleted, erEnabled, erPolicyMembership, and objectClass. Ensure that all attributes being searched on are correctly indexed. See the Sun ONE Directory Server - Indexing section for information on how to index attributes.

2. See if the All IDs Threshold value needs to be increased. If the All IDs Threshold value is too small, Sun ONE Directory Server may not be using the indexes for certain attributes. Determine if queries are searching on these attributes by searching for notes=U in the Sun ONE Directory Server access logs. The presence of this flag for a query indicates that one or more attributes being searched on is over the All IDs Threshold and should probably be increased. See the Sun ONE Directory Server - All IDs Threshold Value section for more information on how to increase this value.

3. See if the database cache hit ratio is low. If the Sun ONE Directory Server database cache is too small, the server must access the disk for the information and will result in a low database cache hit ratio. Identify the hit ratio for the database by viewing the Monitor tab for that database. The hit ratio should be 95% or higher. If it is lower than 95%, increase the size of your database cache. See the Sun ONE Directory Server - Database cache size section for information on increasing the size.


8 IBM DB2 performance monitoring

IBM DB2 provides several tools for troubleshooting and analyzing performance problems. This section contains a primer for how to get started monitoring with DB2. For more extensive information, refer to the DB2 Administration Guide: Performance book.

8.1 Enable monitoring To gather performance information, turn on the DB2 monitoring flags. To turn on monitoring, run the following commands as the database administrator. Note that this should be done for each database.

db2 update dbm cfg using DFT_MON_STMT ON db2 update dbm cfg using DFT_MON_BUFPOOL ON db2 update dbm cfg using DFT_MON_LOCK ON db2 update dbm cfg using DFT_MON_SORT ON db2 update dbm cfg using DFT_MON_TABLE ON db2 update dbm cfg using DFT_MON_TIMESTAMP ON db2 update dbm cfg using DFT_MON_UOW ON

Stop and restart the database instance for the monitoring to take effect.

8.2 Snapshots Snapshots allow you to view the internal state of various DB2 components. The following commands access important DB2 snapshots:

db2 get snapshot for database on database_name db2 get snapshot for dynamic sql on database_name db2 get snapshot for bufferpools on database_name db2 get snapshot for tables on database_name db2 get snapshot for locks on database_name

8.3 Bufferpool hit ratio The bufferpool hit ratio gives a good indication of how many data reads are coming from the bufferpool and how many are coming from the disk. The larger the hit ratio, the less disk I/O used. Calculate the bufferpool hit ratio by enabling bufferpool monitoring and taking a database snapshot. Calculate the bufferpool hit ratio with the following formula:

P = buffer pool data physical reads + buffer pool index physical reads L = buffer pool data logical reads + buffer pool index logical reads Hit ratio = (1-(P/L)) * 100%


9 Other resources

You might find the following resources useful for further tuning of the Tivoli Identity Manager middleware:

DB2 Administration Guide: Performance

http://www.ibm.com/cgi-bin/db2www/data/db2/udb/winos2unix/support/v8pubs.d2w/en_main

IBM Tivoli Directory Server Tuning Guide

http://publib.boulder.ibm.com/tividd/td/IBMDirectoryServer5.2.html

WebSphere Application Server Monitoring and Tuning Performance

http://www.ibm.com/software/webservers/appserv/infocenter.html

Sun ONE Directory Server Documentation

http://docs.sun.com/db/coll/S1_DirectoryServer_52 (Sun ONE Directory Server V5.2) http://docs.sun.com/db/coll/S1_ipDirectoryServer_51 (iPlanet Directory Server V5.1)

Sun Java Garbage Collection References

http://java.sun.com/docs/hotspot/gc/ http://java.sun.com/docs/hotspot/VMOptions.html

http://www.ibm.com/cgi-bin/db2www/data/db2/udb/winos2unix/support/v8pubs.d2w/en_main

http://publib.boulder.ibm.com/tividd/td/IBMDirectoryServer5.2.html

http://www.ibm.com/software/webservers/appserv/infocenter.html

http://docs.sun.com/db/coll/S1_DirectoryServer_52

http://docs.sun.com/db/coll/S1_ipDirectoryServer_51

http://java.sun.com/docs/hotspot/gc/

http://java.sun.com/docs/hotspot/VMOptions.html


10 Appendix A – scripts and files

The most recent version of the scripts can be obtained by going to http://www.ibm.com/support/us. Select “Search technical support”, type “ITIM Tuning Guide” in the box, and click submit. Most of the scripts will need to be customized for your system.

10.1 perftune_runstats.sh #!/bin/sh # perftune_runstats.sh # Last Updated: 2004/03/04 14:24 CST # Desription: # Shell script to run the db2 runstats command on all tables for a given # database and schema. It autodetects the db2 version using db2level # and passes the appropriate string to runstats to allow writes to occur # during the runstats. # Usage: # This script should be run as the user for the database (ie: one that # has connect and runstats abilities and permissions). if [ $# -eq 0 ]; then DATABASE=ldapdb2 SCHEMA=LDAPDB2 elif [ $# -eq 1 ]; then DATABASE=$1 SCHEMA=LDAPDB2 else DATABASE=$1 SCHEMA=$2 fi OPTIONS="with distribution and detailed indexes all" # Find out DB2 version for runstats syntax if db2level | grep "DB2 v7" > /dev/null; then ACCESS="shrlevel change" echo Detected DB2 major version 7 elif db2level | grep "DB2 v8" > /dev/null; then ACCESS="allow write access" echo Detected DB2 major version 8 fi # Connect to the database echo Connecting to $DATABASE db2 connect to $DATABASE # Execute runstats on all tables echo Performing runstats on all tables for schema $SCHEMA echo " with options: $OPTIONS $ACCESS" for i in `db2 connect to $DATABASE > /dev/null; db2 list tables for all | \ grep $SCHEMA | cut -d' ' -f1` do echo Table: $SCHEMA.$I db2 runstats on table $SCHEMA.$i $OPTIONS $ACCESS done # If the database we're tuning is an LDAP database, # update LDAP_DESC and LDAP_ENTRY stats in the statistics table if [ "$SCHEMA" = "LDAPDB2" ]; then echo Updating LDAP_DESC and LDAP_ENTRY stats in the statistics table db2 "update sysstat.tables set card = 9E18 where tabname = 'LDAP_DESC'" db2 "update sysstat.tables set card = 9E18 where tabname = 'LDAP_ENTRY'" fi

10.2 ids_indexes.ldif #---------- beginning of ids_indexes.ldif ---------

# ids_indexes.ldif # - Additional indexes for ITIM's use with IDS # # Usage:

http://www.ibm.com/support/us


# ldapmodify -D cn=root -w <password> -f ids_indexes.ldif #

dn: cn=schema changetype: modify replace: attributetypes attributetypes: ( 2.5.4.10 NAME ( 'o' 'organization' 'organizationName' ) DESC 'This attribute contains the name of an organization (organizationName).' SUP 2.5.4.41 EQUALITY 1.3.6.1.4.1.1466.109.114.2 SUBSTR 2.5.13.4 ) - replace: ibmattributetypes ibmattributetypes: ( 2.5.4.10 DBNAME ( 'o' 'o' ) ACCESS-CLASS NORMAL LENGTH 128 EQUALITY ORDERING )

dn: cn=schema changetype: modify replace: attributetypes attributetypes: ( 2.5.4.32 NAME 'owner' DESC 'Identifies the distinguished name (DN) of the person responsible for the entry.' SUP 2.5.4.49 EQUALITY 2.5.13.1 SYNTAX 1.3.6.1.4.1.1466.115.121.1.12{1000} ) - replace: ibmattributetypes ibmattributetypes: ( 2.5.4.32 DBNAME ( 'owner' 'owner' ) ACCESS-CLASS NORMAL LENGTH 1000 EQUALITY ORDERING ) dn: cn=schema changetype: modify replace: attributetypes attributetypes: ( 2.5.4.11 NAME ( 'ou' 'organizationalUnit' 'organizationalUnitName' ) DESC 'This attribute contains the name of an organization (organizationName).' SUP 2.5.4.41 EQUALITY 1.3.6.1.4.1.1466.109.114.2 SUBSTR 2.5.13.4) - replace: ibmattributetypes ibmattributetypes: ( 2.5.4.11 DBNAME ( 'ou' 'ou' ) ACCESS-CLASS NORMAL LENGTH 128 EQUALITY )

#------------------- end of LDIF ------------------

Documents

IBM Tivoli Identity Manager Performance Tuning Guidepublib.boulder.ibm.com/tividd/td/ITIM/SC32-1496-02/en_US/PDF/ITIM… · IBM Tivoli Identity Manager Performance Tuning Guide Page