BMC Bladelogic

BMC BladeLogic Server Automation Best Practices for Deployment and Configuration

Page 2

BMC BladeLogic Server Automation Best Practices for for Deployment and Configuration

REVISION HISTORY

Date Product version Revisions

March, 2011 8.1.00 Initial version.

Page 3


TABLE OF CONTENTS

Revision history ............................................................................................................................................................... 2

Understanding job behavior ............................................................................................................................................ 4

Job execution framework ............................................................................................................................................ 4

Compliance ................................................................................................................................................................. 5

Component Discovery Jobs ........................................................................................................................................ 6

NSH Script Jobs ......................................................................................................................................................... 7

Deploy Jobs ................................................................................................................................................................ 7

Provision Jobs ............................................................................................................................................................ 9

Patching Jobs ............................................................................................................................................................. 9

Virtualization Jobs ..................................................................................................................................................... 11

Administrative jobs .................................................................................................................................................... 12

Deployment guidance ................................................................................................................................................... 13

BMC BladeLogic Server Automation components .................................................................................................... 13

Simple installations ................................................................................................................................................... 16

Large-Scale installations ........................................................................................................................................... 17

Geographically-distributed installations .................................................................................................................... 19

Configuration guidance ................................................................................................................................................. 20

About Java memory .................................................................................................................................................. 20

About thread pools .................................................................................................................................................... 21

About database connections .................................................................................................................................... 21

Recommendations for job servers ............................................................................................................................ 22

Recommendations for Configuration servers ............................................................................................................ 24

Recommendations for NSH Proxy servers ............................................................................................................... 26

Appendix: TCP/UDP Port Usage ................................................................................................................................... 27

Page 4


UNDERSTANDING JOB BEHAVIOR

This section provides a brief overview of the runtime behavior of the various job types, with emphasis on computation, storage, and network resource requirements.

JOB EXECUTION FRAMEWORK

All BMC BladeLogic Server Automation jobs execute in the job execution framework. This section describes the overall operation of the framework.

Jobs, targets, and work items

The execution of a BMC BladeLogic Server Automation job begins with the job being placed in a work queue of jobs waiting to execute. Each job server maintains a pool of threads, called the job thread pool, dedicated to executing jobs in the job queue.

The main work of a job is the creation and management of individual work items and their results. Work items

are separately-schedulable units of work that are undertaken as part of the execution of a job. It is in the execution of work items that the job carries out its responsibilities.

Like jobs, work items are executed by job servers. A work item may or may not be executed by the same job server that is responsible for executing the job that created it. Work items scheduled for execution are maintained in a work item queue in the job server. A pool of threads, known as work item threads, is

maintained by each job server for the execution of work items.

A work item is almost always bound to one target host, or to a component on a target. Further, a job may generate multiple work items for each target, corresponding to different steps or stages of the job. Thus, a job that is scheduled to execute against 1,000 target hosts can create and schedule execution of possibly several thousand work items.

Lightweight work items

Some work items are designated as lightweight work items because their execution consumes significantly fewer server resources than does the execution of normal work items. In addition to the work item thread pool, each job server also maintains a separate thread pool for lightweight work items, the lightweight work item thread pool. If the lightweight work item thread pool is not empty (that is, has a non-zero size), threads from this pool manage lightweight work items; otherwise, all work items (lightweight and nonlightweight) are managed by the normal work item thread pool.

Asynchronous BlExec tasks

While a work item is executing, a work item thread is assigned exclusively to that work item. Some work items, however, spend much of their time waiting for results from operations being carried out remotely, for example, on a target host. A work item thread assigned to such a work item blocks while it waits. This can present a potential resource issue, as the work item thread is not available to service other work items with more active processing needs.

This resource utilization concern is addressed by the introduction of asynchronous tasks. Asynchronous tasks are a relatively new feature of the job execution framework, having been introduced in BMC BladeLogic Server Automation version 8.0. When a work item must perform a remote operation, instead of performing that remote operation and waiting for a response, the work item can instead create and queue an asynchronous BlExec task to perform the operation. Then the work item itself terminates. Asynchronous BlExec tasks are managed

by the BlExec service, which maintains yet another thread pool, but the tasks occupy a thread from the pool only when they have active processing to perform. While it waits for a response from the remote target, an asynchronous BlExec task does not consume any thread resources.

A work item must be explicitly written to make use of asynchronous tasks; work items whose implementation is not asynchronous task aware still perform remote operations directly and cause their work item threads to wait for the operation to complete. Currently, some BMC BladeLogic Server Automation jobs are asynchronous task aware, and some are not.

Page 5


Version 7.6 and earlier

The BlExec service and asynchronous BlExec tasks are not available earlier than BMC BladeLogic Server Automation 8.0.

Version 8.0 In BMC BladeLogic Server Automation 8.0, NSH Script jobs and patch analysis jobs take advantage of asynchronous BlExec tasks.

Version 8.1 In BMC BladeLogic Server Automation 8.1, Deploy jobs and Virtual Guest Jobs also take advantage of asynchronous BlExec tasks.

COMPLIANCE

This section describes Snapshot, Audit, and rules-based Compliance Jobs.

Snapshot Jobs

Snapshot jobs collect information about assets from a target and convey that information to the Application Server. As it is received by the Application Server, information about each asset is stored in a local (on the Application Server) file with a .snp (snapshot) suffix. The lone exception to this rule is that when a file’s content is included in a snapshot, the file’s content is received as a separate file and not included in the .snp file. One .snp file is constructed for each component part.

When construction of the .snp file is completed on the Application Server, it is then scanned and compared to the most recent prior snapshot from that target, if one exists. This comparison is performed between the new .snp file, and a .bnp (baseline snapshot) representing the previous scan. The exact means by which the .bnp file is obtained depends on the BMC BladeLogic Server Automation version; however, processing steps follow this flow:

Only in version 7.6 and earlier

If the last scan of this particular target ran on the same job server and the old .bnp from that job run is still available on the job server, then the old .bnp file is used directly. Otherwise, the next processing step takes place.

Version 8.0; Version 7.6 and earlier

If the .bnp file from the last scan of this particular target is available on the file server (see below), it is copied from the file server to the local Application Server and then used for comparison. Otherwise, the next processing step takes place.

The job server uses the data in the database to reconstruct a .bnp file as it would have existed following the last scan of this particular target.

Version 8.0; Version7.6 and earlier

After the snapshot job executes, the .snp file is renamed to .bnp and copied to the file server.


The .bnp file also remains on the job server. (In BMC BladeLogic Server Automation version 8.0 and later, the file is deleted).

After the comparison between the two .bnp files (that is, the .bnp of the current snapshot of the asset and the .bnp file from the prior snapshot), only the differences between the two versions are stored in the BMC BladeLogic Server Automation database. Therefore, repeated snapshots of an asset that changes very little or not at all result in relatively little information being stored in the database. For example, in the case of an unchanged file, no additional data is recorded in the database.

Files are the most common type of asset used in Snapshot and Audit Jobs. To perform a full snapshot of a file, the entire contents of that file must be transferred to the Application Server. On the other hand, capturing just the MD5 digest (checksum) of a file, instead of the actual contents, results in the calculation of the 128-bit MD5 digest value on the target and transmission of only that 128-bit value to the Application Server for storage in the .snp file.

The table shows a summary of resource usage for Snapshot Jobs.

Application Server CPU Network Traffic Database Load Agent

High High Moderate Low

Page 6


Audit Jobs

Audit Job behavior is largely similar to that of Snapshot Jobs, in that it involves constructing and comparing two snapshot files on the Application Server (with a .snp suffix this time), and recording only the differences.

For Audit Jobs, master .snp files are always constructed as part of the job. For live audits, a snapshot of the master target is performed and the results captured in .snp files. For a snapshot-based audit, the master .snp file is generated from data in the database. The master .snp files are copied to the file server, if necessary, and shared among any Application Servers that run work items for the Audit Job.

After the master .snp files have been constructed, each target of the audit job is processed by first constructing a target .snp file and then comparing the master and target .snp files. Differences between the two .snp files are recorded in the database. For each difference detected, the asset from the target .snp file is persisted in the database, regardless of how many earlier audits may have detected the same difference. After each audit target is processed, the target .snp files for that target are discarded.

Upon completion of the Audit Job, master .snp files are marked for deletion on the file server.

The table shows a summary of resource usage for Audit Jobs.



Compliance Jobs

Compliance jobs, also called rule-based compliance jobs, operate by:

collecting asset information on the target

transferring that data back to the Application Server

applying the user-specified rules to the returned data to assess the target’s compliance

Unlike Audit and Snapshot jobs, Compliance Jobs do not use temporary snapshot files (.snp or .bnp) on the Application Server.

When a Compliance Job runs, each work item operates by looping through the component parts of its assigned component; for each component part, it decides whether or not it needs to retrieve data from the target. If it does need to retrieve assets for the current component part, a single request is issued to collect the required information for each of the assets to be tested. As each requested asset arrives on the Application Server, the relevant conditions are applied to determine compliance.

The result (compliant, noncompliant, or noncompliant with exception) of applying each condition is recorded in the database. In the case of a noncompliant result, the specific value under test is also recorded in the database. For example, a non-complying condition on file size causes the actual file size to be recorded in the database, but not the contents of the file.

Compliance autoremediation

If a target is noncompliant and if the Compliance Job has the Allow Auto-remediation option specified, then each rule failure selects a BLPackage to be included in a combined remediation BLPackage for that host. (Any other hosts with the same combination of failing rules will use the same remediation package.) The Compliance Job then runs a BLPackage Deploy Job against the noncompliant targets. The Compliance Job does not complete until the BLPackage Deploy Job has completed.

The table shows a summary of resource usage for Compliance Jobs that perform autoremediation.

Application server CPU Network Traffic Database Load Agent

High Moderate – High Moderate Moderate

COMPONENT DISCOVERY JOBS

Component Discovery Jobs first use component applicability rules to select appropriate targets from the requested list of targets. Each suitable target is then contacted and sufficient assets collected to perform a test of the signature condition for the target.

The table shows a summary of resource usage by Component Discovery Jobs.

Page 7



Low – Moderate Low – Moderate Moderate Low

NSH SCRIPT JOBS

Scripts executed by NSH Script Jobs are categorized by the four radio buttons presented in the job’s Add Script dialog:

Execute the script separately against each host (Type 1)

Execute the script once, passing the host list as a parameter to the script (Type 2)

Copy and execute the script against each host separately (Type 3)

Execute the script using the PERL interpreter, passing the host list as a parameter (Type 4)

From a performance perspective, Type 3 jobs differ from the other types in that they execute the script on the target, rather than on the Application Server. Even for scripts executed on the Application Server however, any nexec commands are executed on the target.

Version 8.0 and later

As of BMC BladeLogic Server Automation version 8.0, users have the option of selecting asynchronous execution for Type 3 NSH Script Jobs. Choosing this option causes the job to be executed using asynchronous BlExec tasks; see Asynchronous BlExec tasks on page 4.

Process Spawner

NSH Script Jobs invoke the actual NSH scripts in a separate process. That separate process can be created and managed either by the Application Server or by a separately-running application known as the Process Spawner. Use of the Process Spawner can significantly reduce the overhead of creating and tearing down the process used to execute the NSH script.

Version 8.0; Version 7.6 and earlier

Prior to BMC BladeLogic Server Automation version 8.1, use of the Process Spawner can result in deadlocks or hangs under high workloads. BMC does not recommend using the Process Spawner in these versions.


Issues with deadlock and hangs are resolved in release 8.1. Use of the Process Spawner offers significant performance benefits for NSH Script jobs. BMC recommends its use.

The table shows a summary of resource usage by the Process Spawner.


Varies Varies Low Varies

DEPLOY JOBS

This section describes file and package deploy jobs.

File Deploy Jobs

A File Deploy Job arranges to deploy a file from any NSH-accessible location to one or more remote targets. A File Deploy Job operates by first constructing and then executing an .nsh script to copy (push) the requested file. This script runs on the Application Server.

For a direct File Deploy Job, the script running on the Application Server copies the file from its source to each target. The Application Server acts as an intermediary; there is no direct data transfer between the source and the target.

For an indirect File Deploy Job, the script running on the Application Server copies the file to one or more remote repeaters, with the Application Server again acting as an intermediary; a script then runs on each repeater to push the file to the final target.

Page 8


When pre-commands or post-commands are specified as part of a File Deploy Job, these commands are executed on the remote target.

The table shows a summary of resource usage by File Deploy Jobs.


Moderate High Moderate Moderate

The server from which the files are deployed can experience heavy load during a File Deploy Job. Similarly, any Repeaters involved can experience heavy load during a File Deploy Job.

BLPackage Deploy Jobs

A BLPackage is an aggregation of many types of server objects, including, for example, registry keys and configurations within files, not just files. These server objects are packaged together for unattended deployment on multiple remote hosts.

Phases of the BLPackage Deploy Job

BLPackage Deploy Jobs comprise a sequence of work items run in the following phases:

Phase Work Item Description Notes

Simulate This is a dry run or preflight phase to verify that conditions exist which should lead to a successful execution.

Asynchronous BlExec task

Staging Any necessary files are copied to the target in preparation for deployment, possibly by way of repeater servers. If the package uses the agent mounts source option, no staging is required.

High file server load

Commit

Execute Pre commands, if any, on the target. Lightweight Work Item

Run installation commands on the target. Lightweight Work Item, Asynchronous BlExec task

Execute Post commands, if any, on the target. Lightweight Work Item

Undo If the deployment is unsuccessful, its effects are reverted on the target.

Asynchronous BlExec task

The Staging phase has the potential to generate significant workloads on the file server (or other server providing the package source files). The Commit phase, in contrast, presents almost no load to the BMC BladeLogic Server Automation infrastructure, as most of the work for this phase is carried out on the target hosts.

With the exception of work items for predeploy and postdeploy commands, work items for the BLPackage Deploy Job’s Commit phase are implemented as Lightweight Work Items. See Lightweight work items, page 4.


As of BMC BladeLogic Server Automation version 8.1, several phase work items have been enhanced to use asynchronous BLExec tasks for execution, allowing for increased throughput even without populating the thread pool for lightweight work items. See Asynchronous BlExec tasks, on page 4.

The table shows a summary of resource usage by BLPackage Deploy Jobs.


Low High Low Low

The Staging phase has the potential to generate significant workloads on the file server (or other server providing the package source files. The Commit phase, in contrast, presents almost no load to the BMC BladeLogic Server Automation infrastructure, as most of the work for this phase is carried out on the target hosts

Page 9


PROVISION JOBS

A Provision Job establishes the necessary network resources required for a target machine to be provisioned upon reboot; then the job monitors the progress of the provisioning activity as it occurs on the target. As the target device reboots, it requests progressive instructions from BMC BladeLogic servers and downloads boot images and operating system installation files from servers on the network. The device’s boot process varies, depending on the type of target device, but in all cases, as the target device is rebooting, it requires network access to servers from which it can retrieve instructions and downloadable artifacts.

Provisioning details

Windows and Linux provisioning

Provisioning support for Windows and Linux devices is based on the Pre-Execution Environment (PXE) standard. When booting under the control of a provisioning job, a Windows or Linux target contacts the following:

DHCP server, to identify a PXE server

PXE server, for initial booting instructions

TFTP server, from which it downloads a pre-boot kernel image, in most cases.

Application Server, from which it retrieves the system package

Data store, for operating system installation files

Generally speaking, none of these activities impose significant computational demands on the supporting servers, but the network link between the target device and the data store server may experience substantial bandwidth usage.

Solaris provisioning

The Oracle JumpStart technology used for provisioning Solaris machines relies on three separate JumpStart functions:

JumpStart Boot Server

JumpStart Install Server

JumpStart Configuration Server

These functions may be provided independently, or combined into one or two actual JumpStart servers. The boot server must be on the same network as the provisioning target. Of the three functions, the install server (the data store) bears the greatest load.

AIX provisioning

The IBM AIX Network Installation Manager (NIM) technology uses a NIM master to control the provisioning target, while installation files are served off an NFS server (data store).

HP-UX provisioning

The HP-UX Ignite technology uses a single Ignite master to control the provisioning target and to provide the operating system installation files.

The table shows a summary of resource usage by Provision Jobs.


Very Low Low Low Low

Provisioning servers (whatever the type) must be available to the target host being provisioned. The data store server may experience moderate to high load during provisioning.

PATCHING JOBS

In BladeLogic, software patches released by a patch vendor (that is, Microsoft, Red Hat, Adobe) are conceptualized as comprising metadata, describing the patch and its applicability, and a payload, containing the actual bits of the patch.

Page 10


Patches are stored in a repository in the computing environment, which is an NSH-accessible directory somewhere in the BladeLogic environment. An offline or air-gapped environment is one in which the repository does not have direct access to the internet and therefore patches cannot be directly downloaded from the vendor site to an offline repository. A common strategy for populating an offline repository is to transfer patch content on removable media with the help of a BMC-provided download utility.

Patches are organized into patch catalogs, according to filters defined in BladeLogic. For example, the catalog for Windows patches is separate from the catalog for Red Hat patches.

If a repository is to include Windows patches, and the Application Server is running on a Linux host, then you must identify a Windows Helper Server Location when you create the repository. The Windows Helper Server Location is a user-defined temporary directory on a Microsoft Windows server which is used to decrypt files downloaded from the vendor site.

Catalog Update Jobs

You can create Catalog Update Jobs for each type of patch repository. Online patch catalogs are updated by downloading additional content from vendor and/or metadata-provider websites. The Application Server running a catalog update job for an online repository requires web access to these sites (that is, the Application Server must be configured to allow traffic to pass through any firewalls and web proxy servers).

Offline patch catalogs are updated by transferring content from a local server, which typically mounts removable storage media onto which patch information is already loaded.

The table shows a summary of resource usage by Catalog Update Jobs.



Patch Analysis Jobs

On all supported platforms as of BMC BladeLogic Server Automation version 8.0, patch analysis processing takes place on the affected target. That is, the relevant metadata (typically less than 5 MB) is transferred from the repository to the target agent, where it is decoded; the target agent then performs the necessary calculation to determine which patches to install on the target.


Prior to BMC BladeLogic Server Automation version 8.0, patch analysis for Solaris was performed primarily on the Application Server, rather than on the target. This can present a moderate to high work load on the Application Server.

Version 8.0, Version 8.1 and later

As of BMC BladeLogic Server Automation version 8.0, patch analysis for Solaris now occurs on the target. Further, patch analysis for all target types now uses an asynchronous agent call, allowing greater concurrency on the Application Server.

The table shows a summary of resource usage by Catalog Update Jobs.


Low* Low Low Moderate – High*

*See version-specific notes, above, for Solaris patch analysis.

Patch Remediation Jobs

A patch remediation job does the following:

Runs a patch download job to download patch payloads of missing patches that have not yet been downloaded. For Windows and Solaris, you can run analysis with just the metadata, without downloading the payload. Red Hat requires a payload download. So you can create a Windows patch catalog with all Windows 2008 patches and only download payloads of the patches that are found missing.

Based on the patch analysis results, the patch remediation job runs an algorithm that creates a set of BLPackages and BLPackage Deploy Jobs. If multiple servers have the same set of missing patches, the patch remediation job creates a single Deploy Job with BLPackages that target the servers. If different

Page 11


servers have different patches missing, the patch remediation job creates a Deploy Job for each unique set of missing patches.

The BLPackage Deploy Jobs are wrapped into a Batch Job.

The Batch Job then executes immediately (if specified), or is scheduled to execute at a later time.

The table shows a summary of resource usage by Patch Remediation Jobs.


Low High Low Low

See BLPackage Deploy Job, above, for more information about the resource demands of the deploy operations. Patch resources are stored in the patch repository, which may experience heavy workload during the Staging phase of deployment.

VIRTUALIZATION JOBS

Virtual Guest Job

A Virtual Guest Job constructs a virtual guest, with or without an operating system, from a known VCenter or other virtual infrastructure. Configuration decisions for the new virtual guest are captured in a Virtual Guest Package.

Internally, Virtual Guest Jobs operate as BLPackage Deploy Jobs, so the behavior and resource demands of a Virtual Guest Job correspond to those of the Deploy Job. Virtual Guest Jobs require minimal Application Server resources.

For some steps in its operation, a Virtual Guest Job communicates with the VCenter through a custom object (CO) that must be installed on the Virtual Center host.

The table shows a summary of resource usage by Virtual Guest Jobs.



Virtual Guest Jobs make demands on the Virtual Center host to accomplish construction of the virtual guest.

Virtual Infrastructure Discovery Job

Also called a sprawl job, a Virtual Infrastructure Discovery Job scans the network to identify ESX servers or other virtual hosting environments and then interrogates them to identify guests hosted by that computer.

The table shows a summary of resource usage by Virtual Infrastructure Discovery Jobs.


Low – Moderate Moderate Low – Moderate Low

UCS Provisioning Jobs

A Cisco Unified Computing System (UCS) chassis comprises a number of hardware blades which act as a pool of computing resources. The chassis also includes a hardware entity (the Fabric Interconnect) that manages all the computing, network, and storage connectivity resources. The BladeLogic UCS custom object (CO) communicates with this hardware entity.

A UCS template is a configuration that contains settings to configure a blade to become a server. This template contains server identity information (MAC address), networking configuration, and storage configurations (WWNN and WWPN). The UCS Provisioning Job takes a predefined template, applies the template to a stateless blade (so that the blade becomes a server with an identity), and then provisions the server.

The table shows a summary of resource usage by UCS Provisioning Jobs.

Page 12




UCS Provisioning Jobs make demands on the UCS Fabric Interconnect to accomplish the actual construction of the virtual guest.

ADMINISTRATIVE JOBS

Update Server Properties Job

The Update Server Properties Job invokes miscellaneous remote commands to obtain server name, operating system type and version, IP address, etc.

The table shows a summary of resource usage by Update Server Properties Jobs.


Low Low Low Low

ACL Push Jobs

At its core, an ACL Push Job computes a set of entries for the user file on each target, and then overwrites the user file (the file in the target’s rsc directory whose name is ‘user’) with those entries.

The table shows a summary of resource usage by ACL Push Jobs.


Low Low Low Low

Distribute Configuration Objects Jobs

The table shows a summary of resource usage by Distribute Configuration Objects Jobs.


Low Low – Moderate Low Low – Moderate

Decommission Configuration Object Jobs

The table shows a summary of resource usage by Decommission Configuration Object Jobs.


Low Low Low Low

Page 13


DEPLOYMENT GUIDANCE

A BMC BladeLogic Server Automation deployment typically involves a large number of individual software elements arrayed across a number of physical servers deployed around the environment. This section discusses performance and other considerations for the deployment of the various BMC BladeLogic Server Automation software elements.

BMC BLADELOGIC SERVER AUTOMATION COMPONENTS

This section describes the components that may constitute a BMC BladeLogic Server Automation installation.

Database server

At the center of every BMC BladeLogic Server Automation installation is the BMC BladeLogic database server. Application Servers are tightly coupled to the database and impose significant demands on the server that hosts the database. BMC recommends the use of a dedicated physical machine or cluster to host the database server for BladeLogic.

The database server or cluster should be on the same LAN as the BMC BladeLogic Server Automation Application Server. High latency on the link between the Application Servers and the database server can cause unacceptable performance for BladeLogic. Additionally, there is anecdotal evidence that high packet loss rates on the Application-Server-to-database link may cause issues for (expose defects in) the Oracle JDBC driver.

Application Servers

A BMC BladeLogic Server Automation deployment comprises one or more Application Server (appserver) processes. The number and configuration of Application Servers in a deployment depends on many factors, including the number of targets to be managed and the expected job load for the system. Within limits, it is acceptable to run multiple Application Servers on a single physical server while still maintaining acceptable performance. For more information, see Adding Application Server instances, on page 17.

An Application Server can fulfill any of several distinct profiles, or combinations of profiles, depending on its configuration.

Configuration servers

BMC BladeLogic Server Automation clients (rich client UI, BLCLI command line client, bmi.exe) connect to configuration servers to allow interaction with the BladeLogic system. A configuration (UI) server is an Application Server of type CONFIGURATION of type ALL (which includes CONFIGURATION). A configuration server provides middle-tier functionality, answering requests from BMC BladeLogic Server Automation client applications both for data and for operations on that data.

Job servers

Application Servers configured as job servers are responsible for the execution of BMC BladeLogic Server Automation jobs.

In many environments, job servers are limited by internal resource contention, but, surprisingly, do not impose excessive workload on the hardware. Accordingly, it may be advisable to configure multiple job servers on the same physical machine in order to make more complete use of the available hardware resources. Alternatively, in a virtualized environment, you can run multiple job server guest VMs on the same physical server.

NSH Proxy Servers

NSH Proxy Servers perform a specialized role in BladeLogic installations, as described in NSH proxies, page 16.

Page 14


Process Spawners

A BladeLogic Process Spawner offers improved performance for NSH Script jobs under certain circumstances. See Process Spawner considerations, page 24.

PXE servers

A BMC BladeLogic Server Automation PXE server, technically a specially configured Application Server, performs a specialized role in support of provisioning jobs. PXE servers are discussed in Servers for provisioning, page 14.

Authentication servers

Although not a separate Application Server profile, at least one Application Server in a BMC BladeLogic Server Automation environment must be configured as an authentication server. As its name implies, the authentication server verifies the identity of a BMC BladeLogic Server Automation user, after which the user is allowed to interact with the BladeLogic client.

Authentication servers do not normally experience a high work load. A typical practice is to configure one of the configuration Application Servers also to act as an authentication server. It is not normally necessary to configure more than one authentication server for a single BladeLogic environment.

File server

Every BMC BladeLogic Server Automation environment includes a server designated as the file server. Any server running an RSCD agent can be designated as the file server for the installation. Both the performance of the file server and the network connection between job servers and the file server have a critical impact on Deploy Jobs.

Using NFS as a file server

Because NFS sharing provides higher performance than NSH data transfer, BMC BladeLogic Server Automation performance can be enhanced by employing an NFS-based network-attached storage (NAS) device and mounting the storage on each physical computer hosting an Application Server. In this configuration, localhost should be designated as the file server, so that each Application Server treats the shared mount point as local storage.

Repeaters

For environments in which deploy job performance over the WAN is a concern, BMC recommends the use of one or more repeaters at each data center. Properly configured, a repeater serves as a staging location at each site for packages as they are deployed.

Advanced Repeaters

An Advanced Repeater server is simply a repeater that uses BMC BladeLogic Advanced Repeater technology to enable file servers and repeater servers to store and share data more efficiently. BMC recommends the use of advanced repeaters whenever repeaters are deployed across a WAN.

BMC BladeLogic Server Automation consoles

Communication between the BMC BladeLogic Server Automation console and Application Servers requires significant bandwidth. BMC recommends deploying consoles to servers on the same LAN as the Application Servers to which they connect. It is possible, but not recommended, for the console and Application Server to be separated by a longer network link, but performance under that configuration may be unacceptable.

For environments in which a population of geographically-dispersed users must all have access to the BMC BladeLogic Server Automation console, BMC recommends running the console on a Citrix Presentation Server. This allows users who are offsite from the presentation server to run remote instances of the UI without experiencing excessive latency.

Servers for provisioning

BMC BladeLogic Server Automation provisioning works with different provisioning technologies, depending on the type of server being provisioned.

Page 15


Windows and Linux provisioning

PXE servers support Windows and Linux provisioning jobs by providing boot-time services to target devices. Each target device needs to have access to a local PXE server, usually on the same LAN.

Although PXE servers do communicate with the database, the data volume of that communication is relatively low. It is therefore acceptable to install geographically-removed PXE servers, which must communicate with the database over longer network legs.

The PXE server and the TFTP server must reside on the same physical server.

A provisioning target also needs access to the BladeLogic Application Server and a data store. Traffic to the Application Server is relatively light, so it is not necessary for the Application Server to be geographically proximate to the provisioning target. However, if the provisioning target will be retrieving files from a data store, it is preferable that the data store be local to the provisioning target.

Solaris provisioning

The Oracle Solaris JumpStart technology identifies a JumpStart boot server, a JumpStart config server, and a JumpStart install server, all of which may be (and commonly are) hosted on the same physical device. JumpStart servers should be located on the same LAN as the Solaris servers being provisioned, but may be remote from the BMC BladeLogic Server Automation Application Server. To use a JumpStart server with provisioning jobs, you must install an RSCD agent on the JumpStart server.

AIX provisioning

The IBM AIX NIM technology requires a NIM Master server on the same LAN as the AIX servers being provisioned. To use a NIM Master with provisioning jobs, you must install an RSCD agent on the NIM Master server.

Page 16


HP-UX provisioning

The HP-UX Ignite technology requires an Ignite Master server on the same LAN as the HP-UX servers being provisioned. To use an Ignite Master with provisioning jobs, you must install an RSCD agent on the Ignite Master server.

Proxies

NSH proxies

Historically, NSH proxy servers played an important role in negotiating fire walls in large scale deployments, but this practice is no longer recommended. In BMC BladeLogic Server Automation versions 7.x and later, NSH proxies are used mainly as a security enhancement measure.

SOCKS proxies

To access targets that are behind a firewall (because they are in a remote data center, for example) or otherwise not directly accessible from the Application Servers, BMC recommends using SOCKS Proxy Servers. In this situation, establish a SOCKS Proxy Server in each remote data center and configure any intervening firewalls to allow the Application Servers to contact the SOCKS proxy over port 1080. Configure the Application Server to establish communications with the remote targets by using the SOCKS proxy, usually over port 4750, rather than contacting the remote hosts directly.

A SOCKS proxy normally requires minimal computing power but can be expected to have network bandwidth demands commensurate with its role as a communication concentrator for the remotely-managed targets.

SIMPLE INSTALLATIONS

This section describes basic considerations applicable to all BMC BladeLogic Server Automation installations. See also Large-Scale installations on page 17, and Geographically-distributed installations, page 19, for additional considerations.

You can set up a very small-scale installation using just two physical management servers to host BMC BladeLogic Server Automation. One computer is dedicated to hosting the database, while the other hosts all the essential BladeLogic components:

Application Server offering:

Job server

Configuration (UI) server

Authentication server

File server

Management console UI (BMC BladeLogic Server Automation console)

This simple installation highlights the fact that BMC BladeLogic Server Automation makes significant use of the associated database. Accordingly, BMC recommends a dedicated database server or cluster to support a BMC BladeLogic Server Automation installation.

Page 17


For demonstration or other specialized purpose, it is possible to host all the components on one machine, including the database server. However, doing so is likely to lead to unacceptable performance in most cases.

LARGE-SCALE INSTALLATIONS

Most customer environments are too large to be managed by the simple 2-server infrastructure described in the previous section. Fortunately, you can add BMC BladeLogic Server Automation components to provide greater management capacity. This section describes the use of additional infrastructure to provide greater capacity for a BMC BladeLogic Server Automation installation.

Adding Application Server instances

To meet the demands of a larger data center, it is likely that additional Application Servers will need to be deployed. Most commonly it is necessary to add job servers to provide support for a larger number of managed servers. In some cases, it may also be necessary to deploy additional Configuration (UI) servers to support a larger user population.

Increasing job throughput

To execute more jobs against more targets in a given period of time, it is usually necessary to increase the number of work item threads (WITs) available to execute jobs. The number of WITs is a configurable option of each job server, but the number of WITs per job server is normally limited by the amount of memory available in a single Application Server. In most cases, then, adding WITs means configuring another Application Server.

It is frequently the case that a physical server has CPU and other resources sufficient to host several times the total number of WITs that can be run in a single Application Server. A rule of thumb is to install Application Servers on physical servers based on the assumption that each Application Server requires:

Two CPU cores

Physical memory sufficient for the Application Server process (4 GB for a 32-bit Application Server and 8-10 GB for a 64-bit Application Server).

Under these guidelines, a typical eight-core server computer with sufficient memory can support three to four Application Servers. In figuring required RAM for the physical server, remember to allow memory for the operating system and for other processes running on the computer, including the Application Server launcher.

See Configuration guidance on page 20 for more detailed suggestions on memory and WIT settings for job servers.

Page 18


Support for more users

For environments supporting a large user population, you may need to increase the number of Configuration (UI) Servers in the installation. The workload required to support a user varies widely, of course, but as a starting point, in the absence of additional information, BMC typically recommends:

Install one Configuration server for every 50 concurrent logged-in users.

Expect as many as 20% of total users to be logged in at any one time.

In combination, these guidelines call for one Configuration server for every 250 users.

Limits to growth

Neither Oracle nor SQL Server has a theoretical limit on the number of database connections that a database server can support. However, the actual physical resources available on the database server impose a practical limit on the number of database connections that that particular database server can maintain. This, in turn, limits the total number of Application Servers a particular BMC BladeLogic Server Automation implementation can support.

You can control the minimum and maximum number of database connections maintained by an Application Server through user-configurable settings for the various database connection pools. If you plan to establish an extremely large BMC BladeLogic Server Automation implementation, you should use the information in the Configuration Guidance section to estimate the total number of database connections required for the implementation. Then work with the local DBA and database vendor to ensure that the database server is capable of supporting that load.

Scaling the file server

The BMC BladeLogic Server Automation design requires a designated File Server to host the files in the BladeLogic Depot. The File Server is simply a server running the RSCD agent.

A NAS filer using NFS or SMB can act as a kind of virtual file server, a configuration offering several benefits in terms of performance and scalability. In this configuration, a share exported by the filer is mounted at the same mount point on each computer hosting an Application Server, making the share appear to be local storage for each Application Server. The File Server is then defined to be localhost, and the file storage path is that on which the shared storage is mounted.

This configuration offers potentially improved performance because the NFS protocol used by the filer exhibits better performance over the network than does the NSH protocol. In addition, this configuration allows the use of clustered NAS servers, allowing for redundancy and higher performance.

Considerations for virtualized environments

When BladeLogic Application Servers are hosted in virtual (guest) machines in a virtualized environment, the choice naturally arises whether it is better to deploy multiple Application Servers in a single virtual machine, or spread the Application Servers across separate virtual machines. In both cases, the Application Servers themselves are likely running on the same physical host computer.

In virtualized environments, BMC recommends deploying Application Servers in separate virtual machines. Further, the Application Server performs best when the virtual machine hosting it is configured to have one dedicated virtual CPU (vCPU). In addition, for best performance, avoid allocating more vCPUs than the physical host has physical CPU cores. See BMC BladeLogic Application Server Running on VMware ESX: Performance and Scalability Best Practices, http://documents.bmc.com/supportu/documents/60/54/106054/106054.pdf; and BMC BladeLogic 8.0 Application Server Running on Red Hat Xen: Performance and Scalability Best Practices,

http://documents.bmc.com/supportu/documents/29/84/142984/142984.pdf.

Load balancing

In large deployments involving multiple instances of some or all BMC BladeLogic Server Automation components, it may be necessary to provide load balancing services to ensure that the extra resources being applied are being utilized appropriately.

Job servers effectively perform their own load balancing, scheduling jobs and work items according to availability. No additional load balancing considerations are applicable for job servers.

http://documents.bmc.com/supportu/documents/60/54/106054/106054.pdf

http://documents.bmc.com/supportu/documents/29/84/142984/142984.pdf

Page 19


Two strategies for load balancing are commonly applied for configuration (UI) servers:

For cases where the user population and behaviors support it, you can achieve a crude but effective load balancing simply by assigning different users to use different configuration servers.

For more homogeneous load balancing, you must add an external load balancer to the installation and use it to distribute the load across configuration servers. The BIG-IP product by F5 is a common choice for this purpose.

Load balancing for authentication servers (for example, for purposes of failover) must similarly be provided by an external load balancer.

GEOGRAPHICALLY-DISTRIBUTED INSTALLATIONS

For a variety of reasons, it is unusual for the largest customer environments to be entirely contained within a single data center. In these cases it is necessary to consider not just the scale of the BMC BladeLogic Server Automation installation but also its geographic distribution. This section describes additional infrastructure recommended for managing servers in remote data centers.

SOCKS proxies

For a remote data center accessible only through a firewall, BMC recommends the use of a SOCKS proxy in the remote data center. The firewall can be configured to route connections on port 1080 (the SOCKS proxy port) to the SOCKS Proxy Server, which then brokers a connection to the actual target server agent (on port 4750, the RSCD agent port).

Repeaters

You can configure repeaters as staging areas for deployment files. Appropriate use of repeaters in remote data centers can significantly reduce the amount of network traffic that must be carried over long (slow and/or expensive) data lines.

The BMC BladeLogic Advanced Repeater is an enhancement to the repeater architecture that provides scalable transport of data over wide-area networks. Advanced repeaters also offer additional features, such as bandwidth throttling and secure communications, that may be important for large-scale installations.

BMC recommends the use of advanced repeaters for geographically-distributed deployments, with at least one repeater configured in each remote data center. Deploy Jobs with targets in remote data centers should normally be configured to use indirect push staging. (This staging pushes to the repeater and then pushes from the repeater to the target.)

Provisioning servers

As a rule, support for provisioning targets in remote data centers must be provided from provisioning servers located in the remote data center. Depending on the provisioning technology used, each remote data center must provide support for one or more provisioning-related services, for example DHCP, PXE and TFTP servers.

Citrix Presentation Server

If, in addition to remote managed servers, the installation also requires access to BMC BladeLogic Server Automation for remote users, BMC recommends the use of a Citrix Presentation Server. For performance reasons, it is usually not practical to deploy a management console (BMC BladeLogic Server Automation console) at a remote site, due to the bandwidth and latency requirements for the console-to-configuration server link.

Page 20


CONFIGURATION GUIDANCE

This section offers guidance on appropriate settings for the configuration parameters for a BMC BladeLogic Server Automation Application Server. Refer to the BMC BLADELOGIC SERVER AUTOMATION ADMINISTRATION

GUIDE for details on using the blasadmin tool to control the configuration parameters.

Apart from some general discussion, this section organizes BMC configuration recommendations according to type for single-purpose Application Servers. An Application Server can, of course, be configured to provide the combined services of the single-purpose Application Server (an Application Server of type ALL). In this case, you must modify the parameter recommendations, usually by adding recommended values for the same parameter for different Application Server types.

ABOUT JAVA MEMORY

Effective operation of a large Java system like the BladeLogic Application Server depends critically on the availability of sufficient heap memory. This section provides an overview of some considerations that apply to correctly sizing Java memory for BladeLogic.

Process space, Java heap, and native heap

A Java process comprises two distinct memory areas: the Java heap and the native heap. Both heaps, together with the Java executable code itself, must fit within the footprint of a single process. Increasing the maximum size of the Java heap necessarily decreases the maximum possible size of the native heap that can fit within a certain process size.

The java heap contains Java objects and accounts for most of the memory required by a running Application Server. The Java heap is managed by the Java garbage collector, and so is sometimes called GC heap.

The native heap (also sometimes called the C heap) contains thread stacks, file handles, and other objects not managed by the Java garbage collector.

If the maximum Java heap size is set too low, it is possible to run out of Java heap memory. If the maximum Java heap size is set too high, it is possible to run out of native heap memory. To complicate matters further, peak memory use for either the Java heap or the native heap depends on the precise work load being considered, as well as timing effects between concurrently-operating threads. Therefore, the recommendations that follow are merely that: recommendations, not guarantees or absolute limits.

32-bit processes

A process running under any 32-bit operating system is limited to 4 GB of virtual address space, from which the operating system must reserve a significant portion for itself. For example, 32-bit Windows divides the entire address space in half, allowing an application process only 2 GB total private process space. For large Java applications like Application Servers, this process space limit imposes a ceiling on the number of threads that can be accommodated within a single Application Server.

64-bit processes

A process running under a 64-bit operating system has access to a much larger virtual address space. Compared to a 32-bit Java process performing equivalent work, a 64-bit Java process also requires a larger Java heap, typically 50% or more larger than the 32-bit Java process. For an example, see http://users.elis.ugent.be/~leeckhou/papers/SPE06.pdf.

Recommended Java heap settings

This section describes recommended Java heap sizes for Application Servers running under different operating systems. These are recommendations only and must be adjusted in light of observed conditions, especially out-of-memory errors.

http://users.elis.ugent.be/~leeckhou/papers/SPE06.pdf

Page 21


For 32-bit processes, BMC recommends operating system-specific Java heap size values according to the table below. Due to memory constraints, job servers using 32-bit processes should be configured to use no more than 50 work item threads.

For 64-bit processes, BMC recommends that the Java heap size be increased as indicated in the table, if there is sufficient physical memory to support this setting.

blasadmin Setting

Module Setting Description and Recommendation

AppServer MaxHeapSize Specifies the maximum heap size for this Application Server.

See operating system-specific recommendations for this value summarized in the table below.

Max Java Heap Recommendations

Operating System 32-bit 64-bit

Windows 1024 MB 6144 MB

Linux 1536 MB 6144 MB

Solaris 2048 MB Not applicable

ABOUT THREAD POOLS

An Application Server maintains several thread pools, each dedicated to a specific purpose. Selecting appropriate sizes for each of the various thread pools is one of the most important configuration choices for an Application Server.

Each thread consumes resources, especially memory, even when idle. Java threads may also consume operating system resources such as thread handles. While executing, a thread consumes even more memory. Regardless of additional performance considerations, available process size limits the number of threads available in an Application Server.

Threads within the same process share certain data structures, especially caches, which are not shared between threads in different processes. Increasing the number of threads within a single process, especially threads within a particular thread pool, has two consequences:

Serendipity: Because there are more threads contributing to the process-wide caches, any given item

request from any thread is more likely to be fulfilled from the cache, because another thread is more likely to have already placed the element in the cache. This phenomenon has a mildly positive effect on overall performance as the number of threads increases.

Contention: Because some operations on some data structures require exclusive access, as the

number of threads increases, there is a greater likelihood of one thread having to wait for another thread’s exclusive access to conclude. This effect degrades per-thread performance as the number of threads increases; that is, increasing the number of threads is subject to diminishing returns, sometimes sharply so.

As the number of threads in a process grows, the negative contention effects grow more rapidly than do the positive serendipity effects. Doubling the number of threads in a pool improves performance, but doesn’t double it. Each additional thread provides a smaller and smaller net benefit, while still consuming as much memory and other resources as any other thread.

ABOUT DATABASE CONNECTIONS

Connections between an Application Server and the database are managed in three connection pools, with each pool devoted to a different purpose. Each connection pool allows the configuration of a minimum and maximum number of connections, although BMC recommends leaving the minimum value at zero for all connection pools.

Page 22


Configuring a database pool’s maximum size to be too high wastes resources, and for large installations, may risk exceeding the total capacity of the database server. You must also ensure that the database server has sufficient capacity to service all the connections from all the connection pools for all the Application Servers in the environment. BMC recommends working with the DBA and database vendor to ensure that you have this capacity, particularly for very large installations.

Conversely, configuring a database pool with a maximum size that is too low can degrade performance, as a thread requesting a database connection from an empty connection pool blocks until a connection becomes available.

RECOMMENDATIONS FOR JOB SERVERS

This section provides general recommendations for configuring Application Servers established as job servers.

Recommendations for the work item thread pool

The work item thread pool is the thread pool whose configuration has the greatest effect on overall job performance. However, selecting the best size for this thread pool involves trade-offs, and the best size will be different for different environments.

Most jobs generate one or more work items per target host, so a job targeted at thousands of servers can be expected to result in thousands of work items being queued for processing. Further, the work items themselves tend not to be CPU intensive. In light of these considerations, it is usually desirable to allocate a generous number of work item threads for a job server.

For 32-bit Application Servers, BMC recommends a setting of 50 work item threads. Configuring a larger number of work item threads risks an OutOfMemoryError under the process size limitations of 32-bit processes.

For 64-bit Application Servers, larger available process spaces make it possible to use a larger number of work item threads. The number of work item threads to configure is primarily determined by the effects of contention between work item threads.

Version 7.6 and earlier,

Version 8.0, Version 8.1

Up through BMC BladeLogic Server Automation 8.1, BMC recommends establishing additional Application Servers instead of increasing the number of work item threads. BMC recommends 50 work item threads for each of these Application Servers.

blasadmin Setting

Module Setting Description and Recommended Value

AppServer MaxWorkItemThreads Number of threads that can be used to execute job parts, for parallelism.

BMC recommends 50 work item threads for both 32-bit and 64-bit Application Servers.

Recommendations for the lightweight work item thread pool

Lightweight work item threads are of benefit primarily for Deploy Jobs. For installations in which Deploy Jobs represent a significant fraction of the workload, BMC suggests a value of 200 threads for lightweight work items. The default value of 0 threads for lightweight work items uses ordinary work item threads for the execution of all work items, lightweight or not.

blasadmin Setting


AppServer MaxLightweightWorkItemThreads

Number of threads that can be used to execute lightweight job parts, for parallelism.

BMC recommends a value up to 200 if Deploy Jobs are a primary use; otherwise 0.

Page 23


Recommendations for the BlExec service and thread pool

An Application Server’s BlExec service maintains a pool of threads for the execution of asynchronous tasks involving communication with remote targets. It is not normally necessary to change the BlExec service’s configuration settings; the default values produce good results in most cases.

blasadmin Setting


AppServer EnableAsyncExecution Enables/disables the async execution framework for jobs that allow it, for example, Deploy Jobs. You can set the value to true or false; the default setting is true.

BMC recommends leaving async execution enabled.

BlExec MaxSocketConnections Maximum simultaneous sockets open by the BlExec service.

BMC recommends using the default value of 500 in most cases. A higher value allows more simultaneous connections; a lower value reduces the demand for file descriptors.

BlExec NumWorkerThreads Number of worker threads used by the BlExec service.

BMC recommends using the default value of 20.

Other parameters

For completeness, this section describes some additional configuration parameters related to thread pool sizes for job servers. These configuration parameters do not normally require adjustment from their default values.

blasadmin Setting

Module Setting Description

AppServer MaxApprovalThreads Number of Approval Threads.

This parameter governs a small pool of threads used to communicate with BMC Atrium Orchestrator.

AppServer MaxJobs Maximum number of jobs the Application Server can execute simultaneously.

The maximum number of jobs the Application Server allows to run simultaneously, regardless of the availability of resources to execute the jobs.

AppServer MaxJobThreads Maximum number of threads that can be used to execute a job, for parallelism.

Maximum size for the job execution pool. The job execution pool is distinct from the work item thread pool. While most of the work involved in executing a job is delegated to work items, some of the work, such as creating the work items themselves, remains the responsibility of the job itself, and will be carried out by the job execution pool.

JobFactory GlobalDefaultJobParallelism Global default value for Job Parallelism made available to user.

This parameter has no direct effect on the operation of the Application Server. Rather, it is the default value that appears in the UI for a job’s maximum parallelism option, when that option is selected.

Page 24


Recommendations for database connections

For best job server performance, BMC recommends allowing the job execution connection pool to grow up to twice the number of work item threads (MaxWorkItemThreads in AppServer module).

Version 7.6 and earlier,

Version 8.0

Up through BMC BladeLogic Server Automation 8.0, logging of output from NSH script jobs is handled with connections from the general database connection pool, while other jobs use the job execution database connection pool. For best performance of NSH script jobs in these versions of BladeLogic, BMC recommends setting the maximum size for the general database connection pool to twice the number of work item threads.

blasadmin Setting


Database MaxJobExecutionConnections Maximum connections in the pool for job execution thread group.

BMC recommends a value that is twice the number of work item threads (MaxWorkItemThreads).

Database MaxGeneralConnections Maximum connections in the pool for general thread group.

For BMC BladeLogic Server Automation version 8.0 and earlier, BMC recommends a value of 2 * MaxWorkItemThreads, especially for environments that depend heavily on NSH script jobs. For version 8.1 and later, the default value should be adequate.

Process Spawner considerations

As memory size increases, so too does the cost of spawning a new process directly from the Application Server. For NSH script jobs, a job server can be configured to use a Process Spawner to spawn subprocesses, rather than spawning them directly. The Process Spawner is simply a process with a small memory footprint that can spawn new processes without the penalty of the Application Server’s large memory footprint. As the configured size of the Application Server grows, the benefit of using the Process Spawner increases.

blasadmin Setting

Module Setting Description and Recommendation

ProcessSpawner SpawnExternally Processes should be spawned outside the Application Server or not.

BMC recommends using the Process Spawner for all job servers for BMC BladeLogic Server Automation version 8.1 and beyond.

RECOMMENDATIONS FOR CONFIGURATION SERVERS

Estimating client connections

Parameter value settings for configuration (UI) servers should be based on the number of client connections you anticipate being made to the configuration server. This value is the total of the number of client connections from UI clients (RCP) and from BLCLI clients.

The number of client connections opened by a UI client varies over time and depends on the operations that the user is engaged in at any given moment. As an initial estimate, BMC recommends a planning figure of 2.5 client connections for each concurrent GUI user.

The BLCLI client uses exactly one client connection for its execution, and is usually much more short-lived than an interactive user’s GUI session.

Page 25


Thus, the peak demand estimate for client connections for the configuration server is:

2.5 * (number of simultaneous GUI users) + (number of simultaneous BLCLI commands)

If multiple configuration servers with a load balancer will be established, then the total load for client connections can be divided across the number of configuration servers that will be established.

Recommendations for the client connection service and thread pool

The client connection service is responsible for managing connections from client processes in Application Servers acting as configuration (UI) servers. The client connection service maintains a pool of threads for servicing client requests.

The following table describes the parameters that most strongly affect the performance of the client connection service. In most cases, it is not necessary to change these parameters from their default values.

blasadmin Setting


AppServer MaxClientContexts Number of maximum client connections to the Application Server.

BMC recommends estimating peak client connection demand, as described in the previous section, and setting MaxClientContexts to this value. In the absence of sufficient information from which to form an estimate for peak client connection demand, BMC recommends using the default value of 200.

AppServer MaxWorkerThreads Number of client connection worker threads.

BMC recommends using the default value of 10, or approximately 5% of the value of MaxClientContexts.

Recommendations for database connections

Similarly, for best configuration (UI) server performance, BMC recommends allowing the pool of database connections for client service threads to grow up to twice the number of client connection service threads (MaxWorkerThreads in AppServer module).

blasadmin Setting


Database MaxClientConnections Maximum connections in the pool for client connections.

BMC recommends a value that is twice the number of client connection threads.

Page 26


RECOMMENDATIONS FOR NSH PROXY SERVERS

BMC recommends the use of NSH Proxy servers as a best practice for security.

For best performance, configure the NSH Proxy server for the anticipated number of concurrent NSH connections it will be expected to handle.

blasadmin Setting


AppServer MaxNshProxyContexts Maximum number of NSH proxy connections to the Application Server.

Set this value to the maximum number of concurrent NSH connections the proxy will be expected to handle.

AppServer MaxNshProxyThreads Number of NSH proxy threads.

This value can be significantly less than MaxNshProxyContexts, to account for idle NSH connections. In the absence of usage estimates specific to the installation, BMC suggests an initial estimate of 20% of MaxNshProxyContexts.

Database MaxClientConnections Maximum connections in the pool for client connections.

For an Application Server configured to act exclusively as an NSH proxy server, this value should be the same as MaxNshProxyThreads. For an Application Server configured to act as both a configuration server and an NSH proxy server, this value should be the sum of MaxWorkerThreads and MaxNshProxyThreads.

Page 27


APPENDIX: TCP/UDP PORT USAGE

The following table summarizes the use of TCP/UDP ports across all the elements of a BladeLogic installation:

Port Protocol From To Notes

25 SMTP (TCP) Application Server

Mail Server SMTP

25 SMTP (TCP) BDSSA server SMTP server For emailing scheduled reports and notifications

67 DHCP (UDP) PXE client DHCP service PXE boot broadcasts a DHCP request that includes PXE info. By default, PXE Server binds to 67 UDP.

68 DHCP (UDP) DHCP PXE client Extended DHCP response to an initial extended DHCP request

69 TFTP (TCP/UDP)

PXE client TFTP Server

80 HTTP (TCP) HTTP client BDSSA server

80 HTTP (TCP) PXE client PXE Server HTTP; used for Linux PXE Servers

161 SNMP (UDP) Application Server

SNMP

162 SNMP (UDP) Application Server

SNMPTRAP

443 HTTPS (TCP)

HTTPS client BDSSA server

445 SMB (TCP) PXE client PXE Server SMB; used for Windows PXE Servers

1080 TCP SOCKS client SOCKS proxy SOCKS Proxy protocol

1433 MS-SQL (TCP)

Application Server

SQL Server DB

1433 MS-SQL (TCP)

BDSSA server SQL Server DB

1433 MS-SQL (TCP)

PXE Server SQL Server DB

1521 TNS (TCP) Application Server

Oracle DB

1521 TNS (TCP) BDSSA server Oracle DB

1521 TNS (TCP) PXE Server Oracle DB

4011 DHCP (UDP) PXE client PXE Server PXE discovery when co-located with DHCP

4750 RSCD (TCP) Application Server

RSCD Agent Primary communication channel from Application Server to each managed host

5282 HTTP (TCP) Advanced Repeater

Adv. File Server

Page 28


Port Protocol From To Notes

7717 TCP File Server Adv. File Server Marimba publishing -- if the File Server and Adv. File Server are not co-located

7717 TCP Transmitter Administrator

Adv. File Server usually local traffic only

7717 TCP Proxy Administrator

Adv. Repeater usually local traffic only

7717 TCP Certificate Manager

BMCCM Tuner usually local traffic only

8080 HTTP (TCP) Adv. Repeater Adv. Repeater usually local traffic only

9300 TCP Cognos client BDSSA server Cognos report

9640 TCP BDSSA server BDSSA Auth. Server

BladeLogic SSO

9700 JMX (TCP) BLASAdmin console

Application Server Launcher

JMX listener -- usually local traffic only

9701 TCP Application Server

Launcher

9702 TCP Console Launcher

9831 TCP Provisioning Client

Application Server SSL Provisioning (user guide p. 853, steps 7 and 9)

9836 (base+36*)

TCP Application Server

RMI Registry

9838 (base+38*)

TCP Jconsole JMX listener for Application Server

9840 (base+40*)


Auth. Service Authentication Service

9840 (base+40*)

TCP RCP (Client UI) Auth. Service

9841 (base+41*)

TCP RCP (Client UI) Application Server

9842 (base+42*)

TCP NSH, Application Server

NSH Proxy

9850-9899 (MinPort-MaxPort**)


Application Server

RMI communication ports

* Application Server ports are normally configured from a base port, with 9800 being the default base port. A second Application Server on the same host will typically have a base port of 9900, and so on. Arbitrary port assignments can be made in all cases.

** The MinPort-MaxPort range is configurable, with 9850-9899 being the default for a single Application Server.

Page 29


Business runs on IT. IT runs on BMC Software.

Business thrives when IT runs smarter, faster and stronger. That’s why the most demanding IT organizations in the world rely on BMC Software across distributed, mainframe, virtual and cloud environments. Recognized as the leader in Business Service Management, BMC offers a comprehensive approach and unified platform that helps IT organizations cut cost, reduce risk and drive business profit. For the four fiscal quarters ended September 30, 2010, BMC revenue was approximately $1.96 billion. Visit www.bmc.com for more information.

BMC, BMC Software, and the BMC Software logo are the exclusive properties of BMC Software, Inc., are registered with the U.S. Patent and Trademark Office, and may be registered or pending registration in other countries. All other BMC trademarks, service marks, and logos may be registered or pending registration in the U.S. or in other countries. All other trademarks or registered trademarks are the property of their respective owners. © 2011 BMC Software, Inc. All rights reserved.

AIX and IBM are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both.

Linux is the registered trademark of Linus Torvalds.

Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.

UNIX is the registered trademark of The Open Group in the U.S. and other countries.

*195833*

http://www.bmc.com/

Documents

BMC Bladelogic