Easy Configuration of IBM BladeCenter™ S with …download2.boulder.ibm.com/sar/XS/00m7j/0/ibm_bc_start_now_adviso… · Easy Configuration of IBM BladeCenter™ S with Start Now

Whitepaper

Easy Configuration of IBM BladeCenter™ S with Start Now Advisor 2.0

Version 1.0

May 20th, 2009

China Systems & Technology Lab

Authors: CheKim Chhuor

Hai Jun Zhong

Zhen Zhen Gao

Copyright 2009 IBM Corporation. All Rights Reserved. References in this document to IBM products or services do not imply that IBM intends to make them available in every country. Neither this documentation nor any part of it may be copied or reproduced in any form or by any means or translated into another language, without the prior consent of the IBM Corporation. IBM makes no warranties or representations with respect to the content here of and specifically disclaims any implied warranties of merchantability or fitness for any particular purpose. IBM assumes no responsibility for any errors that may appear in this document. The information contained in this document is subject to change without any notice. IBM reserves the right to make any such changes without obligation to notify any person of such revision or changes. IBM makes no commitment to keep the information contained herein up to date. The following terms are trademarks or registered trademarks of IBM Corporation in the United States or other countries or both: IBM, IBM logo, DB2, DB2 Universal Database, BladeCenter. For more information about IBM trademarks and copyrights, a full list of U.S. trademarks owned by IBM may be found at: http://www.ibm.com/legal/copytrade.shtml. The following terms are trademarks of other companies: Intel is a registered trademark of Intel Corporation or its subsidiaries in the United States and other countries. Opteron, AMD Opteron, AMD, the AMD Arrow logo and combinations thereof are trademarks of Advanced Micro Devices, Inc. VMware, the VMware “boxes” logo and design, Virtual SMP, VMotion, LabManager, virtual infrastructure are registered trademarks or trademarks of VMware. Novell, SUSE, the Novell logo, and the N logo are registered trademarks of Novell, Inc. in the United States and other countries. Microsoft, Windows, Internet Explorer, are trademarks of Microsoft Corporation in the United States, other countries or both. UNIX is a registered trademark of The Open Group in the United States and other countries. Linux is a trademark of Linus Torvalds in the United States, other countries, or both. Java and all Java-based trademarks and logos are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. LSI, LSI Logic, are registered trademarks of LSI Corporation in the United States and other countries. Other company, product and service names may be trademarks or service marks of others. Unless otherwise stated, IBM makes no representations or warranties with respect to non-IBM products. Disclaimer This document is subject to change without notification and will not cover the issues encountered in every customer situation. It should be used only in conjunction with the official product literature. The information contained in this document has not been submitted to any formal IBM test and is distributed AS IS. All statements regarding IBM future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. Contact your local IBM office or IBM authorized reseller for the full text of the specific Statement of Direction. Some information addresses anticipated future capabilities. Such information is not intended as a definitive statement of a commitment to specific levels of performance, function or delivery schedules with respect to any future products. Such commitments are only made in IBM product announcements. The information is presented here to communicate IBM's current investment and development activities as a good faith effort to help with our customers' future planning. The "Solution Assurance/SSPD/Techdocs" information, tools and documentation ("Materials") are being provided to IBM Business Partners to assist them with customer installations. Such Materials are provided by IBM on an "as-is" basis. IBM makes no representations or warranties regarding these Materials and does not provide any guarantee or assurance that the use of such Materials will result in a successful customer installation. These Materials may only be used by authorized IBM Business Partners for installation of IBM products and otherwise in compliance with the IBM Business Partner Agreement. Thanks to the following people for their support: Jing Qian, Information Development, IBM CSTL Geoff Hunt, BladeCenter, Telco Product Manager, IBM UK Andrew Spurgeon, AP BladeCenter Product Manager, IBM New Zealand

Table of Contents 1. Executive Summary ...................................................................................................... 4 2. IBM BladeCenter Start Now Advisor ............................................................................. 5

2.1 Getting started ......................................................................................................... 5 2.2 Features .................................................................................................................. 6 2.3 Workstation requirements ....................................................................................... 7 2.4 Supported chassis components .............................................................................. 7

3. Scenario 1: General business applications in a box ..................................................... 8 3.1 Scenario introduction .............................................................................................. 8 3.2 Storage topology ..................................................................................................... 9 3.3 Chassis setup with Start Now Advisor .................................................................. 10 3.4 Blade server OS installation .................................................................................. 17

3.4.1 Configure SAS adapter boot order ................................................................. 17 3.4.2 Multipath driver configuration ......................................................................... 19

3.5 Options for automated failover of blade servers ................................................... 19 3.5.1 IBM BladeCenter Open Fabric Manager ........................................................ 20 3.5.2 IBM BladeCenter Integrated Manager............................................................ 20

4. Scenario 2: High availability web application .............................................................. 21 4.1 Scenario introduction ............................................................................................ 21 4.2 Storage topology ................................................................................................... 22 4.3 Chassis setup with Start Now Advisor .................................................................. 24 4.4 Blade server OS installation .................................................................................. 26

5. Summary ..................................................................................................................... 28 6. References .................................................................................................................. 29

Version Date Change

0.1 2009/04/20 Initial draft 0.2 2009/04/22 Draft ready for internal review 0.3 2009/04/27 Corrections based on internal review 0.4 2009/04/29 Corrections based on internal review 0.5 2009/05/11 Modifications based on internal review 1.0 2009/05/20 Final modification

© IBM Copyright, 2009 May 20th 2009 Easy Configuration of IBM BladeCenter S with Start Now Advisor 2.0 Page 4 of 29

1. Executive Summary IBM BladeCenter S is an innovative chassis, integrating powerful blade servers, redundant SAN storage and high-speed networking switches, all packaged in a 7U form factor chassis suitable to be deployed in regular office environment or datacenter. BladeCenter S is uniquely designed for small and midsize businesses while delivering the same reliability, availability and serviceability (RAS) characteristics normally demanded by enterprise customers. While the benefits of reduced external cabling and modular design contribute to simplify initial hardware setup and long term investment protection, there are nevertheless a wide range of software configuration tasks to be done to fully exploit the BladeCenter S. That's where IBM BladeCenter Start Now Advisor comes in the picture. It is a wizard-based easy-to-use configuration tool specifically designed for BladeCenter S, taking user through the entire configuration process in a single consistent user interface without switching back and forth between various tools. Start Now Advisor simplifies the IT staff's task and lessens the learning curve during initial setup of BladeCenter S, as well as saving time by automatically detecting hardware and silently handling tasks in the background. Manually configuring and updating a fully populated BladeCenter S could take an IT staff between several hours to two days of time, with Start Now Advisor it only takes about 30 minutes to accomplish the same task1. This paper presents the detailed procedure of configuring a BladeCenter S chassis using Start Now Advisor, with the support of two real life scenarios.

1 Based on the two scenarios described later in this paper, using new healthy and supported hardware


2. IBM BladeCenter Start Now Advisor The IT environments of SMB customers are typically characterized by personnel who have a wide range of roles and responsibilities. Whilst they have the same needs as larger enterprise IT environments, such as ensuring hardware and software are continuously updated and ready to meet business requirements, the enterprise also demands a quick return on the IT investment. For this kind of environment, a shorten learning curve is extremely valuable in helping the the adoption of new technologies and providing the quick ROI demanded by their stakeholders. Start Now Advisor was designed to help the first time users get their chassis up and running quickly, but for experienced users it also helps them handle the lengthy tasks of firmware update and storage setup, saving them precious time and avoiding potential human errors. For readers new to BladeCenter S chassis, Figure 1 shows a real picture of a fully populated chassis with the names of the components as used in the rest of this paper.

Figure 1: BladeCenter S front and rear view

2.1 Getting started Start Now Advisor is shipped on a DVD with every new BladeCenter S chassis. It is also available for download from IBM website2, at no extra cost. It is designed to be the only tool a user needs for getting a new BladeCenter S up and running quickly. To launch Start Now Advisor, just insert the DVD in your laptop and it will come up automatically. There's no need to run an installation program or reboot the system at all. If you're downloading it from the web, just run the self-extracting executable to extract the package to your local drive and follow the instruction to launch it. Start Now Advisor needs to remotely connect to the chassis AMM in order to collect information and apply configuration changes. Depending on the network that the AMM is on and whether it has DHCP service running or not, you will be guided by the wizard to connect your laptop so that it can query AMM and configure it. 2 http://www.ibm.com/systems/support/supportsite.wss/docdisplay?lndocid=MIGR-5076842&brandind=5000020


The overall workflow is illustrated in Figure 2. This process typically takes you longer than 30 minutes. After that the blade servers are ready for operating system installation. There's no need to go to any website or read any user guide.

LaunchSNA

Initial AMM config

Network mode

Health check

Chassis config

Firmware update

Storage config

Apply all config

Finish

DHCPNo DHCP

LaunchSNA

Initial AMM config

Network mode

Health check

Chassis config

Firmware update

Storage config

Apply all config

Finish

DHCPNo DHCP

Figure 2: Configuration flow

2.2 Features Continuing from the first version of Start Now Advisor, Start Now Advisor 2.0 is designed exclusively for BladeCenter S configuration. Among the features in the following list, those marked with an asterisk are new features of version 2.0:

• *Update firmware of AMM and I/O modules • *Update firmware of blade servers • Setup zoning for chassis integrated storage • Setup chassis integrated RAID storage • *Setup blade internal RAID storage • Automatic chassis discovery • Chassis inventory & health check • Setup chassis components connectivity • Define chassis general settings • Change administrative password • Export and import configuration file • Command line interface • *Setup Service Advisor, chassis internal network (CIN) and event notifications • *Support the latest blade servers: HS22, LS22 and LS42. The graphical wizard dynamically adapts to the chassis components to present only the relevant features to the user. No time is wasted going through configurations that might not be applicable to the actual chassis. To avoid information overloading commonly seen in many software configuration tools, the wizard only asks the minimum information required to get the chassis up and running.


2.3 Workstation requirements Start Now Advisor runs on a regular workstation or laptop. More specifically the following minimum configuration is recommended: Hardware requirements • Intel or AMD processor, x86 or x64 • 512MB RAM or more • 500MB of free disk space for temporary usage • Ethernet adapter Software requirements • Microsoft Windows XP • Microsoft Windows Vista3 • Microsoft Windows Server 2003, x86 or x64 • Microsoft Internet Explorer 6.x or 7.x Firmware acquisition requirements • Internet connectivity, directly or through a HTTP proxy server In addition, Start Now Advisor has been successfully tested in virtualized environments running on VMware Server and VMware ESXi hypervisors. For safety reason it is not recommended to run Start Now Advisor from a pure wireless laptop since remote firmware update is involved. Likewise, the laptop should be running on AC power during firmware update to avoid being suspended during a critical operation.

2.4 Supported chassis components The following chassis components are supported4 by Start Now Advisor 2.0:

• BladeCenter S • Blade servers

HS12, HS21, HS21 XM, HS22 LS21, LS22, LS41, LS42

• SAS Connectivity Module • SAS RAID Controller Module • Server Connectivity Module • Intelligent Copper Pass thru Module • BNT Layer 2/3 Copper Ethernet Switch Module • BNT Layer 2/3 Fiber Ethernet Switch Module • Cisco Catalyst Switch Module 3012 • Disk Storage Module for BladeCenter S • Various daughter cards for blade servers. There is no harm at having any other components in the chassis, but Start Now Advisor won't try to configure or update those. Some components have minimum firmware level requirement or dependent hardware, new users are encouraged to consult the prerequisite checklist link on the wizard's Welcome screen.

3 It is not possible to update firmware of SAS RAID Controller Module when running on Windows Vista. 4 Please consult the ServerProven site for complete and latest compatibility information, at: http://www.ibm.com/servers/eserver/serverproven/compat/us/eserver.html


3. Scenario 1: General business applications in a box IBM BladeCenter S is the ideal candidate to host a suite of business applications, deployed and managed in house or by ISV, in a single box that could be hosted in regular office environment while providing the reliability and availability expected by every business owner.

3.1 Scenario introduction In this scenario, we're going to deploy a suite of applications for an online retail business of 20 employees. The overall topology is illustrated in Figure 3.

Figure 3: Overall topology

One fully populated BladeCenter S chassis will be used to host all the above applications. The blade servers will be configured as shown in Table 1. Bay Hardware Functions Storage requirement

Blade 1: WEB HS21 - 2 CPU - 4GB RAM

- Company website - Online store & payment - Discussion forum

- RAID1 pool (146GB) - 2 volumes: OS (20GB), WEB (50GB)

Blade 2: EMAIL HS21 - 1 CPU - 2GB RAM

- Email & calendar - Web content management - Internal wiki

- Shared RAID5 pool (584GB) - 3 volumes: OS (20GB), EMAIL (50GB), CMS (50GB)

Blade 3: FILES HS21 - 1 CPU - 2GB RAM

- LDAP server - File & print server

- Shared RAID5 pool - 2 volumes: OS (20GB), FILES (350GB)

Blade 4: ERP HS21 - 2 CPU - 16GB RAM

- ERP provided by ISV - RAID1 pool (146GB) - 3 volumes: OS (20GB), ERPAPP (20GB), ERPDB (50GB)

Blade 5: CRM HS21 - 2 CPU - 8GB RAM

- CRM provided by ISV - RAID1 pool (146GB) - 3 volumes: OS (20GB), CRMAPP (20GB), CRMDB (50GB)

Blade 6: SPARE HS21 - 1 CPU - 8GB RAM

- Non critical applications - Spare for the other blades

- Shared RAID5 pool - 2 volumes: OS (20GB), DATA (50GB)

Total - 3 RAID1 pools - 1 shared RAID5 pool - 15 volumes (810GB used)

Table 1: Blade server configuration

The blade servers have no internal hard drives, each one of them must have a SAS Expansion Card in order to access the storage volumes of the SAS RAID Controller Modules. The spare blade server is used to run some non critical applications during normal time, and in case any other blade encounters hardware failure, it could be used to temporarily replace the broken blade, albeit at a lower capacity.


Here's the chassis configuration for supporting this scenario: • BladeCenter S model 8886 • 2 SAS RAID Controller Modules (RSSM) • 2 Disk Storage Modules (DSM) • 12 hard drives of 146GB • 1 BNT Layer 2/3 Copper Gb Ethernet switch • 4 power modules Besides the four power cables, we only need to connect three cables to the company central network switch: one cable from AMM and two cables from the Ethernet switch at I/O bay 1, as shown in Figure 4. That's the beauty of the BladeCenter S hardware simplification: using individual rack servers in this scenario would require a dozen of power cables, a dozen of network cables and a dozen of FC cables, without counting all the corresponding ports to plug them to and the burden to maintain them.

Figure 4: BladeCenter S external cabling

We chose to use only one Ethernet switch in this scenario because we believe that most small businesses don't have a redundant network infrastructure. If such redundancy is needed, we could add a second switch at I/O bay 2, and inside each blade server we would need to add an Ethernet daughter card. We run two cables between the chassis switch and external switch for path redundancy. We could actually dedicate each external port of the chassis switch to a blade server if more bandwidth is needed, but that would consume up to six ports at the external switch and that might be a waste. It should be noted that, although there are two onboard Ethernet controllers in every blade server, both of them are wired to the same Ethernet switch at I/O bay 1.

3.2 Storage topology An important thing to mention here is the dual redundant paths to the storage subsystem. As illustrated by Figure 5, every blade server has two paths to the RSSM, and the RSSM have two paths to the two DSM containing the actual hard drives. This entire redundant path provides a pretty solid protection against most hardware failures.


Figure 5: Storage topology with SAS RAID Controller Modules

The RSSM should always be installed in pair, at I/O bay 3 and bay 4, along with two Backup Battery Units (BBU) that are installed through the media tray in front of the chassis. The BBU will guarantee that unwritten I/O cache will be properly committed to the hard drives in the event of a chassis power outage. Both RSSM replicate storage settings among themselves, so user manages them as one entity, but each one has its own IP address. Technically speaking, the RSSM is composed of two subsystems: a SAS switch that provides connectivity between the blades and the hard drives in the DSM, a RAID controller that sits on top of the SAS switch to provide storage volumes to the blade servers. Each subsystem has its own piece of hardware and firmware. When updating firmware of the RSSM using the OBCL utility, it also looks at the firmware level of BBU, DSM and hard drives to determine if those should be updated as well.

3.3 Chassis setup with Start Now Advisor Assuming that the chassis has been installed in place, all the required components are plugged and network cabling are connected, we can now start the initial chassis setup. Here's the configuration tasks that we need to accomplish for this scenario:

• Assign IP addresses to AMM, RSSM and Ethernet switch • Validate inventory and health of the entire chassis • Input chassis administrator's contact info • Input blade servers descriptive names • Set correct date and time in AMM • Change the default password of AMM and RSSM • Verify and update firmware of the entire chassis • Create storage pools • Create storage volumes • Assign volumes to blade servers. Let's go ahead and put the Start Now Advisor DVD in the drive.


1. First, you'll be asked to accept the license agreement, then the Welcome page will show

up. If this is your first time using Start Now Advisor, you're encouraged to read through the prerequisite checklist to learn about hardware requirements and dependencies.

2. At the next page, you'll be asked to choose a connection method based on whether your AMM has been assigned an IP address. In this scenario, we assume that after DHCP request times out, the AMM falls back to the factory default address of 192.168.70.125. To change its IP address, you need to connect the laptop directly to the Ethernet port of AMM with a standard network cable. Choose the second connection method in the wizard and click Next.

3. At the Local Network Configuration page, you have to select the network adapter that is in use if you have more than one adapters. Then click Change Network Settings so that the laptop will be temporarily changed to be in the same IP subnet as the AMM in order to communicate with it.

4. At the Chassis Inventory and Health page, Start Now Advisor automatically detects the AMM and collects inventory of the chassis. A real image view of the chassis will be presented in the wizard, along with any sign of warning or alert if there is hardware problem. At this stage, Start Now Advisor will try to turn on any component that is not in power on state. If any component fails to be powered on, you may give it another try by clicking on Refresh. You should confirm that all the components are found and in healthy state before continuing.


5. At the IP Address Configuration page, you can change the IP settings of AMM, such as host name, address, subnet mask and default gateway. Do not touch other components at this moment. Upon clicking the Next button, the change is validated and applied to AMM immediately. AMM will be restarted to take effect. This process could take up to 5 minutes before the next page is shown.

6. At the Connect Chassis to Network page, the network settings of the laptop will be restored back. You'll be guided to connect your laptop and the chassis back to the normal network.

7. At the Chassis Discovery page, Start Now Advisor will try to discover the AMM through the new IP address. Upon successful discovery, it will show the chassis inventory as in step 4. After confirming that the chassis is in healthy state, select the Express configuration method at the followed page.

8. The IP Address Configuration page shows the same IP configuration info as earlier, but this time you don't have to touch the AMM settings. Input the network addresses for the


applicable I/O modules and click Next to apply immediately. Again this might take up to 5 minutes to complete as the I/O modules are being rebooted.

9. At the Domain Name Service page, enter the DNS server addresses for AMM to use. This is mandatory if you're planning to enable the Service Advisor function later, though it's always useful to define the DNS servers for other purposes such as event notifications.

10. At the Proxy Server page, fill in the appropriate information if your laptop requires a proxy server to reach out to the Internet. This will be used later to download firmware.

11. The Firmware Update page is divided into two sections: a section for I/O modules and a section for blade servers. For I/O modules, including AMM, Start Now Advisor automatically queries the latest available firmware from the web and compares that to the firmware in use. Intelligent suggestion will be made based on the comparison results and also based on firmware dependencies.

For blade server firmware, no comparison are made. You can select to update all the blade servers, or select individual blade to update. The latest firmware for the selected blade models will be downloaded from the web and applied to the blades. A report will show which blade components are updated.

The actual firmware acquisition and update are executed at the end of the session, not


immediately.

12. At the General Settings page, input the appropriate contact information and descriptive names. You can also choose to synchronize the date and time of AMM to the laptop.

13. At the Administrative Password page, you can change the default password of AMM and other applicable components to a new password. You can also enable event notifications at the followed page.

14. At the Service Advisor page, you must first view and accept the license agreement if you wish to enable it. All the fields are mandatory. If AMM needs a proxy server to reach out to Internet, then you should fill the appropriate information at the next page for that.

15. At the Storage Pools Creation page, choose the drop down list for 4 pools and enable a global spare drive. For every storage pool to create, give it a meaningful name, select the RAID level and disk drives to use. An estimated pool size is automatically calculated as drives are being selected. The estimated size is zero if you have not selected enough drives for the chosen RAID type, or if you've selected too many drives.


16. At the Storage Provisioning page, you can create storage volumes on top of the pools defined in previous step. Assigning volumes to blade servers is no-brainer with this easy-to-use GUI, there's no tedious WWN to deal with at all! An estimated pool occupation is shown at the bottom as each volume size is being typed in. If you type a size larger than the free pool space, the wizard will reduce it to a valid size automatically.

17. Skip the Advanced Settings page. At the Temporary IP Address page, you're asked to provided a valid IP address that will be assigned and used during blade firmware update. Only one address is needed for all the blade servers. The temporary address must be accessible from the laptop. Use the Test Connection button to find out if the entered address is currently being used.

18. At the Configuration Confirmation page, all the previous settings are summarized in the table. Upon confirmation, click Begin Configuration to start applying the changes to the chassis.


Alternatively, you can export the settings to a file for applying in a future session. You can also use the exported file for applying the same settings to multiple chassis, but you'd have to modify the data slightly for every chassis, especially the IP addresses. That can be done from within the wizard after importation.

19. At this point, you can watch the Configuration Progress page while changes are being applied and firmware are being downloaded and updated. This could take a few hours to complete depending on various hardware performance and environmental characteristics.

It is suggested to leave the laptop alone while Start Now Advisor is doing the heavy-lifting, and the chassis should be left alone too. Remember to disable auto sleep or hibernation function of your laptop during this operation. This would be a good time to plan for operating system installation details, such as disk partitions, logical volumes, file systems size, mount points, packages to include and exclude, IP addresses, user accounts, services to enable and disable, firewall rules update, and so much more...

20. Eventually all the tasks will be completed. Underneath the overall result message, you can see every task that was executed and the result. You can also view the Final Configuration Report for storage details.


That's it! Your amazing BladeCenter S is now fully configured and up to date. You may now begin the operating system installation using your favorite OS deployment tool.

3.4 Blade server OS installation In this scenario, we decided to standardize on Novell SLES 10 SP2 for all the blade servers. With only one OS to acquire and maintain, there're less technologies to learn and it's easier to keep track of new updates. Notice a few particularities when installing OS on these blade servers, because:

• there are 2 SAS controllers in each blade, one onboard and another one in the SAS Expansion Card

• there are 2 paths to the RAID storage, so the OS will normally see 2 identical disks for every assigned volume.

Let's quickly go through the steps to ensure a successful OS installation.

3.4.1 Configure SAS adapter boot order Before installing the OS on blade server, use the LSI Logic5 configuration utility during the BIOS loading sequence to set the priority of the SAS Expansion Card to be the first boot device. 1. Open a remote control session to the blade server 2. Power on the blade server, during the BIOS loading sequence, press Ctrl-C to enter the

LSI configuration utility

5 The SAS Expansion Card is based on LSI Logic SAS1064 chip


3. Depending on the blade model, in that utility screen you might see an adapter named SAS1064 for the SAS Expansion Card (PCI slot 1) and another one named SAS1064E for the onboard controller (PCI slot 0), as below:

Or you might see both adapter as SAS1064 (PCI slot 0 and 1):

In both cases, you should set the SAS1064 adapter at PCI slot 1 to boot order 0.

4. Change the boot support option as follow:

At this point you can reboot the blade server and begin OS installation. Remember that OS installer will see each assigned RAID volume as two similar disks, just pick the first one as installation target. Actually the proper method to install OS in a multipath storage environment is to enable only one path during the installation, and enable the secondary path afterward. But in our experience, omitting that step caused no harm, as long as you're aware of the duplicate disks for every volume.


3.4.2 Multipath driver configuration Once the OS is installed, you have to follow the procedure to enable the Device Mapper multipath driver for the OS to recognize the dual paths to the volumes, and to be able to failover in the event of storage path interruption. The whitepaper SAS RAID Controller Module Remote Boot Documentation 1.206 has very detailed instructions on this subject. It is highly recommended to read through it before attempting the first installation. After setting up the multipath environment, you'll see the two paths to every volume as shown below:

At this point, you're ready to create partitions and filesystems on the multipath'd storage devices, knowing that your storage subsystem is good enough to endure most hardware failures. Installing the business applications on the blade servers are out of the scope of this paper, so we will not cover that here.

3.5 Options for automated failover of blade servers In this scenario, we designated a spare blade server (blade #6) in case any blade runs into hardware trouble. If that happens, you could:

• from RSSM, manually re-assign storage volumes of the failed blade to the spare blade's WWN, then restart the spare blade; or

• pull out the SAS Expansion Card in the failed blade and plug it into the spare blade, assuming that the card is not the source of failure, then restart the spare blade.

If you have any network setup that uses the Ethernet MAC address of the failed blade, you'd have to update that manually as well. Although the above procedure doesn't seem very complicate, it requires the IT staff to be on site, and be knowledgeable enough of blade server failover in order to accomplish the task. Recently, IBM provides two solutions for automatic failover of blade servers: IBM BladeCenter Open Fabric Manager and IBM BladeCenter Integrated Manager. In depth coverage of those two solutions can be found on IBM web. Let's us briefly introduce them here.

6 http://www.ibm.com/systems/support/supportsite.wss/docdisplay?lndocid=MIGR-5078491&brandind=5000020


3.5.1 IBM BladeCenter Open Fabric Manager IBM BladeCenter Open Fabric7 is an integrated server I/O portfolio that offers an open, high performance and comprehensive set of interconnects and smart management tools to help run your business. Integrated across all blades, switches and chassis, BladeCenter Open Fabric supports open standards and industry interoperability across 5 I/O fabrics, including Fibre Channel, InfiniBand, Ethernet, iSCSI and SAS, helping to simplify blade I/O management and deployment so you can concentrate on realizing innovation in your business. BladeCenter Open Fabric Manager can run in standalone mode or as a plugin of IBM Systems Director8.

3.5.2 IBM BladeCenter Integrated Manager BladeCenter Integrated Manager9 is a Java™ application that works with the Advanced Management Module of the BladeCenter S to monitor blades status. The tool runs independently on a hot spare blade that automatically takes over a failed blade when the system detects a failure.

7 http://www.ibm.com/systems/bladecenter/hardware/openfabric/openfabricmanager/index.html 8 http://www.ibm.com/systems/management/director/ 9 http://www.alphaworks.ibm.com/tech/bib


4. Scenario 2: High availability web application IBM BladeCenter S is also ideal to host a transactional web application, with high availability and storage fault tolerance built in the chassis. The benefit of a fully redundant and self-contained chassis should appeal to ISV selling turn-key solutions with pre-configured OS, application and database, such as web portal or e-commerce solutions.

4.1 Scenario introduction In this scenario we're going to build a three-tier transactional web application environment. The first tier is composed of two web servers, the second tier is two application servers and third tier is the database servers. We're going to configure both database servers to have its own database, instead of a single shared database mostly implemented by database server clustering. The overall topology is shown in Figure 6.

Figure 6: Overall topology The advantage of this topology is that both database servers could be used concurrently during disruptive maintenance activities. For instance when a new release of an application requires database schema update, which is done by executing a long batch job; in a single shared database model you'd have to take the whole web site down during the maintenance window. With this independent database topology, you can update the database schema at the standby server while the web site continue to run on the primary server. When it's done, disconnect one set of web server and the application server, update them and point them to the standby database server, run some local testing to make sure that the new application is working properly, then finally bring it online by refreshing the load balancer. Later on when you're confident that the new application release is doing fine, you could update the other set of servers. If the new release isn't going well, you could always roll back to the previous release easily by refreshing the load balancer once again. This architecture gives you a fully redundant and extremely dynamic environment to deploy mission critical applications. There are many techniques for keeping both databases in sync. For example in case of IBM DB2 9.5 Data Server, you could use the High Availability Disaster Recovery (HADR) function, along with log shipping and mirroring, to keep the primary and standby databases in sync and consistent. DB2 also has the automatic client reroute function to automatically reroute the application servers database connections to the standby server in case the primary fails.


The downside of this architecture is that you'll need twice the storage space to contain the databases, and managing two databases will certainly be more complicate than having just one. But the benefit of a truly redundant infrastructure and the flexibility of maintenance outweigh the trouble by large and contribute to a highly available web environment. One fully populated BladeCenter S chassis will be used to host the above middleware. The blade servers are configured as in Table 2. Bay Hardware Functions Storage requirement

Blade 1: DB1 HS22 - 2 CPU - 16GB RAM

- Primary database server - RAID1 volume with 2 drives of 300GB - OS, software, database, transaction logs

Blade 2: DB2 HS22 - 2 CPU - 16GB RAM

- Standby database server - RAID1 volume with 2 drives of 300GB - OS, software, database, transaction logs

Blade 3: APPSRV1 LS42 - 2 CPU - 16GB RAM

- Application server - RAID1 volume with 2 drives of 146GB - OS, software, files

Blade 4: APPSRV2 LS42 - 2 CPU - 16GB RAM

- Application server - RAID1 volume with 2 drives of 146GB - OS, software, files

Blade 5: WEB1 HS12 - 1 CPU - 2GB RAM

- Web server - RAID1 volume with 2 drives of 73GB - OS, software, files

Blade 6: WEB2 HS12 - 1 CPU - 2GB RAM

- Web server - RAID1 volume with 2 drives of 73GB - OS, software, files

Total - 6 RAID1 volumes, 12 drives

Table 2: Blade servers configuration The blade servers have no internal drives, each one of them must have a SAS Expansion Card in order to access the disk drives of the chassis integrated storage. All operating system, software, files and databases will be stored on those drives. Here's the chassis configuration for supporting this scenario:

• BladeCenter S model 8886 • 2 SAS Connectivity Modules (NSSM) • 2 Disk Storage Modules (DSM) • 4 hard drives of 300GB • 4 hard drives of 146GB • 4 hard drives of 73GB • 1 Cisco Catalyst Switch Module • 4 power modules Chassis cabling is similar to the first scenario, illustrated in Figure 4.

4.2 Storage topology The NSSM is a simple SAS switch that allows any blade server to use any disk drives in the DSM and external disk enclosures. In this scenario we only focus on the disk drives in the DSM, which is called chassis integrated storage in Start Now Advisor. You can install the NSSM individually in I/O bay 3 or in pair in I/O bays 3 and 4, in which case both modules will provide twice the bandwidth to the disk drives.


Assigning disk drive bays to each blade bay is done through the predefined zoning system. There are predefined zones for 1, 2, 3 and 6 blades, in which the 12 disk drives are equally distributed to the selected number of blades. Figure 7 shows the disk drives mapping for 6 blades. The blade 1 will see disk 1 of DSM 1 and disk 1 of DSM 2. The blade 2 will see disk 2 of DSM 1 and disk 2 of DSM 2, and so on.

Figure 7: Storage topology with SAS Connectivity Modules Note that the NSSM only provides SAS connectivity service, hence the name of SAS Connectivity Module. It doesn't provide RAID volume or shared storage service. When a drive is zoned to a blade, it is similar as if the drive is directly attached to the SAS Expansion Card of the blade. So we can create RAID volume on those drives using the SAS Expansion Card. For fault tolerance purpose, it is critical to evenly distribute disk drives of a RAID1 volume in both DSM. If a DSM or NSSM fails, the RAID volume can continue to run in degraded mode until the unit is repaired or replaced. One might ask the justification of using 2 disk drives in the DSM instead of simply using blade internal drives. Actually the benefits are several, such as:

• Larger quantity: each blade could be assigned up to 12 drives by NSSM, while some blade models only support 1 internal drive.

• Hot swap: drives in DSM are hot swap capable while most blade internal drives are not In other words, using the chassis integrated storage allows customers to choose the blade model that really meet their requirement in terms of processor and memory, without worrying about its built-in storage capability. An indirect benefit of using chassis integrated storage is that failover is easier when the storage is outside of the blade. For example if a blade encounters hardware failure, you can simply install a replacement blade at the same bay and it will have access to the same disk drives. Or, if blade 1 fails and you want to use blade 2 as temporary replacement, you can simply swap the drives between them. Later on when blade 1 is repaired or replaced, swap the drives back to their original places.


4.3 Chassis setup with Start Now Advisor Assuming that the chassis has been installed in place, all the required components are plugged and network cabling are connected, we can now start the initial chassis setup. Here's the configuration tasks that we need to accomplish for this scenario:

• Assign IP addresses to AMM, NSSM and Ethernet switch • Validate inventory and health of the entire chassis • Input chassis administrator's contact info • Input blade servers descriptive names • Set correct date and time in AMM • Change the default password of AMM and NSSM • Verify and update firmware of the entire chassis • Choose the predefined storage zone for 6 blades • Create RAID volumes on every blades. The configuration steps for this scenario is mostly similar to the previous scenario except a few places. Actually steps 1 to 10 are identical to the previous scenario. You can refer to page 11 for details. At the step 11, the Firmware Update page shows selectable check boxes for the DSM and NSSM, meaning that you can choose to update the firmware of DSM independently of NSSM. With RSSM there is no such choice.

Steps 12 to 14 are identical to the previous scenario. You can refer to page 14 for details. At the Chassis Integrated Storage page, you can see how easily Start Now Advisor helps you to map all the drives to the 6 blades, and you can simultaneously create RAID volumes on top of the mapped drives within the SAS Expansion Card of the blades. The user interface has been simplified and unified for a task that would otherwise require the user to hop through the NSSM web interface, and then to the LSI configuration utility inside each blade.


At the Blade Internal Storage page, you can create RAID volumes on the blade internal drives as illustrated in the screen shot below. But actually we're going to keep the current settings because there's no drives in our blade servers.

Skip the advanced settings, confirm the changes and click Begin Configuration. You may go to lunch or have a coffee now... When the tasks are completed, you'll see something similar to the following screen shot.


At this point you may begin operating system installation on the blade servers using your favorite deployment tool.

4.4 Blade server OS installation As in the previous scenario, we're going to use Novell SLES 10 SP2 on all the blade servers. Since there are two SAS controllers in every blade as in the previous scenario, we should follow the procedure in section 3.4.1 to set the SAS controller boot order and boot support option. Note that there is no need to install multipath driver in this scenario, because using predefined zone, there is only one path to a disk drive in DSM at any given time. When the chassis has one NSSM installed at I/O bay 3, there is only one path to both DSM. When the chassis has two NSSM, the one at I/O bay 3 handles the path to DSM 1, while the one at I/O bay 4 handles the path to DSM 2, whence the increase in bandwidth. In case the one in I/O bay 4 fails and cannot be repaired promptly, you have to set a different predefined zone so that the remaining one in I/O bay 3 provides connectivity to both DSM again, otherwise disk drives in DSM 2 will be unaccessible from the blade servers. Start Now Advisor automatically detects the number of NSSM in the chassis and use the corresponding predefined zone during configuration, but if you add or remove a NSSM later on, you'll have to reset to the corresponding zone to make that effective. You may use the AMM web interface or the NSSM web interface to accomplish the task. As for the RAID volumes that were created on the SAS Expansion Card by Start Now Advisor, if you swap for another card and want to keep the data on the RAID volume (i.e. hardware replacement), follow this procedure: 1. Go to the LSI Logic configuration utility by pressing Ctrl-C during BIOS loading sequence

2. In the View Array screen, you will see that the array and drive status are all inactive.


3. Choose the Manage Array action, then Activate Array. Then the drives status will change to Primary and Not Syncd, the array status will show synchronization progress.

4. When the synchronization is completed, the drives status will change to Primary and Secondary, the array status will be Optimal.

Now you can resume usage of the RAID volume again, with the peace of mind that the data is well protected from drive failure.


5. Summary In this paper we explained that Start Now Advisor is ideal for SMB IT staffs to use for setting up a new BladeCenter S chassis, but even for experienced IT staffs familiar with the hardware, Start Now Advisor can save time and avoid potential human error in the chassis setup process. We shown that it is much easier to configure and update firmware of a new BladeCenter S using Start Now Advisor thanks to the integration of all the disparate tasks together in a consistent interface where the user is guided step-by-step to the end, and only needs to make a few decisions to get the chassis up and running. The rest are cleverly handled by Start Now Advisor in the background. We provided two real life scenarios to illustrate how to use Start Now Advisor to setup the chassis to the point where operating system is installed, including consideration of storage fault tolerance. We also briefly introduced blade server failover in both cases. While striving to keep the user interface of Start Now Advisor easy to use and reduce to minimum the choices that user has to make, we sacrificed some flexibilities and capabilities that might otherwise be useful in some scenarios. For instance we expect every chassis component to retain the default factory authentication. We do not support mapping a RAID volume to multiple blade servers which is useful for shared storage applications such as VMware ESX and some clustering solutions. We do not support custom zoning in NSSM which is useful for irregular distribution of chassis integrated storage drives to blade servers. If such storage configuration is needed in your environment, you may use IBM Storage Configuration Manager10 for advanced storage configuration purposes. It is free to download but is designed strictly for managing storage devices, unlike Start Now Advisor which covers the entire chassis.

10 http://www.ibm.com/systems/support/supportsite.wss/docdisplay?lndocid=TOOL-SCM&brandind=5000016


6. References IBM BladeCenter Start Now Advisor download http://www.ibm.com/systems/support/supportsite.wss/docdisplay?lndocid=MIGR-5076842&brandind=5000020 IBM BladeCenter S Chassis http://www.ibm.com/systems/bladecenter/hardware/chassis/blades/index.html IBM ServerProven BladeCenter compatibility http://www.ibm.com/servers/eserver/serverproven/compat/us/eserver.html SAS RAID Controller Module Remote Boot Documentation http://www.ibm.com/systems/support/supportsite.wss/docdisplay?lndocid=MIGR-5078491&brandind=5000020 IBM Systems Director http://www.ibm.com/systems/management/director/ IBM BladeCenter Open Fabric Manager http://www.ibm.com/systems/bladecenter/hardware/openfabric/openfabricmanager/index.html IBM BladeCenter Integrated Manager http://www.alphaworks.ibm.com/tech/bib IBM Storage Configuration Manager download http://www.ibm.com/systems/support/supportsite.wss/docdisplay?lndocid=TOOL-SCM&brandind=5000016

Documents

Easy Configuration of IBM BladeCenter™ S with …download2.boulder.ibm.com/sar/XS/00m7j/0/ibm_bc_start_now_adviso… · Easy Configuration of IBM BladeCenter™ S with Start Now