Upload
kwame
View
26
Download
2
Embed Size (px)
DESCRIPTION
dell training
Citation preview
1 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
DellTM Blade Server Configuration and Troubleshooting
Contents Introduction .................................................................................................................................................. 8
Course Safety ............................................................................................................................................ 8
Electrostatic Discharge .............................................................................................................................. 8
Module 1: Blade System Technology .......................................................................................................... 10
Introduction ............................................................................................................................................ 10
Overview of Blade Technology ............................................................................................................... 10
Blade or Blade Server .......................................................................................................................... 11
Blade System ....................................................................................................................................... 11
Common Terms ....................................................................................................................................... 12
Fabric ................................................................................................................................................... 12
Lane ..................................................................................................................................................... 12
Link ...................................................................................................................................................... 12
Port ...................................................................................................................................................... 13
Product Lines ........................................................................................................................................... 13
Summary ................................................................................................................................................. 13
Module 2: Dell Blade System Enclosures .................................................................................................... 14
Module Objectives .................................................................................................................................. 14
M1000e Modular Blade Enclosure: Enclosure Basics ......................................................................... 14
Enclosure Overview ................................................................................................................................ 14
Basic Configuration and Expansion ..................................................................................................... 15
The Front of the Enclosure .................................................................................................................. 16
The Back of the Enclosure ................................................................................................................... 16
Cabling Order ...................................................................................................................................... 17
Cooling ................................................................................................................................................ 18
Enclosure Components ........................................................................................................................... 19
2 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Chassis Management Controller (CMC) .............................................................................................. 19
iDRAC .................................................................................................................................................. 20
iKVM .................................................................................................................................................... 20
IOMs .................................................................................................................................................... 21
Fans ..................................................................................................................................................... 21
Midplane ............................................................................................................................................. 23
PSUs .................................................................................................................................................... 24
Power Distribution Units ..................................................................................................................... 24
Hot‐Swappable Components .............................................................................................................. 25
I/O Modules ............................................................................................................................................ 26
Topic Objectives .................................................................................................................................. 26
Fabrics ................................................................................................................................................. 26
Pass‐through Modules ........................................................................................................................ 27
Choosing IOMs .................................................................................................................................... 28
Fabric Consistency Checking ............................................................................................................... 28
Stacking IOMs ..................................................................................................................................... 29
PowerConnectTM M6220 Switch .......................................................................................................... 30
Cisco Switch ........................................................................................................................................ 31
Redundancy ............................................................................................................................................ 32
CMC Redundancy ................................................................................................................................ 32
Fabric Redundancy .............................................................................................................................. 32
Fan Redundancy .................................................................................................................................. 33
PSU Redundancy ................................................................................................................................. 33
Summary ................................................................................................................................................. 34
Module 3: Key Components ....................................................................................................................... 37
Module Objectives .................................................................................................................................. 37
Common Features ................................................................................................................................... 37
Blades .................................................................................................................................................. 37
Features .............................................................................................................................................. 37
Mezzanine Cards ..................................................................................................................................... 40
Hard Drive Options.................................................................................................................................. 40
Serial Advanced Technology Attachment (SATA) II ............................................................................ 40
3 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Serial Attached SCSI (SAS) ................................................................................................................... 40
Embedded Ethernet Controllers ......................................................................................................... 41
iDRAC .................................................................................................................................................. 41
Differences .............................................................................................................................................. 42
M600 Unique Features ....................................................................................................................... 43
M805 and M905 Unique Features ...................................................................................................... 44
Drive Controllers ..................................................................................................................................... 45
Drive Controller Options ..................................................................................................................... 45
CERC 6i ................................................................................................................................................ 46
CERC 6i Cache Policy ........................................................................................................................... 46
CERC 6i Configuration Utility............................................................................................................... 47
Creating an Array on the SAS6/iR ....................................................................................................... 48
SAS6/iR Virtual Disk Management ...................................................................................................... 53
View virtual disk properties ................................................................................................................ 53
Synchronize a virtual disk .................................................................................................................... 54
Activate a virtual disk .......................................................................................................................... 54
Delete a virtual disk ............................................................................................................................ 54
Internal Storage ...................................................................................................................................... 55
Diskless Configurations ....................................................................................................................... 55
SATA Repeater Daughter Card ............................................................................................................ 55
SAS6/iR Modular Daughter Card ......................................................................................................... 56
CERC 6i Modular Daughter Card ......................................................................................................... 57
Storage Daughter Card Installation and Removal ................................................................................... 58
Blade Storage Daughter Card Installation ........................................................................................... 59
Blade Storage Daughter Card Removal ............................................................................................... 59
SAS/SATA HDD Backplane Components ................................................................................................. 60
SAS/SATA HDD Backplane ................................................................................................................... 60
HDD Indicator LEDs ............................................................................................................................. 61
Mechanical Issues ............................................................................................................................... 61
Clustering and Virtualization ................................................................................................................... 62
Cluster Computing .............................................................................................................................. 62
Virtualization ....................................................................................................................................... 62
4 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Summary ............................................................................................................................................. 63
Module 4: Systems Management Hardware .............................................................................................. 65
Module Objectives .................................................................................................................................. 65
Management Basics ................................................................................................................................ 65
Management Connections .................................................................................................................. 65
Management Features ........................................................................................................................ 66
Operator Panel and LCD ...................................................................................................................... 68
Interpreting the Operator Panel LED .................................................................................................. 68
LCD Menus .......................................................................................................................................... 69
Chassis Management Controller ............................................................................................................. 69
Features and Capabilities .................................................................................................................... 70
CMC Cabling ........................................................................................................................................ 71
Daisy‐chain Connection ...................................................................................................................... 71
Redundancy ........................................................................................................................................ 72
Initial Configuration ............................................................................................................................ 72
Integrated KVM ....................................................................................................................................... 73
Features .............................................................................................................................................. 73
Capabilities .......................................................................................................................................... 74
OSCAR Configuration .......................................................................................................................... 75
Integrated DRAC...................................................................................................................................... 75
Embedded but Separate ..................................................................................................................... 76
Management Features ........................................................................................................................ 76
Management Interface ....................................................................................................................... 77
Security Features ................................................................................................................................ 79
Out‐of‐band Communication .............................................................................................................. 79
Configuration Interfaces ..................................................................................................................... 80
iDRAC Configuration Utility ................................................................................................................. 81
iDRAC Web interface ........................................................................................................................... 81
CMC Web Interface ............................................................................................................................. 81
Chassis LCD Panel ................................................................................................................................ 81
Local racadm ....................................................................................................................................... 81
OpenManage Integration ........................................................................................................................ 82
5 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
OpenManage Server Administrator .................................................................................................... 82
IT Assistant Integration ‐ now referred to as DMC ‐ provides Power Monitoring for servers
supporting PMBus reports with the following types of data: ............................................................. 83
Summary ................................................................................................................................................. 84
Module 5: Dell OpenManage Software ...................................................................................................... 87
Module Objectives .................................................................................................................................. 87
System Management Basics ................................................................................................................... 87
What Is Management? ........................................................................................................................ 88
Dell OpenManage System Overview .................................................................................................. 88
The Dell OpenManage Model ............................................................................................................. 89
Systems Build and Update Utility ........................................................................................................ 90
Deployment toolkit ............................................................................................................................. 91
Server Management: Remote Management ...................................................................................... 91
Server Management: In‐band Interfaces ............................................................................................ 92
Systems Management Protocols ........................................................................................................ 93
Review ..................................................................................................................................................... 93
Dell OpenManage Server Administrator ................................................................................................. 93
OMSA: Instrumentation Service ......................................................................................................... 94
OMSA Storage Management Service .................................................................................................. 94
OMSA: Remote Access Controller ....................................................................................................... 96
Monitoring Tools: IT Assistant ........................................................................................................... 97
Monitoring Tools: Baseboard Management Controller ...................................................................... 97
Maintenance Tools: Dell Server Update Utility .................................................................................. 98
Maintenance Tools: Dell Online Diagnostics ...................................................................................... 98
Summary ................................................................................................................................................. 99
Module 6: iDRAC ....................................................................................................................................... 100
iDRAC Management and Security Features .......................................................................................... 100
iDRAC Management Features ........................................................................................................... 101
iDRAC Security Features .................................................................................................................... 101
iDRAC Security Features .................................................................................................................... 102
Configuration Tasks ............................................................................................................................... 103
Configuring the Management Station .............................................................................................. 103
6 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Configuring the iDRAC ....................................................................................................................... 105
Configure the Managed Server ......................................................................................................... 108
Configuring the iDRAC Using the Web Interface .............................................................................. 110
Configuring the iDRAC Using the Web Interface .............................................................................. 110
Using Console Redirection .................................................................................................................... 112
Console Redirection Considerations ................................................................................................. 112
Configuring Console Redirection in the iDRAC Web Interface ......................................................... 113
Using the Video Viewer ..................................................................................................................... 113
Using the Video Viewer ..................................................................................................................... 113
Summary ............................................................................................................................................... 114
Module 7: Network Fabrics and IOMs ...................................................................................................... 116
Chassis Overview .................................................................................................................................. 116
Fabric A Broadcom Ethernet Controllers .............................................................................................. 117
Fabric A Features .............................................................................................................................. 117
Dell LOM TOE License Key ................................................................................................................. 117
Expansion Cards Overview .................................................................................................................... 118
Mezzanine Cards ................................................................................................................................... 118
Dell BCM57086S ................................................................................................................................ 118
Emulex LPe1105‐M4 FC HBA ............................................................................................................. 118
QLogic QME2472 FC HBA .................................................................................................................. 119
Qlogic SANBlade QNE247M Fibre Channel HBA Technical Specifications ........................................ 120
IOMs ...................................................................................................................................................... 121
Dell/Emulex PT‐1016 Fibre Channel Pass‐Through ........................................................................... 121
Dell/Emulex PT‐1016 ......................................................................................................................... 121
Dell/Emulex PT‐1016 Fibre Channel Pass‐Through ........................................................................... 122
Key Features ...................................................................................................................................... 122
Technical Specifications .................................................................................................................... 122
Brocade 4424 Fibre Channel Switch ................................................................................................. 123
InfiniBand Technology .......................................................................................................................... 124
About InfiniBand ............................................................................................................................... 124
How InfiniBand Works ...................................................................................................................... 124
Advantages of InfiniBand .................................................................................................................. 125
7 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Cabling............................................................................................................................................... 126
Link Speeds........................................................................................................................................ 126
InfiniBand Physical‐Layer Characteristics ......................................................................................... 127
Parallel Transmission in InfiniBand Links .......................................................................................... 127
Mellanox ConnectX IB MDI InfiniBand Host Channel Adapter (HCA) Mezzanine Card .................... 128
InfiniBand IOM: Cisco SFS M7000E IB Switch ................................................................................... 131
Technical Specifications .................................................................................................................... 133
Power‐On Indicators ......................................................................................................................... 134
Summary ........................................................................................................................................... 135
Module 8: Troubleshooting ...................................................................................................................... 137
Common Configuration Issues for Blades ............................................................................................. 137
Objectives.............................................................................................................................................. 137
Receive Message stating that the CMC’s are not redundant ............................................................... 137
Solutions............................................................................................................................................ 137
Update the Firmware ........................................................................................................................ 137
Message received in the iDRAC sel log ................................................................................................. 138
FC (QLGC or EMLX) Mezzanine would experience "F1/F2" during post. .......................................... 138
Server non responsive at power up ...................................................................................................... 138
Troubleshooting Resource .................................................................................................................... 139
Support.dell.com ................................................................................................................................... 140
Ftp.dell.com .......................................................................................................................................... 141
Module 9: Navigating Dell Information and Tools .................................................................................... 142
Dell System E‐Support Tool (DSET) ....................................................................................................... 142
Support.dell.com ................................................................................................................................... 142
Learndell.com ........................................................................................................................................ 143
Enterprise Support Services .................................................................................................................. 143
Thank You! ............................................................................................................................................ 143
8 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Introduction
Course Safety
Throughout the course, you will be expected to observe strict safety procedures at all times. These
include (but are not limited to):
Always use and carry tools in the correct manner.
Never lift anything that you feel is too heavy.
Never leave parts in places where others may trip over them.
When disassembling systems, always think about other people near you.
If you see anything that you feel is a possible safety concern, please alert your instructor
immediately.
A lot of the equipment used during this course is extremely heavy. If you feel that any piece of
equipment is too heavy for you to easily carry, please seek assistance.
Electrostatic Discharge
Electrostatic discharge (ESD) is a major concern when handling components, especially expansion cards
and system boards. Very slight charges can damage circuits in ways that may not be obvious.
Intermittent problems or a shortened product lifespan can also result.
Static Electricity ‐ A charge stored in any body.
Electrostatic Discharge ‐ A sudden transfer of electrostatic charge between bodies at different
electrostatic potential, usually as a spark as the bodies approach one another.
ESD can damage electronic components immediately, or with delayed effect.
You must take precautions to prevent ESD.
A discharge of static electricity can damage the components of your computer. Electrostatic discharge
occurs when, by touching a component, you cause a sudden voltage surge that equalizes the electricity
between you and the component. By using the wrist‐grounding strap that comes in the ESD kit and
taking several precautions, you can minimize the chances of a discharge occurring.
If a wrist‐grounding strap is not available, you can discharge the static electricity in your body by
touching an unpainted metal surface, such as on the computer chassis.
Before you unplug the machine, perform the following steps: while the system is plugged into the Earth
circuit via the power socket, attach the ESD wrist strap to your wrist and clip the other end to a ground
object.
Then you can unplug the machine. Static‐sensitive components arrive wrapped in anti‐static packing
material.
Do the following when handling static‐sensitive components:
9 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Use an ESD wrist‐grounding strap.
Handle all sensitive components in a static‐safe area.
If possible, use anti‐static floor mats and workbench pads.
When unpacking a static‐sensitive component from its shipping carton, do not remove the component
from the antistatic packing material until you are ready to install the component into your system.
10 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Module 1: Blade System
Technology
Introduction
This module introduces the basic
terms and concepts that are
developed in this course. After
completing this module, you will
be able to:
Define the basic terms and
concepts used in this
course.
Explain the benefits of
blade technology.
List the components of the
M‐series blade servers.
Overview of Blade Technology
Blade servers can be viewed as a complete server chassis, a component on a system board, a type of
computer configuration, or simply as a type of server:
A blade server is a server chassis housing multiple thin, modular electronic circuit boards, known
as server blades. Each blade is a server in its own right, often dedicated to a single application.
The blades are literally servers on a card, containing processors, memory, integrated network
controllers, an optional Fibre channel host bus adaptor (HBA) and other input/output (IO) ports.
A blade server is a computer on a System board, which includes processor, memory, and
sometimes storage. The blade server is intended to address the needs of large‐scale computing
centers to reduce space requirements and lower costs.
Blade servers are part of a computer configuration where power, cooling, storage and
connectivity are largely provided by an outer housing or chassis. The chassis contains and
services a number of specialized, stripped down System board units ‐ the blade servers ‐ each
one a complete computer or service device containing only vital processing and storage
elements.
Blade servers are self‐contained computer servers, designed for high density.
11 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Blade or Blade Server
Blade or Blade Server refers to the individual servers in a
blade enclosure. Dell blades include but are not limited
to:
PowerEdge 1955 9G blade
PowerEdge M600 10G Intel‐based blade
PowerEdge M605 10G AMD‐based blade
PowerEdge M805 10G AMD‐based blade
PowerEdge M905 10G AMD‐based blade
PowerEdge M610 11G Intel‐based blade
PowerEdge M710 11G Intel‐based blade
Blade Enclosure
Blade enclosure refers to the enclosure that provides
power, cooling, and connectivity to the blades. Dell
blade enclosures include:
PowerEdge 1955 enclosure
PowerEdge M1000e enclosure
Blade System
A blade system refers to the gestalt of the blade enclosure
and the blades it contains.
The enclosure provides power, cooling, and connectivity for
the blades. Within the enclosure, the individual blade servers
can be clustered (like disk arrays) or function independently.
12 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Common Terms
Fabric
A method of encoding, transporting, and synchronizing data between devices. Examples of fabrics
include:
Fibre Channel (FC)
Gigabit Ethernet (GE)
InfiniBand® (IB)
Fabrics are carried inside the PowerEdge M1000e enclosure. They carry information between blades and
I/O modules (IOMs) through the midplane.
Fabrics are also carried outside the M1000e through the physical interfaces on the IOMs.
Lane
A single fabric data transport path between IOMs and end devices. Each lane comprises one transmit
and one receive circuit. Differential signaling reduces noise interference.
Physically, a lane is four wires in a cable or four copper traces on a printed circuit board, each of
which carries one of the following signals:
A positive transmit signal
A negative transmit signal
A positive receive signal
A negative receive signal
Specific terms for lanes vary by technology:
Ethernet: Link
Fibre Channel: Link
InfiniBand: Physical Lane
PCIe: Lane
In this course, link has a different definition.
Link
A collection of multiple fabric lanes used to form a single communication transport path between lOMs
and end devices. Examples include:
Two, four, or eight lane PCIe
Four lane 10Gb‐KX4 PCIe
13 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Although Ethernet and Fibre channel use this term for both single and multiple lanes, this course uses
the terms distinctly to avoid confusion in the learning process. A link, as used in this course, provides
synchronization across multiple lanes so that they act as a single transport.
Port
The physical I/O end interface of a device to a link or lane. A port can connect to multiple lanes.
Product Lines
Dell currently offers two blade product lines.
The PowerEdge 1955 line, which packages both
blades and a blade enclosure
The M‐series, which comprises:
o The M1000e blade enclosure
o The M600 Intel®‐based blade
o The M605 AMD®‐based blade
o The M805 AMD‐based blade
o The M905 AMD‐based blade
.
Summary
Review the summary of this module before taking the self‐check to see how well you mastered
the topic.
Certain blade terminology can have different meanings in the industry. For the purposes of this
course, we define these terms as follows:
o Blade or Blade Server refers to the individual servers in a blade enclosure.
o Blade enclosure refers to the enclosure that provides power, cooling, and connectivity
to the blades.
o Blade system is a data center in a chassis.
o A fabric is a method of encoding, transporting, and synchronizing data between devices.
o A lane is a single fabric data transport path between IOMs and end devices.
o A link is a collection of multiple fabric lanes used to form a single communication
transport path between IOMs and end devices.
o A port is the physical I/O end interface of a device to a link or lane.
14 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Module 2: Dell Blade System Enclosures
Module Objectives
M1000e Modular Blade Enclosure: Enclosure Basics
This module describes the PowerEdge M1000e Modular
Blade Enclosure and the components that make it work.
The M1000e integrates the latest in management, I/O,
power, and cooling technologies in a convenient
modular package. Designed to support current and
future generations of server, storage, networking, and
management technologies, the M1000e includes the
headroom necessary to scale your environment to meet
your future needs.
After completing this topic, you will be able to:
List the M1000e features and components.
List the minimal configuration of the M1000e.
Explain how mezzanine cards and IOMs interact
in fabrics.
List the order in which you should connect cables when installing the M1000e.
Describe the cooling paths of the M1000e.
Enclosure Overview
The M1000e is a 10U enclosure that holds up to 16 blades. That means you can have 64 blades in a 40U
space.
The M1000e solution comprises:
Network, storage, and cluster interconnect modules (switches and pass‐through modules)
A high performance and highly available passive mid‐plane that connects blade server modules
to the infrastructure components
Power supplies
Fans
Integrated KVM (iKVM) and Chassis Management Controllers (CMCs)
The PowerEdge M1000e uses redundant and hot‐pluggable components throughout to provide
maximum uptime.
Benefits:
Simplified operations, with expanded management features
15 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Energy savings and increased compute capacity with efficient power supplies, fans and dynamic
power management
Increased computing density: 64 blades, 512 processor cores in a 42U rack (2U remain free)
Comparison of the PowerEdge 1955 and Ml000e Blade Enclosures 1955 Ml000e Form Factor 7U 10U
Blade Capacity 10 (60 in 42Us) 16 (64 in 4OUs) Multi‐slot Blades No Yes I/O Bays 4/2 fabrics 6/3 Fibre channel Switch or pass‐through Switch or pass‐through Ethernet Switch or pass‐through Switch or pass‐through InfiniBand No Yes Power Supplies 4 6 (3+3) Fans 4 (2 fans in 2 bays) 9 (3 fans in 3 zones) Chassis Management Modules
No Up to 2
KVM Yes Optional Simultaneous Virtual KVM
No Yes
Interactive LCD No Yes Power Management No Yes Power Monitoring No On CMC Internal Persistent Storage
Blade level Blade and chassis level
Basic Configuration and Expansion
The minimal configuration of the M1000e is:
One CMC: Adding a second CMC provides redundancy.
Three power supply units (PSUs): To make it easier to upgrade, Dell recommends adding an
additional package of three PSUs.
Nine fans: With all nine fans populated, you never have to worry about having sufficient cooling
for expansion.
You also need at least one blade to make the enclosure useful.
16 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
The Front of the Enclosure
Up to 16 blades are accessible from the front of the
M1000e. On the bottom front left corner of the
chassis there is a control panel to connect to the
integrated KVM (iKVM) on the back of the enclosure.
This connection allows you to connect a crash cart, if
needed.
The Back of the Enclosure
From the back of the enclosure, you can access:
One or two CMCs (one on each upper
right and left corner)
An integrated KVM (iKVM)
Up to six PSUs
Nine fans
Fabric connections (A, B, and C in two
banks)
17 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Cabling Order
For optimal cable management when setting up the M1000e, connect the cables in this order:
1. CMC A
2. CMC B (if populated)
3. iKVM
4. Power Supplies
5. Fabric A IOMs
6. Fabric B IOMs
7. Fabric C IOMs
18 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Cooling
The M1000e contains many
devices, all of which generate heat
and require power. Each of these
devices reports its temperature
and power status to the CMC,
which manages the PSUs and fans
to conserve energy and maximize
performance.
Air enters the front of the chassis
and exits through the rear of the
chassis. On its way, it passes:
Through the blades
Through the holes in the
midplane
Past the IOMs and other
rear components
Out of the back of the chassis
Plenums distribute the air among three
columns of fans. This arrangement breaks
the airflow into three zones. The CMC
directs the fans in each zone to pull as much
air as needed to cool that zone.
Input/output modules (IOMs), CMCs, and
the iKVM use a bypass duct to draw ambient
air from the top‐front of the enclosure. It
passes from the top of the IOM to the
bottom and exits into the plenum between
fans.
19 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Each PSU also has its own fan. Air flows
through the PSU from front to back,
keeping the unit cool.
The CMC also throttles the utilization of
each PSU and distributes the workload
among all of the PSUs. When the CMC
throttles back the utilization of a PSU, the
PSU draws less current and generates less
heat.
Enclosure Components
Now that you know what the M1000e contains, we can explore the components in greater depth. This
topic provides more information about the M1000e components, including:
One or two CMCs (for redundancy)
Integrated Dell Remote Access Controller (iDRAC)
Nine fans
Three, four, or six PSUs
Optional iKVM
After completing this topic, you will be able to:
Describe the components in the M1000e.
List the hot‐swappable enclosure components.
Chassis Management Controller (CMC)
The CMC is a hot‐swappable systems management device
that has a hardware and a software element. It provides
remote management and power control for the M1000e
and installed components. You can configure the CMC to
send e‐mail alerts or SNMP trap alerts for warnings or
errors related to:
Temperature
Hardware misconfiguration
Power outage
Fan speed
Each CMC includes an integrated 24‐port Ethernet switch
for out‐of‐band internal use only. The ports are all
20 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
dedicated to specific devices:
iDRAC on the blades (16)
CMCs (2)
Fabric IOMs (6)
Gb connectors (2)
The M1000e must have at least one CMC. You can add a second CMC for redundancy. Each CMC has the
equivalent processing power of a small server.
You can initially configure your CMCs via:
A serial connection
The operator panel and LCD on the front of the enclosure
iDRAC
The M‐series enhances management with multiple DRACs:
DRAC functionality is built into the M1000e enclosure’s CMC.
An integrated DRAC (iDRAC) in each blade has full hardware management device functionality at
the blade level.
The M‐series supports the Simplify IT model:
The iDRAC rolls up to the CMC.
The CMC rolls up to the Dell OpenManage
systems management software.
The Dell OpenManage systems
management software rolls up to other
products.
iKVM
The KVM on the M1000e is optional. It is a card
that mounts in a special slot in the rear of the
enclosure, rather than a discrete unit. Dell calls it
an integrated KVM (iKVM) because, when
installed, it functions as part of the enclosure. The
connections are accessible from the rear of the
enclosure to provide access to all installed blades and CMCs. There are no cables beyond those that
connect the mouse, keyboard, and monitor to the KVM.
When installed and configured, the iKVM activates the VGA and two USB ports on the front of the
enclosure. The connections provide crash cart access to the enclosure.
Features of the iKVM include:
21 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Fault indicator
Power indicator
ACI port
Two USB connectors for keyboard and mouse
Video connector
IOMs
IOMs provide access to the internal and external networks in the M1000e. The enclosure can house up
to six IOMs, two on each of three fabrics. This flexibility facilitates greater role diversity in the blades.
Fabric A
o Dual 1 Gb fabric for the enclosure’s internal network
Fabric B & C
o Dual 1 Gb fabric for the enclosure’s internal network
o Up to 2x 10Gb links per fabric
o Dedicated Ethernet and Serial Management connections
o Support for Fibre Channel and InfiniBand
Ethernet and Fibre Channel pass‐
through modules are also available.
More information is available in the
IOM topic of this module.
Fans
Each M1000e ships with all nine fans
installed. This provides the following
benefits:
Fully populated enclosures
have a redundant fan.
Cooling is already in place
when upgrading an
enclosure that is not fully
populated.
The fans install in three banks of
three fans to provide for zoned
cooling. The CMC controls fans in each zone to:
Maximize cooling effectiveness
Reduce power consumption
Reduce noise
22 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
23 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Midplane
The nervous system of the M1000e is its midplane. The midplane connects all of the various
components and modules together and facilitates communication among them. Ports on the blades
connect to the midplane. So do the IOMs in the rear of the enclosure. Signals pass from the blades to
the midplane to the IOMS to the external network and back again.
The midplane routes traffic from the three fabrics through the IOMs on the back of the enclosure.
When designing the M1000e’s midplane, Dell was thinking beyond the next generation of technology.
The midplane is already certified for:
Fibre Channel 8 (FC8)
InfiniBand
Blades connect to the front of the midplane. Signal
and power connectors on the blades slide into
receptors on the midplane.
24 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Because of the diversity of
devices, the rear of the midplane
looks more complex than the
front. All six PSUs connect to the
midplane to provide power for the
enclosure. In turn, the midplane
provides:
Power to nine fans
Power and
communications for the:
o CMCs
o iKVM
o LCD and operator
panel
o IOMs
PSUs
The M1000e ships with a minimum
of three PSUs in a non‐redundant
configuration. For redundancy, you
need a minimum of four PSUs, but
Dell recommends fully populating
the enclosure with six PSUs. If you
do not want to use all six power
supplies, you can:
Manually turn off power supplies
not in use
Automatically turn off unused PSUs
with dynamic power engagement
o Three PSUs running
o One is redundant and not running
o Two are in full standby
When using more than three PSUs, Dell recommends using a separate power distribution unit (PDU) for
each PSU. For more information, see PSU Redundancy and Grid Redundancy later in this module.
Power Distribution Units
The power supply units (PSUs) do not plug directly into a wall outlet. Instead a variety of power
distribution units (PDUs) provide 220V power to the PSUs.
25 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Type Required Outlet Description Example Installation Single‐phase 20 amp (20A) (16A De‐Rated in US)
NEMA L6‐30R
Provides one 20A circuit per PSU.
The recommended configuration is one PDU per PSU.
Single‐phase 30A (24A De‐Rated in US)
NEMA L6‐30R
Two PDUs provide grid redundancy, though the recommended configuration is one PDU per PSU.
Total power consumption for the enclosure must not exceed 24 amps in the non‐recommended configuration.
Little headroom remains for other devices.
Three‐phase 30A (24A De‐Rated in US)
NEMA L15‐30R
One PDU supports up to three PSUs.
Two PDUs provide grid redundancy.
Each PDU breaks out three 20 amp single phase connections for the PSUs.
Each connector has its own 20 amp breaker.
Single‐phase 60A (48A De‐Rated in US)
Hubbell 363R6W
One PDU supports up to three PSUs.
Two PDUs provide grid redundancy.
Each PDU breaks out three 20 amp single phase connections for the PSUs.
Each connector has its own 20 amp breaker.
Hot‐Swappable Components
Components that you can add or remove without powering down the M1000e include:
CMC (if redundant with two CMCs)
iKVM
Individual blades
IOMs
Fans
PSUs
26 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Before removing the blade or the
drives, you must shut down the
operating system on the blade.
I/O Modules
Topic Objectives
I/O Modules (IOMs) handle all of their fabric traffic between the blades in the enclosure and devices
outside the enclosure.
After completing this topic, you will be able to:
Explain the function of the three fabrics supported by the M1000e.
List fabric options for the M1000e.
Explain how mezzanine cards and IOMs interact in fabrics.
Fabrics
The M1000e supports three high‐speed fabrics, labeled A, B, and C. Blades in the enclosure
communicate with each fabric using:
Dedicated connections through the midplane
Mezzanine cards installed in the blade
IOMs installed in the back of the enclosure
Fabric A Fabric A supports Gigabit Ethernet for communication between devices in the enclosure. It can also
connect to external networks.
Ethernet controllers on the blades are based on Broadcom technology and support:
o TCP/IP Offload Engine (TOE)
In addition, full SCSI HBA capability with full protocol offload and a broader array of operating
system boot support is available.
While blades released in this series initially have a dual GbE LOM configuration, the system midplane is
designed to allow future support for up to four GbE LOMs in the half‐height form factor.
Fabrics B and C You can configure Fabrics B and C to support your unique needs by adding up to two dual‐port
mezzanine cards to each blade. These optional mezzanine cards include:
27 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
o Ethernet (including iSCSI and 10 Gb Ethernet)
o Fibre Channel
o InfiniBand
Like the connector, mezzanine card form factors are common to both fabric B (card 1) and C (card 2).
The optional mezzanine cards connect to the blade’s chipset via 8‐lane PCIe.
No matter which technology you choose for Fabrics B and C, the mezzanine cards installed in the blades
and the lOMs installed in the enclosure must be compatible.
Pass‐through Modules
A pass‐through module provides a transparent connection between the LAN and individual blades. Dell
currently offers Ethernet and Fibre Channel pass‐through modules.
28 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Ethernet The Ethernet pass‐through module supports 10/100/1000Mb
connections as determined by the LOM firmware or blade
operating system.
Fibre Channel The Fibre Channel pass‐through module supports 1, 2, or 4
Gigabit per second (GBs) connections.
Choosing IOMs
The IOM installed in a fabric determines the types of
mezzanine cards that you can install in the blades that
attach to that fabric. For example, you cannot install an
Ethernet pass‐through module in A1 and an Ethernet
switch in A2. Fabrics expect the same type of IOM in
both slots. For best results, you should install two
modules of the same brand and model.
So, if you install a Fibre Channel IOM in Fabric B, blades
that attach to Fabric B must have a Fibre Channel
mezzanine card installed in the first mezzanine. If a
mezzanine card for a different technology were installed
in that slot, the blade would not boot.
You can mix the mezzanine cards in the blades, but not
the IOMs. So you could install a QLogic Fibre Channel
card in one blade and an Emulex Fibre Channel
controller in another, but you should install matching
IOMs in the fabric.
Also, Ethernet IOMs designed for Fabrics B and C are not interchangeable with Ethernet IOMs for
Fabric A.
Fabric Consistency Checking
The M1000e system management hardware and software includes Fabric Consistency Checking, which
prevents accidental activation of any misconfigured Fabric device on a blade.
Since mezzanine‐to‐IOM connectivity is hardwired yet fully flexible, it is possible to inadvertently hot
plug a blade server module with the wrong mezzanine into the system. So if Fibre Channel IOMs are
installed in Fabric C I/O slots, all blade server modules must have either:
29 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
No mezzanine in Fabric C.
Fibre Channel cards in Fabric C.
For example: If an Ethernet mezzanine card were in a C slot of a blade being swapped into the
enclosure, the Fabric Consistency Check would automatically detect this misconfiguration, and the CMC
would alert you of the error.
In this situation, no damage occurs to the system, and you can reconfigure the faulted module.
Blade Mezzanine Card Slot B
Blade Mezzanine Card Slot C
I/O Bay A1/A2 I/O Bay B1/B2 I/O Bay C1/C2
None None Ethernet switch or pass‐through module
None None
Ethernet None Ethernet switch or pass‐through module
Ethernet switch or pass‐through module
None
None Ethernet Ethernet switch or pass‐through module
None Ethernet switch or pass‐through module
Ethernet Ethernet Ethernet switch or pass‐through module
Ethernet switch or pass‐through module
Ethernet switch or pass‐through module
Fibre Channel None Ethernet switch or pass‐through module
Fibre Channel switch or pass‐through module
None
None Fibre Channel Ethernet switch or pass‐through module
None Fibre Channel switch or pass‐through module
Fibre Channel Fibre Channel Ethernet switch or pass‐through module
Fibre Channel switch or pass‐through module
Fibre Channel switch or pass‐through module
Ethernet Fibre Channel Ethernet switch or pass‐through module
Ethernet switch or pass‐through module
Fibre Channel switch or pass‐through module
Fibre Channel Ethernet Ethernet switch or pass‐through module
Fibre Channel switch or pass‐through module
Ethernet switch or pass‐through module
The relationship of the blade mezzanine cards to the chassis fabrics
Stacking IOMs
You stack switches differently, depending on what you want to accomplish.
Switches for the M1000e have optional 10GbE uplinks and/or stacking connectors. These let you:
Manage/configure multiple switches as one with stacking
Consolidate uplinks from multiple enclosures into 2‐4 x 10GbE ports
30 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Reduce port consumption on external switches
Simplify cabling
Stacking IOMs
PowerConnectTM M6220 Switch
The Dell PowerConnect M6220 is a
key component of the FlexIO
architecture of the M1000e.
The PowerConnect M6220 packages a
high performance 24‐port stackable
Ethernet switch with modular bays
that can be populated with 10GbE or
stacking modules. Stacking switches
creates a single logical switch that can be managed and configured. This stacking capability is combined
with the ability to uplink to the Ethernet core via high bandwidth 10GbE connections.
The PowerConnect M6220 switch has four external Ethernet ports and two external option module bays
for 10GbE uplink or stacking modules. The four external gigabit Ethernet ports have standard RJ‐45
connectors, but the external option module bays require a 10GbE or stacking module. Each module bay
accepts a dual‐port 10GbE uplink module. Module bay 1 will also accept a dual‐port stacking module.
31 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
You can stack up to 12 PowerConnect M6220 switches for 240 1Gb Ethernet ports and 24 10Gb ports.
To stack the PowerConnect M6220, you must:
Install an optional 48Gb stacking module in bay 1 of each stacked switch.
Assign a separate unique ID for the stack.
Interconnect all the stacked switches.
The PowerConnect M6220 can be upgraded in place with stacking and 10GbE modules installed in the
option modules.
Cisco Switch
Cisco Ethernet switches can be purchased in one of
three base configurations:
Cisco Catalyst 3130X — 10G Rack Switch
Cisco Catalyst 3130G — GbE Rack Switch
Cisco Catalyst 3032 — Entry Level GbE Switch
These three switches cannot be differentiated by appearance.
Upgradability depends on the configuration option. For example, if you want to stack Cisco switches,
you need the Cisco Catalyst 3130G or 3130X.
You can stack up to nine CBS 3130 switches to create a single logical switch that:
Simplifies manageability
Consolidates uplinks
Lowers total cost of ownership (TCO)
Requires one of the following switches that allows Virtual Blade Switch interconnect:
o Cisco Catalyst 3130X — 10G Rack Switch
o Cisco Catalyst 3130G — GbE Rack Switch
Requires a Cisco proprietary cable
Some conditions apply:
You can stack PowerConnect switches with standalone PowerConnect switches.
You cannot stack Cisco Ethernet switches with standalone Cisco Ethernet switches.
You can stack both PowerConnect and Cisco switches between enclosures.
Cisco provides three base configurations. Upgrading requires a rip and replace of the existing base
configuration if it was not purchased with stacking and/or 10GbE options.
32 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Redundancy
Redundancy is the inclusion of multiple devices
that perform the same function to reduce the
likelihood of downtime when one of the devices
fails. The M1000e supports the following
redundancies:
CMC
Fabric
Fans
PSU
Fabric redundancy typically relies on additional configuration, software, and technologies such as NIC teaming or clustering.
CMC Redundancy
Each M1000e can house two CMCs, even though only one CMC is active at
any given time, and only one is required. The standby CMC mirrors the active one. If the active CMC
fails, the standby CMC takes over. You can configure the CMCs to alert you when a failover occurs.
On boot, the CMCs check to see if another CMC is active in the enclosure. The first CMC to boot
becomes active. The second one automatically becomes the standby CMC
Fabric Redundancy
I/O port redundancy within a fabric
Each mezzanine card or LOM has two ports on each blade. If one of the ports fails, traffic can be rerouted by software through the other port.
IOM redundancy within a fabric
Each of the three fabrics has two IOMs in each enclosure. If either of the IOMs fails, traffic can be rerouted by software through the alternate IOM, depending on how the fabric is configured.
Redundancy between fabrics
To have fabric redundancy, Fabrics B and C must use the same protocol. If Fabrics B and C are Ethernet, failure of any of the three fabrics is covered by the remaining fabrics. If Fabrics B and C use the same protocol, failure of either fabric is covered by the remaining one.
33 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Fan Redundancy
All nine fans are required within the M1000e enclosure to power up the chassis. If any of the fans fails,
the remaining eight fans can keep a fully populated enclosure cool until the failed fan can be replaced.
Fans are N+1 redundant, meaning that any single fan can fail without impacting system uptime or
reliability. If a fan fails, system behavior is dependent on the resultant temperatures of the system, as
monitored by the Server Module iDRAC and I/O Modules. The CMC continues to interpret the airflow
needs of each server and I/O module to control the fan speeds appropriately. The system will not ramp
the fans to full speed in the event of a fan failure unless deemed necessary by on‐board monitoring.
The M1000e is engineered for good sound quality in accordance with the Dell Enterprise acoustical
specification. Compared to previous generations of products, the fans have more levels of control,
allowing much finer tuning of the fan behavior. Firmware is optimized to choose the lowest fan speeds
and therefore the lowest acoustical output for any condition and configuration.
When the fans are running at normal rpm they fall just within unprotected hearing safety parameters.
However, when starting up, or during a components failure, the increase in fan speeds bring the noise
beyond unprotected regulations for noise, therefore hearing protection must be used.
If the enclosure is not fully populated, fewer fans may be required, but you should still keep all nine
working. The CMC can manage the fans so that only the requisite number runs at any time.
PSU Redundancy
The M1000e can house up to six PSUs but only requires a minimum of three. So installing any number
greater than three PSUs provides some degree of redundancy.
Four PSUs can cover the failure of any one PSU.
Six PSUs provides maximum redundancy.
No redundancy (N+0)
Power from all three power supplies on one AC circuit (grid) is used to power‐on the entire chassis.
34 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Power supply redundancy (N+1)
The additional power supply is kept as a spare, ensuring that a failure of any one power supply will not cause the servers or chassis to power‐down. All six power supplies are never in use at the same time. Based on load and availability, from two to four of the available power supplies are used at any given time. The rest are held ready for failover.
Grid redundancy (N+N)
All six power supplies are used at 50% capacity. Each bank of three power supplies connects to a separate AC grid.
Summary
Review the summary of this module before taking the self‐check to see how well you mastered the
topic.
The PowerEdge M1000e comprises all of the components needed to support blade networking
and clustering.
o The minimum configuration of the M1000e includes a CMC, three PSUs, and nine fans.
For the enclosure to be usable, you must add at least one M‐series blade.
o You can access the blades, crash cart connection, and operator panel from the front of
the M1000e.
o The following components are accessible from the back of the enclosure:
One or two CMCs (one on each upper right and left corner)
An integrated KVM (iKVM)
Up to six PSUs
Nine fans
Fabric connections (A, B, and C in two banks)
35 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
You should connect the cables on the M1000e in a specific order.
The M1000e is an air‐cooled enclosure. Air moves from front to back through three distinctive
paths:
o Through the blades, past the midplane, into a plenum, and out an exhaust fan
o Past the CMC, iKVM, and IOMs, into a plenum, and out an exhaust fan
o Through the PSUs
The components of the M1000e include:
o At least one CMC communicates with all of the enclosure components and with the
iDRACs built into each blade. You can configure the CMC to send e‐mail alerts or SNMP
trap alerts for warnings or errors related to:
Temperature
Hardware misconfiguration
Power outage
Fan speed
o The M1000e’s optional iKVM integrates into the enclosure to provide access to all
blades.
o IOMs communicate with the three fabrics supported by the M1000e:
Fabric A is always Ethernet and supports the enclosure’s private network.
Fabrics B and C can be:
o Ethernet
o Fibre Channel
o InfiniBand
The M1000e ships with all nine fans populated. The fans install in three banks of three fans.
The midplane facilitates communication between all of the devices in the enclosure.
o Fabric A is a dual 1Gb fabric that supports the enclosure’s internal network.
o Fabrics B and C can support various fabrics, including Fibre Chanel and InfiniBand.
You can install three to six PSUs in the M1000e, but you need at least four for redundancy.
The M1000e supports both PDUs available for 9G servers and two 10G server PDUs.
Hot‐swappable components in the M1000e include:
o CMC (if redundant with two CMCs)
o iKVM
o Individual blades
o IOMs
o Fans
o PSUs
IOMs and mezzanine cards on a fabric must use the same technology. While you can mix
mezzanine cards of the same technology on a fabric, both IOMs should be identical.
o The Fabric Consistency Check prevents accidental activation of misconfigured fabric
devices on a blade.
o The M6220 is a stackable 24‐port Ethernet switch designed for Fabric A, but 16 of the
ports are reserved for blades.
36 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Redundancy is the inclusion of multiple devices that perform the same function to reduce the
likelihood of downtime when one of the devices fails.
o When two CMCs are installed in the M1000e, the first to boot becomes the active CMC.
The other mirrors the active one and stands ready to take over if the active CMC fails.
o The M1000e supports:
I/O port redundancy within a fabric
IOM redundancy within a fabric
Redundancy between fabrics
o The M1000e ships with all nine fans installed.
o Four or more PSUs provide some degree of redundancy. For full redundancy, you should
install all six PSUs and attach them to two separate grids.
37 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Module 3: Key Components
Module Objectives
This module discusses the modular servers (blades) that
are compatible with the M1000e Modular Blade
Enclosure (M1000e).
After completing this module, you will be able to:
Identify M‐series blades.
List the features of M‐series blades.
Describe the differences between the M600, M605, M805, and M905.
Explain the benchmarks used to measure M‐series blades.
Explain the difference between clustering and virtualization.
Common Features
Blades are the reason you have a blade enclosure. This topic introduces the blades that are compatible
with the M1000e.
After completing this topic, you will be able to:
Identify M‐series blades.
Describe the features and options of M‐series blades.
Blades
The M1000e supports the following blades:
M600, an Intel‐based single‐height blade
M605, an AMD‐based single‐height blade
M805, an AMD‐based single‐height blade
M905, an AMD‐based single‐height blade
M610, an Intel based single‐height blade
M710, an Intel‐based full‐height blade
Features
M‐series blades share the following features:
Two mezzanine card slots with dual porting
o Ethernet
o Fibre Channel
o InfiniBand
38 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Midplane interface
Integrated Dell Remote Access Controller (iDRAC)
o IPMI 2.0
o Virtual media
o Virtual KVM
o Serial over LAN (SOL)
o Out‐of‐band GUI and CLI
Six I/O ports (with both mezzanine cards installed)
PCIe I/O technology
(M805 and M905) Up to 32GB DDR‐2 SDRAM or 64GB with qualified DIMMs
o Memory ECC, SDDC, Spare Bank, Mirroring
Up to two hot‐pluggable hard drives
o Static Disk Drive support
o SAS
o SATA
Integrated RAID
o CERC 6i RAID options for local drives
o SAS6/iR
Dual port embedded network interface card with TOE
The chart on this screen, shows the features of some of the Dell M‐Series Blade Servers
Features of Dell M‐series Blade Servers
Feature M600 M605 M805 M905 Form Factor Single‐height Single‐height Full‐height Full‐height Processors Intel AMD AMD AMD Processor wattage options
Standard and LV Standard and LV Standard, HE and SE
Standard, HE and SE
RAM slots 8 8 16 24 Total RAM To 64GB To 64GB To 128GB To 128GB Hard drive types SAS and SATA SAS and SATA SAS SAS LOMs 2 2 4 4 Mezzanine card slots
2 2 4 4
RAID card SAS/iR CERC 6i SAS/iR CERC 6i SAS/iR CERC 6i SAS/iR CERC 6i
In addition, both blades can:
Boot from iSCSI using Ethernet connections embedded on the system board
39 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Boot from Fibre Channel SAN
Support enterprise solutions:
o Oracle and SQL database program integration
o VMware program integration
o Dell | EMC integration
o Dell | EqualLogic iSCSI storage
o Dell PowerVault storage
o High performance computing cluster (HPCC)
40 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Mezzanine Cards
In addition to the two Ethernet
controllers embedded on the
system board, M‐series blades*
have two mezzanine card slots.
These slots enable you to install
mezzanine cards to communicate
with the fabrics.
For the blade to boot, the mezzanine card must match the fabric as determined by the I/O module
(IOM) installed in the enclosure.
If the card does not match the fabric, the Chassis Management Console (CMC) can be programmed to
send you an alert. Because the blade does not boot in this situation, you don’t have to worry about
causing any damage to the blade or the enclosure.
*The M710, M805 and M905 blade servers have four mezzanine slots.
Hard Drive Options
Although using local hard drives is optional, M‐series blades support the following internal storage
buses:
Serial Advanced Technology Attachment (SATA) II
SATA II (Serial Advanced Technology Attachment), a hot‐swappable, serial revision of the popular legacy
ATA bus, features:
A raw data transfer rate of 3.0 gigabits per second (Gb)
Significantly smaller connectors and cables
Dedicated bandwidth and cable for each device.
SATA II drives are less costly than their SAS counterparts and offer cost effective alternatives for
customers who have strict budgets and are less concerned with maximum mean time before failure
(MTBF) or performance.
SATA II drives operate at 7.2 RPM, considerably faster than previous generations. SATA II delivers a
blend of high performance and configuration convenience.
Serial Attached SCSI (SAS)
SAS combines elements of SCSI and SATA. It supports the traditional SCSI command structure, so SCSI
software tools continue to work.
Like SATA II, SAS’s raw data transfer rate is 3.0 Gb. SAS, however, is a full‐duplex bus, so data can be
routed in both directions, yielding higher simultaneous data transfers to and from multiple devices than
SATA II.
41 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Finally, SAS supports more devices and longer external cable lengths than SATA II. SAS is recommended
for the most high‐end, high‐demand enterprise server deployments.
Both architectures improve speed and reliability over previous generations of hard drive technology.
Embedded Ethernet Controllers
Two embedded Broadcom Ethernet controllers attach to Fabric A. These controllers support the TCP
offload engine (TOE).
The controllers’ circuitry is embedded in the planar as independent Ethernet interface devices. The
following information details the features of the LAN devices.
PCI‐e x4 interface
SMB for network manageability only with LOM1:
o IPMI
o ASF 2.0
64KB on‐chip packet buffer
MAC and SERDES interface integrated
Wake‐on‐LAN (WOL) 10/100 speeds
PXE is supported
iSCSI Firmware Assisted Boot capability
iSCSI protocol offload option enabled via hardware license key
All NICs are capable of running at 10/100/1000Mb even with the pass‐through modules. This speed was
not possible in previous pass‐through technology.
iDRAC
The Integrated DellTM Remote Access Controller (iDRAC) is a systems management hardware and
software solution that provides:
Remote management capabilities (out‐of‐band)
Crashed system recovery
Power control functions
iDRACs are integrated in M‐series blades. The iDRAC hardware component provides support for:
Virtual Media (vMedia)
Virtual KVM (vKVM)
Out‐of‐band graphical user interface (GUI)
Out‐of‐band inventory
Out‐of‐band status messaging
The iDRAC uses an integrated system‐on‐chip for the remote monitoring and system control. The iDRAC
co‐exists on the system board with the managed M‐series blade servers.
42 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
All network connections to the iDRAC are through the CMC Gb1 port. The CMC routes traffic to the
iDRACs on its servers through the enclosure’s private network. This private management network is
between the CMC and the iDRAC. Private network connects all the networks and the CMCs plus the
iKVM.
Differences
Although M‐series blades have many common features and can be used interchangeably—assuming
they have the same mezzanine cards—significant differences between models exist. This topic explores
those differences.
After completing this topic, you will be able to:
Identify differences between the M600, M605, M805, and M905.
43 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
M600 Unique Features
The M600 has the following features:
Up to two dual or quad core
Intel processors
Intel 5000P chipset
Fully buffered 567MHz RAM
To 64GB with qualified 8GB
DIMMs
M605 Unique Features
The M605 has the following features:
Up to two dual or quad core
AMD processors
NVIDIA MPC55 chipset
667 MHz, single/dual/quad‐
ranked DIMMs
To 64GB with qualified 9GB
DIMMs
44 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
M805 and M905 Unique Features
Leveraged Design
The system board on the PowerEdge M905 is the same
system board that is used on the PowerEdge M805. The
number of processors installed during the initial P.O.S.T.
determines the system name displayed upon P.O.S.T.
The only difference between the two systems
cosmetically is that the M805 has two processors and
two hyper‐transport bridge cards in processor sockets 3 and 4. Also, because of the processor reduction
from 4 to 2 processors, the total amount of memory supported is 128 GB of RAM on the PowerEdge
M805 versus the 196 GB supported on the PowerEdge M905.
Feature PowerEdge M805 (“Full Height” AMD Blade)
PowerEdge M905 (“Full Height” AMD Blade)
Chipset Dual Nvidia MCP 55 Dual Nvidia MCP 55
Processor Opteron 2000 series
(split plane) Dual or quad core
Dual core (non‐split plane)
AMD
Memory 16 X DDR2 667/800Mhz DIMMS (128 GB total)
24 X DDR2 667/800Mhz DIMMS (192 GB total)
Hard drives Hot‐pluggable 2 x 2.5” SAS Hot‐pluggable 2 x 2.5” SAS
SAS/RAID controllers Daughter card Options for: 1. LSI 1068e w/IR functionality
(SAS6/ir) 2. CERC6
Daughter card Options for: 1. LSI 1068e w/IR
functionality (SAS6/ir) 2. CERC6
Embedded LOMs 4 x GB Ethernet w/hardware TCP/IP Offload Engine, iSCSI Offload accelerator and iSCSI Firmware Boot
4 x GB Ethernet w/hardware TCP/IP Offload Engine, iSCSI Offload accelerator and iSCSI Firmware Boot
Other I/O Four x8 PCIe mezzanine cards Four x8 PCIe mezzanine cards
Embedded Management iDRAC iDRAC
45 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
External Ports 3 x USB 2.0 ports 3 x USB 2.0 ports
Internal Persistent Storage 1. CMC based 2. Dedicated Internal SC Slot
per blade for FI embedded hypervisor (same implementation as 805)
1. CMC based 2. Dedicated Internal SC Slot per blade for F1 embedded hypervisor (same implementation as 805)
Drive Controllers
The M‐series blade servers support up to two optional hard drives. The type of drives you can install
depends on the controller.
After completing this topic, you will be able to:
List the drive controllers available for M‐series blades.
Identify the features of the M‐series drive controllers.
Describe how to configure RAID options on the drive controllers.
Drive Controller Options
Optional storage cards provide internal RAID capabilities for the M‐series blade servers as described in
the following table:
These cards attach into a card slot under the hard drive bays in the front of the blade, not into one of
the I/O mezzanine (card slots).
CERC 6i is for specific applications for optimal performance (for example, Citrix). Read‐only applications
are best because the 128MB cache is not currently battery‐backed.
Dell also supports the option for no hard drives. In this configuration, no card would be installed.
46 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
CERC 6i
The CERC 6i is an internal RAID card for customers who require basic RAID functionality. Features
include:
Host‐based daughter card to connect to internal drives
PCI Express (PCIe) Host Interface
128MB cache
Support for RAID‐0 and RAID‐1
This version of the CERC 6i card does not provide a battery to back up the cache. So, it is possible to lose
data during a power outage.
CERC 6i Cache Policy
The cache controller writes a block of data to cache memory, which is much faster than writing to the
physical disk. The cache controller sends an acknowledgement of data transfer completion to the host
system.
The CERC 6i can use two types of caching:
With write‐back caching, the controller sends a data transfer completion signal to the host when
the controller cache has received all the data in a
transaction. The controller then writes the cached data
to the storage device when system activity is low or
when the write buffer approaches capacity. The cached
data is not written to the storage device immediately.
With write‐through caching, the controller sends a data
transfer completion signal to the host system when the
disk subsystem has received all the data in a
transaction.
The default cache setting is write‐through caching.
The risk of using write‐back cache is that the cached data can
be lost if there is a power failure before it is written to the
storage device. Write‐back caching has a performance
advantage over write‐through caching.
Write‐Back mode is available when the user selects Force WB with no battery in the CERC 6i interface
while booting the server. When Forced Write‐Back mode is selected, the virtual disk is in Write‐Back
mode even if the battery is not present.
47 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
The CERC 6i does not have a battery backup as of this writing. Because of the possibility of data loss, Dell
recommends that you use write‐through caching with the CERC 6i.
CERC 6i Configuration Utility
The CERC 6i's BIOS configuration utility can be accessed by pressing the <Ctrl><R> keys when booting
the blade. It is a storage management application embedded in the controller firmware that performs
the following actions independently of the operating system:
Configures and maintains RAID disk groups and virtual disks
Manages the RAID system
CTRL+R is to be used for initial setup and disaster recovery. Advanced features can be set through
OpenManageTM Storage Manager.
You can apply CTRL+R using only the keyboard. Other keystroke functionality:
The arrow keys move the cursor in a specific direction.
Pressing <Alt> and an underlined letter acts as a menu shortcut.
Pressing <Enter> activates the highlighted option.
Pressing <Tab> moves to the next control on the page.
Pressing <Esc> closes a pop‐up.
48 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Creating an Array on the SAS6/iR
This procedure configures the SAS6/iR to use RAID‐0 integrated striping (IS) or RAID‐1 integrated
mirroring (IM) on the blade’s hard drives.
Step 1 When the system is booting, press
<Ctrl><C> to enter the SAS6/iR
controller BIOS.
The system prompts you to press
<Ctrl><C>, then the text changes to
indicate that the SAS6/iR user
interface is loading.
If the system did not recognize the
<Ctrl><C> keystroke, it displays a list
of attached devices.
49 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Step 2 When the user interface loads, use
the Arrow keys to highlight and then
press <Enter> to select the desired
controller.
In this example, both the SAS5 and
SAS6 are listed. Notice that the SAS6
is listed with its chipset 1068E.
The controller indicates that it is
reading non‐volatile settings.
50 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Step 3
The Adapter Properties screen can be
manipulated by using the Arrow keys to
highlight RAID Properties and press
<Enter>.
Step 4 On the Select New Array Type screen,
use the arrow keys to highlight the type
of array you want to create and press
<Enter>.
Choose Create IM Volume for a
mirror or RAID‐1 array.
Choose Create IS Volume for a
striped or RAID‐0 array.
51 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Step 5 Create the striped volume.
1) For each drive to be included:
a) Use arrow keys to highlight the
Yellow text.
Disks that do not display yellow
text are not selectable. They
may already be used in
another configuration—either
another volume or a hot spare
configuration.
b) Press the spacebar to change
the RAID Disk selection to Yes.
2) Press <C> to create the striped
volume.
If the space bar is pressed on any other
column than the RAID disk column the
following message should display.
After selecting the disk drives and
pressing <C>, the SAS6/iR BIOS asks you
to create and save the new array.
Step 6 Choose Save Changes, then exit by
navigating with the arrow keys and
pressing <Enter>.
The interface gives you one last chance
to abandon the changes.
52 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Step 7 Press <F3> to finalize the new array.
The volume is now created.
The interface returns to the Adapter
Properties screen.
Step 8 Use the arrow keys to highlight SAS
Topology and press <Enter>.
53 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Step 9 Make sure the new virtual disk appears in
the list.
Step 10 Notice that the drives in Bay 0 and Bay 1
are listed as RAID physical disks, because
they are part of an array. The rest of the
bays show the drive manufacturer and
product number as well as version
number. This information can be obtained
for the drives in Bay 0 and Bay 1 by
expanding the IM Volume to show the
disks that are in that volume.
Step 11 Press <ESC> to exit the user interface.
SAS6/iR Virtual Disk Management
You can perform the following virtual disk management tasks with the SAS6/iR:
View virtual disk properties
Follow these steps to view the properties of IS and IM virtual disks:
1. Select a controller from the Adapter List in the Configuration Utility.
2. Select the RAID Properties option.
If there are no existing virtual disks, you will be prompted to create an IM or an IS virtual
disk.
If there is one existing virtual disk, select View Existing Array.
If there are two existing virtual disks, press <Alt><N> to view the next virtual disk.
3. Press <Enter> when the Manage Virtual Disk item is selected to manage the current virtual disk.
54 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Synchronize a virtual disk
Synchronizing a virtual disk means that the firmware synchronizes the data on the secondary disk(s) of
the mirror. Follow these steps to start synchronization for an IM virtual disk:
1. Select Synchronize Mirror.
2. Press <Y> to start the synchronization or <N> to cancel it.
This process can take several hours to complete.
Activate a virtual disk
A virtual disk can become inactive if, for example, it is removed from one SASS/iR controller or computer
and moved to another one. The Activate Mirror option lets you reactivate an inactive virtual disk that
has been added to a system. This option is only available when the selected virtual disk is currently
inactive.
1. Select Activate Mirror.
2. Press <Y> to proceed with the activation or press <N> to abandon it.
After a pause, the virtual disk will become active.
Activation of migrated virtual disks is only supported when the migrated virtual disk(s) is in an optimal
state and contains all the physical disks.
Delete a virtual disk
Before deleting a virtual disk, be sure to back up all data on the virtual disk that the customer wants to
keep.
Follow these steps to delete a selected virtual disk:
1. Select Delete Virtual Disk.
2. Press <Y> to delete the virtual disk or press <N> to abandon the deletion.
3. Press <F3> to confirm deletion of the virtual disk. After a pause, the firmware deletes the virtual
disk.
If the physical disks of a virtual disk are removed and the virtual disk's configuration is subsequently deleted from the SASS/iR controller, the physical disks show up only as simple disks with no RAID association if they are placed back onto the same SASS/iR controller. Once the virtual disk is removed from a SAS6/iR controller using the BIOS Configuration Utility (regardless of whether the physical disks are present), the virtual disk cannot be restored.
55 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Internal Storage
Dell blade server storage can be configured in the following ways:
Diskless Configuration
SATA Repeater Daughter Card
SAS6/iR Modular Daughter Card
CERC 6i Modular Daughter Card
After completing this topic, you will be able to:
Enumerate the types of blade server storage devices.
List the features of each type of storage.
Diskless Configurations
Characteristics of diskless configurations include:
No storage daughter card populated
Two HDD blanks populated
SAS/SATA backplane present (for cooling purposes)
SATA Repeater Daughter Card
Features of the SATA Repeater Daughter Card include:
Provides standalone hard‐disk drive support
No hot‐plug support
No RAID configurations supported
Two SATA HDDs supported
System BIOS configurable (Press F2 during POST)
BIOS Option: SATA Configuration is present only if a SATA controller card is installed
56 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
SAS6/iR Modular Daughter Card
Features of the SAS6/iR Modular Daughter Card include:
Up to two SAS or SATA hot‐pluggable HDDs supported
SAS and SATA HDD mixing is not supported
RAID‐0 or RAID‐1 supported
Press <Ctrl><C> to enter the SAS6/iR Configuration Utility
57 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
CERC 6i Modular Daughter Card
Features of the CERC 6i Modular Daughter Card include:
Up to two SAS hot‐pluggable HDDs supported
RAID‐0 or RAID‐1 supported
Read and write cache memory for improved performance
Cache is not battery backed up
58 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Storage Daughter Card Installation and Removal
This topic covers installing and removing blade storage daughter cards in Dell PowerEdge blade servers.
After completing this topic, you will be able to:
Explain how to add a storage card to a blade enclosure.
Explain how to remove a storage card from a blade enclosure.
59 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Blade Storage Daughter Card
Installation
Installation and removal of blade
storage daughter cards is
straightforward.
To install a blade storage daughter
card:
1. Align the controller board so that
the tabs on the metal system board
tray fit through the notches on the
edges of the controller board.
2. Slide the board towards the
connector on the system board
until the board's edge connector is
fully seated.
Blade Storage Daughter Card
Removal
The blade storage daughter card is
located underneath the hard drive
bays of the blade enclosure. To
remove the card:
1. Open the release lever to
disengage the controller
board edge connector
from the system board
connector.
2. Lift the controller board
straight up from the
system board.
60 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
SAS/SATA HDD Backplane Components
This section describes components of the SAS/SATA HDD backplane and the functions of HDD blanks in
Dell blade servers.
After completing this topic, you will be able to:
List the components of the SAS/SATA HDD backplane.
Describe the functionality of the SAS/SATA HDD Indicator LEDs.
Explain the function and convenience factors of HDD blanks in Dell blade servers.
List requirements for populating the HDD slots in a Dell blade server.
SAS/SATA HDD Backplane
Components of the backplane include the following:
Two SAS/SATA HDD Connectors for support of two SAS/SATA 2.5” HDDs
Cable‐less card edge interface to blade planar
Tool‐less HDD backplane installation/removal via latch and hook retention method
Common HDD backplane mechanical/electrical/ board solution shared between the M‐series
blade servers
A common HDD backplane mechanical/electrical/board solution shared between the M805 and
M905
For proper system cooling, the blade must have an HDD or an HDD blank installed in each HDD bay.
61 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
HDD Indicator LEDs
Features of the HDD status indicator
LEDs, found on the front of the
enclosure, include:
The HDD status indicator
LED is only functional for
RAID HDD configurations.
For non‐RAID
configurations, only the
drive‐activity indicator is
active.
o The HDD status
indicator LED is not
supported for SATA
Repeater SATA HDD
configurations.
Mechanical Issues
This topic covers HDD blanks and cooling
issues.
Dell PowerEdge blade servers leverage the
existing 2.5” HDD Blank, common to already
shipping platforms. To maintain proper
system thermals, HDD Blanks must be
installed for HDD slots that do not have a
2.5” SAS or SATA HDD populated.
62 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Clustering and Virtualization
Blades lend themselves to two popular server technologies: cluster computing and virtualization. In a
sense, clustering and virtualization are opposite ends of the server spectrum.
Clustering is grouping two or more servers together so that they function as a single unit.
Virtualization is causing a single server to behave as if it were two or more servers.
After completing this topic, you will be able to explain the difference between clustering and
virtualization.
Cluster Computing
Clustering is grouping two or more servers together so that they function as a single unit. Clustered
computers communicate with each other over a private local area network.
Different types of cluster architecture serve different computing needs.
High‐performance computing clusters (HPCCs): A high‐performance computing cluster (HPCC) is
a powerful and economical solution for enhancing your research environments. An HPCC is a
group of commercial off‐the‐shelf computers interlinked via high‐speed interconnects to create
a robust, high‐speed computing resource that rivals the performance of traditional mainframe
supercomputers at a fraction of the cost.
Grid computing: Grids work much like HPCCs, except that the nodes each retain their identities.
SETI@home is one example of a very large grid.
High availability (HA) clusters: Also known as failover clusters, these clusters improve the
availability of critical resources. Most HA clusters consist of two nodes; one active and one
redundant, The redundant node mirrors the active node and takes over its functionality in the
event of a failure.
Load‐balancing clusters: A front‐end controller allocates workload among the nodes in the
cluster to achieve maximum throughput and balance the workload of each node.
Blade servers give you the advantage of high‐density computing, reduced cabling, reduced cooling
requirements and optimized power.
Virtualization
Virtualization allows you to run multiple applications and operating systems independently on a single
server. Administrators can quickly move workloads from one virtual workspace to another—easily
prioritizing business needs while maximizing server resources. Flexibility in allocating computing
capacity allows the consolidation of applications and servers, easing IT management requirements and
lowering expenses.
Dell offers a range of virtualization solutions, including:
Microsoft® Virtual Server
VMware® ESX ServerTM
63 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Citrix® XenServerTM
Getting the most out of any of these solutions requires in‐depth training that goes beyond the scope of
this course.
Summary
The M1000e supports the following blades:
o M600, an Intel‐based single‐height blade
o M605, an AMD‐based single‐height blade
o M805 and M905, AMD‐based full‐height blades
M‐series blades:
o Support the same midplane interface
o Have an integrated iDRAC
o Have six I/O ports when both mezzanine cards are installed
o Have bays for two hot‐swappable hard drives
o Include Broadcom Ethernet controllers that attach to Fabric A
o Support two mezzanine cards to communicate with Fabrics B and C
o Support two internal hard drives that can be configured in a two‐drive RAID array
Differences between the M‐series blade servers include:
o The M600 is an Intel‐based, single‐height blade that is designed for heavy use in data
centers.
o The M605 is an AMD‐based, single‐height blade that supports TPM.
You have a choice of 2 RAID drive controllers.
o The CERC 6i provides RAID‐0 and RAID‐1 functionality and 128MB RAID caching.
Although you can set the CERC 6i to use write‐back caching, you should avoid this
option, because the lack of battery backup places your data at risk to power loss. The
CERC 6i’s configuration utility is commonly known as CTRL+R, because you press those
keys when booting the blade to access the utility.
o The SAS6/iR is an entry‐level RAID‐capable drive controller for cost sensitive customers.
Administrators can perform RAID configuration on the SAS6/iR by pressing <Ctrl><C>
during POST. The SAS6/iR configuration utility provides an option to configure the
Virtual disks. The user can activate or delete the virtual disks.
Internal Storage consists of a variety of daughter card configurations.
o Types of blade storage configurations:
Diskless Configuration, where there are no disks, and HDD blanks are used to
populate the backplane
SATA Repeater Daughter Card
SAS6/iR Modular Daughter Card
CERC 6i Modular Daughter Card
o SAS/SATA HDD Backplane:
Has connectors for two SAS/SATA 2.5” HDDs
Has Cable‐less card edge interface to Blade Planar
64 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Has Tool‐less HDD backplane installation/removal
Common HDD Backplane mechanical/electrical/board solution shared between
models
LEDs for drive activity and HDD status, but HDD status LED is only functional in
RAID HDD configuration
The 2.5” HDD blank is common to existing platforms
HDD blanks must be installed for any slot without a SAS/SATA HDD populated
o Blade storage daughter card installation/removal:
To install, align the controller board so that the tabs on the metal system board
tray fit through the notches in the edges of the controller board, then slide the
board towards the connector on the system board until the board’s edge
connector is fully seated.
To remove, open the release lever to disengage the controller board edge
connector from the system board connector, then lift the controller board
straight up from the system board.
Different types of cluster architecture serve different computing needs.
Blade servers give you the advantage of high‐density computing, reduced cabling, reduced
cooling requirements and optimized power.
65 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Module 4: Systems Management Hardware
Module Objectives
This module outlines the systems management options available with the M‐series blade servers.
After completing this module, you will be able to describe the management features of M1000e blade
servers.
Management Basics
Dell’s M1000e modular server delivers major enhancements in management features. These
enhancements derive from direct customer feedback and assist customers deploying modular (blade)
servers. Each subsystem has been reviewed and adjusted to:
Optimize efficiencies
Minimize impact to existing management tools and processes
Provide future growth opportunities to standards‐based management
After completing this topic, you will be able to:
Explain how management information is communicated through the enclosure.
Describe the M1000e enclosure management features.
Management Connections
Management connections in the M1000e enclosure transfer health and control traffic throughout the
enclosure. The system management fabric uses 100 Mb Ethernet over differential pairs routed to each
module.
Two 100 Mb interfaces, one switched and one unswitched, communicate between Chassis
Management Controllers (CMCs).
Every Integrated DRAC (iDRAC) module has a management network link to each CMC.
66 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Redundancy is at the controller level. Failure of any individual link triggers failover to the redundant
CMC, as long as the system is configured with a redundant CMC.
Management Features
Efficiency is the core of the blade philosophy, and easy, secure management is at the heart of efficiency
in a blade environment. Management features available on M‐series blade servers include:
Chassis Management Controller (CMC) The CMC is the central point for infrastructure monitoring, inventory, and control. It provides dedicated
Ethernet management connectivity to iDRACs and I/O switches. Adding an optional second CMC
provides redundancy. Failure of the active CMC will trigger failover to the standby CMC if a second CMC
is installed.
A 24‐port Ethernet switch is reserved for internal communication between the iDR4C on the blades to
the CMC. It also provides communication between the CMC and the external management network.
If two CMCs are installed, the heartbeat for CMC redundancy is also present and CMC redundancy is
supported over the internal network. The internal network is separate from the networks used by the
host LOMs and the mezzanine cards in the blades.
The CMC provides internal persistent storage with a flash card.
67 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
iDRAC
The M1000e supports the Integrated Dell Remote Access Controller (iDRAC) that is integrated on each M‐series blade.
All network connections to the iDRAC are through the CMC network interface (Gb1). The CMC routes traffic to the iDRACs on its servers through a private, internal network. This private management network is outside of the servers’ data path and outside of the operating system's control (out‐of‐band). The managed servers' in‐band network interfaces are accessed through I/O modules (lOMs) installed in the chassis.
Each blade server module's iDRAC has its own IP address and can be accessed, if security settings allow, directly through a supported browser, Telnet, SSH, or IPMI client on the management station. The internal network is separate from the networks used by the host LOM.
iKVM
The iKVM:
Switch local KVM (front or rear)
Seamlessly tier into Dell and Avocent KVM infrastructure
Provides local console access to all the
blades at the CMC (one at a time)
LCD operator panel Supports simplified initial chassis setup
wizard
Provides enhanced local troubleshooting
and information
Displays configuration, status information,
and system health for each blade’s iDRAC
and IOM, respectively
68 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Operator Panel and LCD
The operator panel contains the local user
interface that provides blade and enclosure
health and information. Functions include:
Chassis level diagnostic LEDs
LCD
Power button
The operator panel is always powered, even in
chassis standby mode.
The operator panel pivots so that you can adjust it for easy viewing, wherever the enclosure is in the
rack. It also retracts fully into the chassis when it is not needed.
Interpreting the Operator Panel LED
The LED in the operator panel changes color to indicate
the enclosure’s status:
Blue — All devices in the enclosure are
functioning properly.
Orange — Check the status of the devices using:
o The LCD menus
o Remote chassis management
69 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
LCD Menus
The LCD shows chassis information, chassis
status and faults, and major blade server
module faults. It also provides an interface for
blade server module and CMC network
configuration. Included is an error message
when no CMC is present in the chassis. Service
Tag, Asset Tag and IP Address are easily visible
on every M1000e modular system through the
LCD display.
Health: Displays alerts, status information, and health information for:
Enclosure
Blades
IOM
Information:
Module model/type and user defined name
Hardware configurations
Service tag
IP Address
You can initially set the CMC’s IP address using the LCD Configuration Wizard.
Chassis Management Controller
The Chassis Management Controller
(CMC) is a hot‐pluggable systems
management hardware and software
solution. The M1000e always has at least
one CMC, but may have two. Redundancy
is provided in an Active–Standby pairing
of the modules in the enclosure. Failover
occurs when the active module fails or
degrades.
After completing this topic, you will be able to:
Describe the features and capabilities of the CMC.
Explain how to access the CMC and how it communicates with other devices in the enclosure.
Explain redundancy in M‐series CMCs.
70 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Features and Capabilities
The CMC has its own microprocessor and memory but receives power from the enclosure. It interfaces
through dual stacking Ethernet ports and one serial port. The CMC’s serial port interface provides
common management of up to six I/O modules (IOMs) through a single connection.
The CMC provides secure remote management access and power control functions for the chassis and
installed modules. It manages or facilitates the management of the following:
Status, Inventory and Alerting for blade server modules, chassis infrastructure and I/O Modules
Centralized configuration for iDRAC, I/O Modules, CMC, and iKVM
Power sequencing of modules in conjunction with the defined chassis power states
Power budget management and allocation
Configuration of the embedded management switch, which facilitates external access to
manageable modules
Remote user management
User interface entry point (Web, Telnet, SSH, serial)
Monitoring and alerting for chassis environmental conditions or component health thresholds
This includes but is not limited to the following:
o Real time power consumption
o Power supplies
o Fans
o Power allocation
o Temperature
o CMC redundancy
o I/O fabric consistency
Ethernet traffic management (firewall)
Private VLAN
Private DHCP service
Private TFTP service
Virtual Media
Virtual KVM (vKVM) interface between the external IP network and the blade server module
iDRAC
Firmware management of CMC, IOM, iDRAC, iKVM
All serial port configurations of IOMs go through the serial port of the CMC
The CMC can also be configured to send e‐mail alerts or SNMP trap alerts for warnings or errors related
to:
Temperatures
Hardware configuration errors
Power outages
Fan speeds
71 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Protocols and other tools supported by the CMC include:
SMASH‐CLP features providing chassis blade server module power control functions
WSMAN
CIM XML
SNMP
racadm
SSL/SSH
CMC Cabling
At least one CMC must be installed in the primary
CMC bay for the enclosure to power up. If a second,
optional CMC module is installed, failover protection and hot‐plug replacement is available.
Each CMC has two Ethernet ports:
Gb1 is used to connect to the external management network.
Gb2 will allow CMCs in adjacent enclosures to be daisy‐chained.
With basic cabling, you connect the Gb1 port to the management network and leave the Gb2 port
unused. DO NOT connect both ports to your network. Gb2 is only for
daisy‐chaining CMCs.
A 24‐port Ethernet switch provides internal communication between:
The iDRAC on each blade
IOMs
Optional KVM
Optional second, redundant CMC
Daisy‐chain Connection
If you have more than one enclosure in a rack, you can reduce the
number of connections to the management network by daisy‐
chaining up to four chassis together.
If each of four chassis contains a redundant CMC, by daisy‐
chaining you reduce the number of management network
connections required from eight to two.
If each chassis has only one CMC, you reduce the connections
required from four to one.
When daisy‐chaining chassis together, Gb1 is the uplink port and Gb2
is the stacking port. A Gb1 port must connect to the management
network or to the Gb2 port of the CMC in an enclosure that is closer
to network. The Gb2 port must only receive a connection from a Gb1 port further from the chain.
72 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Create separate chains for the CMCs in the primary CMC slot and the second CMC slot.
Redundancy
When both CMC slots in an M1000e are populated, you have CMC redundancy.
The first CMC booted is the active (primary) CMC, while the other becomes the standby CMC. The
standby CMC mirrors the active CMC. The settings for both CMCs are held in NVRAM on the M1000e
chassis operator panel.
Both CMCs must run the same firmware revision. If the firmware revisions differ, the system reports as
redundancy degraded.
Failovers can occur when:
The active CMC fails.
The network cable is removed from the active CMC.
The active CMC is removed from the chassis.
A CMC firmware flash is initiated on the active CMC.
If a CMC failover occurs:
All external iDRAC sessions and all active CMC sessions disconnect and must be reestablished.
Internal iDRAC sessions move to the new active CMC.
Initial Configuration
There are four ways to configure the CMC:
LCD Configuration Wizard: This wizard is accessed via the LCD control panel on the front of the
chassis.
CLI using a null modem cable: This is a direct serial connection to the CMC via a null modem
cable.
Syntax: setniccfg ‐s <IP address><network mask><gateway>
Example: setniccfg ‐s192.168.1.60 255.255.255.0 192.168.1.1
Web GUI: Can be used if you set your management station's IP address to be on the same
subnet as the default CMC IP address.
The default IP address is 192.168.0.120.
CLI using entry 17 on the OSCAR interface for the iKVM.
73 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Integrated KVM
The integrated keyboard, video, and mouse (iKVM)
switch for the M1000e is an optional, hot‐pluggable
module. It provides local keyboard, mouse, and video
access to any of the 16 blades in the enclosure. It also
provides access to the additional Dell CMC command‐
line console.
After completing this topic, you will be able to:
List the features and capabilities of the iKVM.
Access the OSCAR Setup dialog box.
Features
The optional Avocent® Integrated KVM Switch Module (iKVM):
Provides local keyboard, mouse, and video access to any of the 16 blades in an M1000e
enclosure
Provides access to the additional Dell CMC command line console
Maintains all blade connections as input is switched from each blade
Occupies a single slot accessible through the rear of the enclosure
Redirects local blade video, keyboard, and mouse electrical interfaces to either:
o The iKVM local ports
o The M1000e front panel ports
Local iKVM access can be remotely disabled on a per blade basis, using the blade's iDRAC interface
(access is enabled by default).
The iKVM also has an Analog Console Interface (ACI) compatible RJ45 port that allows the iKVM to tie
the interface to a KVM appliance upstream of the iKVM via CAT5 cabling. Although the ACI port is an RJ‐
45 connector and uses CAT5 (or better) cabling, it is not an Ethernet network interface port. It is only
used for connection to external KVM switches with Analog Rack Interface (ARI) ports. The ACI port
reduces cost and complexity by giving access for sixteen simultaneous servers using only one port on an
external KVM switch.
The iKVM available for the M1000E chassis is analog. The iDRAC handles functions, such as virtual media
or remote desktop, that digital KVMs provided in the previous blade models.
74 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Capabilities
iKVM user interface: On‐Screen Configuration and Activity Reporting (OSCAR). The OSCAR
display allows you to select, configure and manage the blades in the enclosure.
Security: The OSCAR user interface allows you to protect the system with a screen‐saver
password.
Video: The iKVM video connections support video display resolutions ranging from 640 x 480 at
60 Hz up to 1280 x 1024 at 60 Hz.
Flash upgrades: You can update the iKVM firmware using:
o CMC Web‐based interface (requires the firmware file)
o racadm fwupdate command (requires a TFTP server to upload the file using the OSCAR
interface)
CMC access through entry 17: The CMC command‐line interface can be accessed through entry
17 on the OSCAR interface.
Enabling and disabling access: You can enable or disable access through the front panel or CMC
(through entry 17) with the CMC.
75 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
OSCAR Configuration
The OSCAR menu has a common feel found throughout the Dell
range of KVMs.
To access the Setup menu:
1. Press <Print Screen> to launch OSCAR.
2. Click Setup to access the Setup dialog box.
You can soft switch the iKVM between blades, using a hotkey
sequence, by pressing <Print Screen> and then typing the first
few characters of its name or number. If you previously set a
delay time (the number of seconds before the Main dialog box is
displayed after <Print Screen> is pressed) and you press the key
sequences before that time has elapsed, the OSCAR interface
does not display.
Integrated DRAC
The Integrated Dell Remote Access Controller (iDRAC) is a systems management hardware and software
solution that provides:
Remote management capabilities
Crashed system recovery
76 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Power control functions
After completing this topic, you will be able to:
Describe the features of the iDRAC.
List the configuration utilities available on the iDRAC.
Embedded but Separate
iDRACs are embedded in the M‐series family of blades. The iDRAC combines the functionality of the
baseboard management controller (BMC) and Dell Remote Access Controller (DRAC) used on traditional
servers into a single device on the blade’s system board.
The iDRAC has both software and hardware components. The hardware component provides:
Virtual media (vMedia) and virtual KVM (vKVM) functions
Out‐of‐band graphical user interface (GUI)
Out‐of‐band status and inventory information
The iDRAC is powered by standby power. With a blade in the chassis, the iDRAC is always on and
provides management and monitoring capabilities.
Management Features
The iDRAC provides the following management features:
Console Redirection: Provides remote system keyboard, video, and mouse functions.
Dell OpenManage software integration: Enables you to launch the iDRAC Web interface from
Dell OpenManage Server Administrator or IT Assistant.
Dynamic Domain Name System (DDNS) registration.
iDRAC alerts: Alerts you to potential managed node issues through an e‐mail message or SNMP
trap.
IPMI support.
Microsoft Active Directory authentication: Centralizes iDRAC user IDs and passwords in Active
Directory using the standard schema or an extended schema.
Monitoring: Provides access to system information and status of components.
Password‐level security management: Prevents unauthorized access to a remote system.
Remote power management: Provides remote power management functions, such as
shutdown and reset, from a management console.
Remote system management and monitoring: Using:
o Web interface.
o Local racadm command‐line interface.
o SM‐CLP command line over a Telnet/SSH connection.
Role‐based authority: Provides assignable permissions for different systems management tasks.
Secure Sockets Layer (SSL) encryption: Provides secure remote system management through
the Web interface.
77 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
System log access: Provides access to the system event log, the iDRAC log, and the last crash
screen of the crashed or unresponsive system. Access is independent of the operating system
state.
Virtual Media: Enables a managed server to access a local media drive on the management
station or CD/DVD ISO images on a network share.
Management Interface
The iDRAC network interface is disabled by default. You must configure it before you can access the
iDRAC. After accessing the iDRAC on the network, you can access it at its assigned IP address with:
The iDRAC Web interface
Telnet
SSH
Supported network management protocols, such as:
o Simple Network Management Protocol (SNMP)
o Intelligent Platform Management Interface (IPMI)
You can configure the iDRAC to send you an e‐mail message or Simple Network Management Protocol
(SNMP) trap alerts for warnings or errors. To help you diagnose the probable cause of a system crash,
iDRAC can log event data and capture an image of the screen when it detects that the system has
crashed.
The CMC monitors and manages all components in the enclosure, including iDRACs. The LCD that is part
of the enclosure displays the iDRAC's integration information and the health status of the blade.
All network connections to the iDRAC are through the network interface (Gb1) on the active CMC. The
CMC routes traffic to the iDRACs on its servers through a private, internal network.
78 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Traditional IPMI‐based baseboard management controller (BMC) features such as hardware monitoring
and power control are supported. With up to two simultaneous sessions per blade, remote
management satisfies virtually any usage model.
Virtual media over IP is standard. The iDRAC emulates:
DVD‐ROM
CD‐ROM
USB Flash Drive
ISO image
USB/legacy floppy
Floppy image
Full USB access is available through the blade server module front panel. Connection to console
redirection and virtual media is through the CMC, with an option available to encrypt those sessions.
Highlights of the iDRAC solution include:
Dedicated management interface
Virtual Media
Console redirection via Virtual KVM
IPMI 2.0 Out‐Of‐Band management
79 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Serial over LAN
SMASH CLP
Blade status and inventory
Active power management
Local Security and Integration with Active Directory
Security Features
The iDRAC provides the following security features:
User authentication through Microsoft Active Directory (optional) hardware‐stored user IDs and
passwords
Role‐based authority, which enables an administrator to configure privileges for each user
User ID and password configuration through the Web interface or SM‐CLP
SM‐CLP and Web interfaces, which support 128‐bit SSL encryption and 40‐bit SSL encryption (for
countries where 128‐bit is not acceptable)
Session time‐out configuration (in seconds) through the Web interface or SM‐CLP
Configurable IP ports (where applicable)
Telnet does not support SSL encryption.
Secure Shell (SSH), which uses an encrypted transport layer for higher security
Login failure limits per IP address, with login blocking from the IP when the limit is exceeded
Limited IP address range for clients connecting to the iDRAC
Out‐of‐band Communication
All network connections to the iDRAC are through the CMC network interface (Gb1). The CMC routes
traffic to the iDRACs on its servers through a private, internal network. Features of the out‐of‐band
network include:
Dedicated internal 100Mb Ethernet connection to each CMC
o The iDRAC connects to the CMC via dedicated, fully redundant 100 Mbps Ethernet
connections.
Passes through the midplane to a dedicated 24‐port Ethernet switch on the
CMC
Exposed to the outside world through the CMC’s external Management
Ethernet interface (Gb1)
o This connection is distinct from the three redundant data Fabrics A, B and C.
Unlike previous generations of Dell blade servers, the iDRAC’s connectivity is
independent of, and in addition to, the onboard GbE LOMs on the blade.
Configurable Web or command‐line interface (CLI) using Virtual Media
80 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Configuration Interfaces
You have several options for configuring the iDRAC. However, using more than one configuration
method at a time can generate unpredictable results.
81 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
iDRAC Configuration Utility
Access this at boot time to set up the network and basic security features and for enabling other
features. Press <Ctrl>+<E> during POST to access the utility. If the Web interface to the CMC is not being
used, this is the best opportunity to turn on the Ethernet interface to the iDRAC and set the IP address
for remote access.
iDRAC Web interface
The iDRAC Web interface is a browser‐based management application that you can use to interactively
manage the iDRAC and monitor the managed server. It is the primary interface for day‐to‐day tasks such
as:
Monitoring system health
Viewing the system event log
Managing local iDRAC users
Launching the CMC web interface and console redirection sessions
CMC Web Interface
In addition to monitoring and managing the chassis, you can use the CMC web interface to:
View the status of a managed server
Configure iDRAC network settings
Start, stop, or reset the managed server
The current firmware does not properly perform a graceful operating system shut down.
Chassis LCD Panel
The LCD panel on the chassis containing the iDRAC can be used to view the high‐level status of the
servers in the chassis. During initial configuration of the CMC, the configuration wizard allows you to
enable configuration of iDRAC networking.
Local racadm
The local racadm command‐line interface runs on the managed server. The server is accessed from
either the iKVM or a console redirection session initiated from the iDRAC Web interface.
Racadm is installed on the managed server when you install Dell OpenManage Server Administrator.
Racadm commands provide access to nearly all iDRAC features. You can:
Inspect sensor data, system event logs records, and the current status and configuration values
maintained in the iDRAC.
Alter iDRAC configuration values.
Manage local users.
Enable and disable features.
Perform power functions such as shutting down or rebooting the managed server.
82 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
The CMC also has a racadm interface available through slot 17 of the iKVM. It can be accessed remotely
via Telnet, SSH, or from a management station.
OpenManage Integration
M‐series blade servers fully integrate Dell OpenManage, OpenManage Server Administrator, and IT
Assistant. After completing this module, you will be able to:
Describe how OpenManage Server Administrator and IT Assistant integrate with the M1000e.
OpenManage Server Administrator
OpenManage Server Administrator (OMSA) supports the following M1000e management functions:
Power monitoring: This list shows some of the features that you can monitor:
o Amps per power supply
o Aggregate watts consumed
o Aggregate cumulative power drawn
o Resetting of peak power values
o Maximum peak power (watts and amps)
o Power supply input voltage
Dell OpenManage software integration: You can launch the iDRAC web interface from OMSA.
83 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
OMSA is covered in greater depth in the OpenManage Server Administrator module of this course.
IT Assistant Integration ‐ now referred to as DMC ‐ provides Power Monitoring for servers supporting
PMBus reports with the following types of data:
Aggregation of power and energy consumed by various groups of servers
Amperage per power supply
Energy and power consumption per blade
Peak power and peak amperage per server
Power consumption and total energy consumed
IT Assistant is covered in greater depth in the IT Assistant module of this course.
84 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
ITAssistant
Summary
Review the summary of this module before taking the self‐check to see how well you mastered the
topic.
Management basic concepts include:
o Management information from each blade and module in the enclosure to the CMC
over redundant 100BaseT paths.
o Management features of M‐series blade servers include:
CMC
iDRAC
iKVM
LCD operator panel
o The retractable operator panel contains the LCD panel, power button, and diagnostic
LEDs.
o The LED on the operator panel is blue when all devices are functioning properly. It blinks
orange to indicate that one of the devices in the enclosure has a problem.
The M‐series operator panel has two menus:
o Health — displays alerts, status information, and health information for the devices in
the enclosure.
o Information — displays module information, service tag, and IP address.
The Chassis Management Controller (CMC) is a hot‐pluggable systems management hardware
and software solution.
85 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
o Status, Inventory and Alerting for blade server modules, chassis infrastructure and I/O
Modules
o The CMC can be configured to send e‐mail alerts or SNMP trap alerts for warnings or
errors related to:
Temperatures
Hardware configuration errors
Power outages
Fan speeds
o The CMC has two Ethernet ports
Gb1 is used to connect to the external management network.
Gb2 will allow CMCs in adjacent enclosures to be daisy‐chained.
o Daisy‐chaining CMCs in the rack significantly reduces the number of cables required.
o CMC redundancy is attained when both CMC slots in an M1000e are populated.
o Failovers can occur when:
The network cable is removed from the active CMC.
The active CMC is removed from the chassis.
A CMC firmware flash is initiated on the active CMC.
o The four ways to configure the CMC are:
LCD Configuration Wizard
CLI using a null modem cable
Web GUI
CLI using port 17 on the iKVM
The integrated keyboard, video, and mouse (iKVM) switch for the M1000e is an optional, hot‐
pluggable module.
o The M‐series iKVM is:
Optional
Hot‐swappable
Analog
o The iKVM provides:
An on‐screen utility to select, configure, and manage blades in the enclosure
A screen‐saver password
Flash upgrades
Access to the CMC CLI
o You can configure OSCAR by pressing <Print Screen> and clicking Setup in the resulting
menu.
The Integrated Dell Remote Access Controller (iDRAC) is a systems management hardware and
software solution.
o The iDRAC resides on the blade’s system board and provides
Virtual media (vMedia) and virtual KVM (vKVM) functions
An out‐of‐band graphical user interface (GUI)
Out‐of‐band status and inventory information
o M‐series iDRAC management features include:
86 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Console Redirection
Dell OpenManage software integration
Dynamic Domain Name System (DDNS) registration
iDRAC alerts
IPMI support
Microsoft Active Directory authentication
Monitoring
Password‐level security management
Remote power management
Remote system management and monitoring
Role‐based authority
Secure Sockets Layer (SSL) encryption
System log access
o The iDRAC communicates through the Gb1 connection on the CMC.
o All communication with the iDRAC passes through the CMC.
o The following configuration interfaces are available on the iDRAC:
iDRAC Configuration Utility
iDRAC Web interface
CMC Web interface
Chassis LCD Panel
Local racadm
iVM‐CLI
iDRAC SM‐CLP
IPMI
M‐series blade servers fully integrate Dell OpenManage.
o You can use OpenManage Server Administrator with an M1000e enclosure to:
Monitor power usage
Launch the iDRAC Web interface
o Power management features available through IT Assistant include:
Aggregation of power and energy consumed by various groups of servers
Amperage per power supply
Energy and Power consumption per blade
Peak power and Peak amperage per server
Power consumption and total energy consumed
87 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Module 5: Dell OpenManage Software
Module Objectives
After completing this module, you will be able to:
Explain how to use Dell OpenManage Server
Administrator (OMSA) to manage
interactions between blade servers and a
storage area network (SAN).
System Management Basics
This topic introduces the basic concepts of system management. After completing this topic, you will be
able to:
Define systems management.
Describe Dell OpenManage.
List the systems management protocols supported by OpenManage.
88 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
What Is Management?
Systems Management uses tools (hardware and software) to perform mundane, simple, or complex
tasks. Systems management can call on multiple components:
Hardware
Remote management hardware, in addition to the base hardware that makes the server work
Software
Agents to monitor status and generate alerts
Software to manage same or different systems
Dell OpenManage System Overview
Dell OpenManage systems management software is a suite of application programs for Dell systems that
allows you to manage your system with proactive monitoring, diagnosis, notification, and remote
access. Each of your managed systems will use applications that include Server Administrator, IT
Administrator, and remote access controller (RAC) software. A management station can be used to
remotely manage one or more managed systems from a central location. By installing IT Assistant on a
management station, you can effectively manage from one to thousands of remote managed systems.
89 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
This graphic illustrates the relationship between a management station and its managed systems. It also
shows the operating systems and the Dell OpenManage software products that may be installed on the
managed systems.
The Dell OpenManage Model
Review the OpenManage Model on
this page. Note that OpenManage
consists of Deployment Tools,
Monitoring Tools, and Maintenance
Tools. Review the names of all the
individual tools in the OpenManage
suite, and then focus on the tools in
the Deployment category.
90 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Systems Build and Update Utility
The Dell Systems Build and Update
Utility is a bootable CD that provides:
Step‐by‐step configuration for
an unattended install of the
operating system.
Installation support for
Windows and Linux operating
systems.
Dell‐optimized device drivers,
firmware, and utilities for Dell
PowerEdgeTM servers.
91 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Deployment toolkit
Dell OpenManage Deployment Toolkit
(DTK) provides some of the
components that facilitate unattended
server deployment. DTK includes a set
of utilities that supports both CLI and
file‐based parameters, as well as
documentation that describes its
components. DTK components are:
Unattended server hardware
configuration
Utility partition (WP)
Basic Input/Output System
(BIOS)
Redundant Array of
Independent Disks (RAID)
Network Interface Cards (NICs)
Unattended installation of Dell drivers and management agents per operating system. This feature
contains a set of tools and scripts that can be used by most third‐party deployment software that
has an Automated Deployment Service (ADS).
Server Management: Remote Management
Remote Management Hardware Remote management hardware is often optional. It generally enables you to perform extra
management tasks that would not otherwise be possible. For Dell systems, this additional hardware is
part of a Dell Remote Access Controller (DRAC), which bundles the hardware and software needed for
remote management.
Remote management hardware:
Often works out‐of‐band
Does not require operating system interaction
Functions even when the system is powered down
Systems management software uses management protocols (such as SNMP or CIM) to IMPI components
(such as BMC) and to change settings. These protocols are embedded within the operating system.
Therefore, they are in‐band, available only when the operating system is running.
92 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
OutofBand Management An additional adapter card is required to perform out‐of‐band management. Dell recommends the DRAC
family of adapters. These cards enable you to remotely access the information supplied by IMPI and
control the manageable hardware.
DRAC hardware supplied
interfaces include:
Serial
Telnet
Ethernet
InBand Management In‐band management is
accomplished via instrumentation
in the operating system. In‐band
management means that a
management system is dealing
with a managed node through the
operating system and through
agents and drivers.
The Dell Remote Access hardware
supports “in‐band” management,
and allows you to perform three
major types of tasks:
View system related
management information
such as temperature, voltages via new BMC
Perform power management
Take control of the server GUI remotely
Server Management: In‐band Interfaces
OpenManage Server Administrator Server Administrator provides a comprehensive, one‐to‐one systems management solution from an
integrated, Web browser‐based GUI (the Server Administrator home page), and from a command line
interface (CLI) through the operating system. Server Administrator is designed for system administrators
to both locally and remotely manage systems on a network.
93 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
IT Assistant IT Assistant provides a one‐to‐many management solution when OpenManage Server Administrator
(OMSA) is also installed on the managed systems. Server Administrator enables you to configure the
RAC.
Systems Management Protocols
Protocols can be described as:
Rules of communication between agents and the system management application
Industry‐standard technologies for cost‐effective management
Protocols gather and transmit management information. Systems management protocols include:
Simple Network Management Protocol (SNMP)
Common Information Method (CIM)
Intelligent Platform Management Interface (IPMI)
Review
Systems management protocols supported by Dell OpenManage software include:
Common Information Method (CIM)
Intelligent Platform Management Interface (IPMI)
Simple Network Management Protocol (SNMP)
Dell OpenManage Server Administrator
Bundled with Dell OpenManage software, Dell OpenManage Server Administrator (OMSA) provides
configuration and monitoring, and replacing functionality. This topic covers the basics of OMSA.
After completing this topic, you will be able to:
Describe the functions of the OMSA management service.
Describe the OMSA storage management service.
Describe the OMSA remote access controller.
Describe IT Assistant.
Describe the Dell Online Diagnostics.
If you need more in‐depth information, you may want to take the Dell OpenManage course.
94 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
OMSA: Instrumentation Service
The Server Administrator Instrumentation Service monitors the health of a system and provides rapid
access to detailed fault and performance information. The reporting and viewing features allow retrieval
of overall health status for all chassis in the system. At the subsystem level, users can view information
about the voltages, temperatures, fan speed, and memory in the system. A detailed account of every
relevant cost of ownership (COO) detail about your system can be seen in summary view. Version
information for BIOS, firmware, operating system, and all installed software is easy to retrieve.
Additionally, systems administrators can
use the Instrumentation Service to
perform the following essential tasks:
Specify minimum and maximum
values for certain critical
components. The values, called
thresholds, determine the range
in which a warning event for that
component occurs.
Specify how the system responds
when a warning or failure event
occurs. Users can configure the
actions that a system takes in
response to notifications of
warning and failure events.
Populate all of the user‐specific
values for the system, such as the
name of the system, the phone
number of the system’s primary
user, the depreciation method,
whether the system is leased or
owned, and so on.
OMSA Storage Management Service
The Dell OpenManage applications used to
control and monitor the storage subsystem
are OpenManage Storage Management
(OMSM). The application provides storage
management information in an integrated
graphical view. Features of the Storage
Management Service include:
The Storage Management Service
enables you to view the status of local and remote storage attached to a monitored system
95 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Supports SCSE, SATA, and SAS
Does not support Fibre Channel
Enables you to perform controller and enclosure functions for all supported controllers and
enclosures from a single graphical or command line interface without using controller BIOS
utilities
Protects your data by helping you:
o Configure data redundancy
o Assign hot spares
o Rebuild failed drives
Provides a graphical interface that is wizard‐driven with features for novice and advanced users
and detailed online help
Provides a command line interface that is fully featured and scriptable
96 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
OMSA: Remote Access Controller
The Server Administrator Remote
Controller (RAC) provides a complete
remote system management solution for
systems equipped with any of the
following instrumentation:
Integrated Dell Remote Access
Controller (iDRAC)
Dell Remote Access Card (DRAC) 5
Embedded Remote Access (ERA)
controller
ERA Option (ERA/O) card
ERA/MC controller
Remote access requires instrumentation
that supports either:
Simple Network Management
Protocol (SNMP)
Common Information Mode (CIM)
These hardware and software solutions are collectively known as Remote Access Controllers (RACs).
Users may choose to perform some of these functions through the BMC, but working through a RAC is
more efficient.
The RAC provides:
Remote access to an inoperable system, enabling you to get the system up and running as
quickly as possible.
Alert notification when a system is down.
Remote restart of a system.
Logs of the probable causes of system crashes and the most recent crash screen.
Remote access through the Server Administrator home page or by directly accessing the
controller’s IP address using a supported browser.
97 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Monitoring Tools: IT Assistant
Use e‐mail, paging, or console
alerting to monitor faults with:
o Disks
o Fans
o Memory
o Thermal conditions
o Voltage
Inventory and asset reporting
that includes:
o Service tag
o Cost of ownership
information
o Specifics of BIOS,
processor(s), and memory
In‐context launch of device‐
specific tools, such as:
o Storage Manager
o Server Administrator
o Remote Access Controller
o PowerConnect Switch Manager
Monitoring Tools: Baseboard Management Controller
The baseboard management controller
(BMC) is responsible for monitoring the
health of the server and components
associated with the server’s system board. It
does not usually monitor devices on the I/O
buses, and it operates independently of the
system board.
The monitoring technology is based on the
Intelligent Platform Management Interface
(IPMI) standard. This standard is defined in
conjunction with Intel and other Original
Equipment Manufacturers, including Dell.
IMPI is being used in the systems
management working group at the
Distributed Management Task Force
(DTMF).
98 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
The Dell BMC supports IPMI 1.5 and later. It delivers:
Monitoring of server hardware.
Alerts when potential or actual faults are encountered.
Network and serial access to control server power and reset the server regardless of the server’s
power status.
Maintenance Tools: Dell Server Update Utility
The Dell Server Update Utility (SUU)
simplifies the task of updating BIOS,
firmware, and drivers on servers. Insert
the Dell Server Updates DVD into a
server’s DVD drive to automatically:
Inventory the Dell firmware and
drivers on that server.
Compare the installed
configuration with the content on
the Dell Server Updates DVD.
Report on discrepancies and
recommend updates, including
any prerequisites that impact the update sequence.
Update and (if needed) reboot the server.
The Dell Server Update Utility is designed to simplify single‐server updates and is available on the Dell
Server Updates DVD included as part of the Dell OpenManage Subscription Service.
Maintenance Tools: Dell Online Diagnostics
The Dell Online Diagnostics:
Provides testing of the system
without bringing it down:
The system administrator can run
the diagnostics as a preventative
measure, or after a non‐fatal
failure, to verify the status of the
system hardware.
Helps determine if the cause of the
failure is the hardware:
The online diagnostics enable the administrator to determine if the problem is hardware and to
make a more accurate assessment of the problem.
99 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Provides hardware Field Replacement Unit (FRU) isolation.
Dell Online Diagnostics are now available on the following sources:
Dell Systems Service and Diagnostics Tools CD
Dell Systems Management Tools and Documentation DVD
The OpenManage Subscription Service
Summary
Review the summary of this module before taking the self‐check to see how well you mastered the
topic.
Basic systems management concepts include:
o Systems management consists of:
Using a variety of tools to remotely manage computer hardware.
o Dell OpenManage consists of:
Deployment tools
Monitoring tools
Maintenance tools
o The Systems Build and Update Utility provides:
Configuration for an unattended operating system installation
Installation support for Windows and Red Hat Linux
Dell‐optimized device drivers, firmware, and utilities
o The Deployment toolkit includes components that facilitate unattended server
deployment.
o Systems management protocols supported by Dell OpenManage software include:
Common Information Method (CIM)
Intelligent Platform Management Interface (IPMI)
Simple Network Management Protocol (SNMP)
OpenManage Server Administrator (OMSA) provides configuration, monitoring, and reporting
functions.
o The OMSA Instrumentation service monitors the health of a system and provides rapid
access.
o The Dell OpenManage Storage Management application provides storage management
information.
o The OMSA remote access controller provides a complete remote system management
solution instrumentation:
Dell Remote Access Card (DRAC) 5
Embedded Remote Access (ERA) controller
100 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
ERA Option
(ERA/O)
card
ERA/MC
controller
o Using IT Assistant
you can:
Be alerted
of specific
system
faults
Generate
inventory
and asset
reports
Launch
device‐
specific tools
o The Dell Online Diagnostics:
Enables you to test a system without downing it
Helps distinguish between hardware and software issues
Provides hardware isolation
Module 6: iDRAC The Integrated Dell Remote Access Controller (iDRAC) is a fully functional DRAC that is embedded on the
system board of M‐series blade servers. This module assumes you are already familiar with DRAC
functionality. If not, you should review the DRAC chapter of the PowerEdge server course.
After completing this module, you will be able to:
Describe the basic features of the iDRAC
Explain how to configure the iDRAC
iDRAC Management and Security Features
The Integrated Dell Remote Access Controller (iDRAC) is a systems management hardware and software
solution that provides remote management capabilities, crashed system recovery, and power control
functions for Dell PowerEdge systems.
101 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
After completing this topic, you will be able to:
Describe the iDRAC’s management features
Describe the iDRAC’s security features
iDRAC Management Features
The iDRAC provides the following management features:
Dynamic Domain Name System (DDNS) Registration: Provides the ability to register the iDRAC
name to the Domain Name Servers (DNS).
Remote system management and monitoring: Provides management and monitoring
capabilities using:
o A Web interface
o The local racadm command‐line interface (CU)
o The SM‐CLP command line over a Telnet/SSH connection Support for Microsoft Active Directory authentication: Centralizes iDRAC user IDs and
passwords in Active Directory using the standard schema or Dell extended schema.
Console Redirection: Provides remote keyboard, video, and mouse functions.
Virtual Media: Enables a managed server to access a local media drive on the management
station or CD, DVD, floppy, CD/DVD ISO images or floppy images on a network share. Monitoring: Provides access to system information and status of components.
Access to system logs: Provides access to the system event log, the iDRAC log, and the last crash
screen.
Dell OpenManage software integration: Enables you to launch the iDRAC web interface from
Dell OpenManage Server Administrator or IT Assistant.
iDRAC alert: Alerts you to potential managed node issues through an e‐mail, IPMI, alerts or
SNMP trap.
Remote power management: Provides remote power management functions, such as
shutdown and power up, from a management console.
Intelligent Platform Management Interface (IPMI) support: Provides support for IPMI features
as defined in the IPMI Specification version 1.5 and 2.0.
Secure Sockets Layer (SSL) encryption: Provides secure remote system management through
the Web interface or SSH.
Password‐level security management: Prevents unauthorized access to a remote system by
using DRAC defined users and password authentication.
Role‐based authority: Provides assignable permissions for different systems management tasks.
iDRAC Security Features
The iDRAC provides the following security features:
User authentication through Microsoft Active Directory (optional) or hardware‐stored user IDs
and passwords
102 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Role‐based authority, which enables an administrator to configure specific privileges for each
user
User ID and password configuration through the Web interface or SM‐CLP
SM‐CLP and Web interfaces, which support 128‐bit SSL encryption and 40‐bit SSL encryption (for
countries where 128‐bit is not acceptable)
Session time‐out configuration (in seconds) through the Web interface, SM‐CLP, or racadm
Configurable IP ports (where applicable)
Secure Shell (SSH), which uses an encrypted transport layer for higher security
Telnet does not support SSL encryption.
Login failure limits per IP address, with login blocking from the IP address when the limit is
exceeded
Limited IP address range for clients connecting to the iDRAC
iDRAC Security Features
When opening firewalls for remote access to an iDRAC, you need to make sure that all of the necessary
ports are available. The iDRAC client and listening ports are:
iDRAC client ports iDRAC server listening ports
Port Function Port Function
25 SMTP 22 SSH
53 DNS 23 Telnet
68 DHCP‐assigned IP address 80 HTTP
69 TFTP 443 HTTPS
162 SNMP traps 623 RMCP/RMCP+
636 LDAPS 3668 Virtual Media Service
3269 LDAPS 3669 Virtual Media Service
3770 Virtual Media Secure Service
3771 Virtual Media Secure Service
5900 Console redirection keyboard and mouse
5901 Console redirection video
All ports except 623 are configurable.
103 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Configuration Tasks
Since the iDRAC is part of the blade’s system board, no installation is required. But you still need to do
several configuration tasks to make it useful. This topic discusses these tasks in detail.
After completing this topic, you will be able to explain how to:
Configure the iDRAC
Configure the management station
Configure the managed systems
Use the iDRAC Web interface
Configuring the Management Station
A management station is a computer used to monitor and manage the PowerEdge servers and other
modules in the chassis.
The following are the software installation and configuration tasks that set up a management station to
work with the iDRAC. Before you begin configuring the iDRAC, follow the procedures in this section to
ensure that you have installed and configured the tools you will need.
Step Procedure
1. Set up the management station network
To access the iDRAC, the management station must be on the same network as the CMC RJ45 connection port labeled “GB1.” You can isolate the CMC network from the network the managed server is on, so that your management station may have LAN access to the iDRAC but not to the managed server. Using the iDRAC console redirection feature (see Using GUI Console Redirection), you can access the managed server’s console even if you do not have network access to the server's ports. You can also use iDRAC facilities to perform several management functions on the managed server, such as rebooting the computer.
2. Install and configure a supported Web browser.
For a list of supported Web browsers, see Supported Web Browsers. Note : The iDRAC Web interface is not supported on 64‐bit Web browsers. If you open a 64‐bit browser, access the Console Redirection page, and attempt to install the plug‐in, the installation procedure fails. If this error was not acknowledged and you repeat this procedure, the Console Redirect Page loads even though the plug‐in installation fails during your first attempt. This issue occurs because the Web browser stores the plug‐in information in the profile directory even though the plug‐in installation procedure failed. To fix this issue, install and run a supported 32‐bit Web browser and log in to the iDRAC.
If you use a proxy server, make sure the browser bypasses the proxy for local addresses.
104 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Step Procedure
Make sure the iDRAC is in the Trusted Domains list.
If you use Firefox, disable the Whitelist feature. Otherwise you must install a console redirection viewer for each iDRAC you access—even though all viewers are identical.
3. Install a Java Runtime Environment (JRE).
This step is optional for stations running Internet Explorer. If you use Internet Explorer, an ActiveX control is provided for the console viewer. You can also use the Java console viewer with Internet Explorer if you install a JRE and configure the console viewer in iDRAC Web interface before you launch the viewer. See Configuring Console Redirection in the iDRAC web Interface for more information. If you use Firefox you must install an IRE (or a Java Development Kit [JDK]) to use the console redirection feature. The console viewer is a Java application that is downloaded to the management station from the iDRAC web interface and then launched with Java web Start on the management station. Go to java.sun.com to install a JRE or JDK. Version 1.6 (Java 6.0) or higher is recommended. Note: In the Java Console, set the Runtime Settings parameter to a value of ‐xmx256M.
4. Install Secure Shell clients, if required.
By default, the iDRAC Telnet service is disabled and the SSH service is enabled. Since telnet is an insecure protocol, you should use it only if you cannot install an SSH client or your network connection is otherwise secured. Note: The iDRAC supports only one active SSH connection. If an SSH session is already active, the iDRAC denies all other SSH connection attempts.
5. Install a TFTP server, if required.
Trivial File Transfer Protocol (TFTP) is a simplified form of the File Transfer Protocol (FTP). It is used to update iDRAC firmware using SMCLP. Certificates can be transferred to the iDRAC using racadam or SMCLP. If you choose to use SM‐CLP or racadmracadm when you perform these tasks, an active TFTP server must be running on the computer that can be accessed by the iDRAC. Note: No TFTP server is required if you only use the iDRAC Web interface to upload new iDRAC firmware or install SSL certificates.
6. Install Dell OpenManage IT Assistant.
Your system includes the Dell OpenManage System Management Software Kit. This kit includes, but is not limited to, the following components:
Dell Systems Service and Diagnostics Tools CD
Dell Systems Management Tools and Documentation DVD
The OpenManage Subscription Service
105 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Step Procedure
Use the Dell System Management Consoles CD to install the management console software, including Dell OpenManage IT Assistant, on the management station. For instructions on installing this software, see your Quick Installation Guide.
Configuring the iDRAC
Before you can configure the iDRAC, the blade must be installed in an enclosure with at least one
functioning CMC and IOM. The following list describes the configuration tasks you may need to perform.
It also points to resources you can use to accomplish these tasks.
Step Procedure
1. Configure the management station.
Set up a management station by installing Dell OpenManage software, a browser, and other software utilities as described in Configuring the Management Station.
2. Configure iDRAC networking. Enable the iDRAC network and configure IP, netmask, gateway, and DNs addresses. Note: Changing the iDRAC network settings terminates all current network connections to the iDRAC.
The option to configure the server using the LCD panel is available only during the CMC initial configuration. Once the chassis is deployed, the LCD panel cannot be used to reconfigure the iDRAC.
The LCD panel can be used to enable DHCP to configure the iDRAC network. If you want to assign static addresses, you must use the iDRAC Configuration Utility or the CMC Web interface.
Chassis LCD Panel‐see the Dell Chassis Management Controller User’s Guide.
DRAC Configuration Utility — see LAN.
CMC Web Interface — see Configuring Networking Using the CMC Web Interface.
racadmracadm — see cfgLanNetworking.
3. Configure iDRAC users. Set up the local iDRAC users and permissions. The iDRAC holds a table of sixteen local users in firmware. You can set usernames, passwords, and roles for these users.
iDRAC Configuration Utility (configures administrative user only): see LAN User Configuration
iDRAC Web interface: see Adding and Configuring iDRAC Users
racadmracadm: see Adding an iDRAC User 4. Configure Active Directory. In addition to the local iDRAC users, you can use Microsoft Active
Directory to authenticate iDRAC user logins. See Using the iDRAC with Microsoft Active Directory.
106 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Step Procedure
5. Configure IP filtering and blocking.
In addition to user authentication, you can prevent unauthorized access by rejecting connection attempts from IP addresses outside of a defined range and by temporarily blocking connections from IP addresses where authentication has failed multiple times within a configurable time span.
iDRAC Web interface: see Configuring IP Filtering and IP Blocking
racadmracadm: see Configuring IP Filtering (IpRange), Configuring IP Blocking
6. Configure Platform Events. Platform events occur when the iDRAC detects a warning or critical condition from one of the managed server’s sensors. Configure platform Event Filters (PEFs) to choose the events you want to detect, such as rebooting the managed server.
iDRAC web interface — see Configuring Platform Event Filters (PEF)
racadmracadm — see Configuring PEF Configure Platform Event Traps (PETs) to send alert notifications to an IP address, such as a management station with IPMI software, or to send an e‐mail to a specified e‐mail address.
iDRAC Web interface — see Configuring Platform Event Traps (PET)
racadmracadm — Configuring PET 7. Configure serial over LAN (SOL).
Serial Over LAN (SOL) is an IPMI feature that allows you to redirect the managed server's serial port I/O over the network. SOL enables the iDRAC console redirection feature.
iDRAC Web interface — see Configuring Serial Over LAN
See also Using GUI Console Redirection. 8. Configure iDRAC services Enable or disable the iDRAC network services, such as Telnet, SSH,
and the Web server interface, and reconfigure ports and other service parameters.
iDRAC Web interface: see Configuring iDRAC Services
racadm: see Configuring iDRAC Telnet and SSH Services Using Local racadm
9. Configure Secure Sockets Layer (SSL).
Configure SSL for the iDRAC Web server.
iDRAC Web interface: see secure sockets Layer (SSL)
racadm: see cfgRacSecurity, sslcsrgen, sslcertupload, sslcertdownload, sslcertview
10. Configure virtual media for loading an operating system.
Configure the virtual media feature so that you can install the operating system on the PowerEdge server. Virtual media allows the managed server to access media devices on the management station
or CD/DVD, ISO images, and floppy images as if they were devices
on the managed server, among other functions.
107 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Step Procedure
iDRAC Web interface — see Configuring and Using Virtual Media
iDRAC configuration utility — see Virtual Media Note: Using the iDRAC configuration utility, you can only attach or detach virtual media devices.
11. Install the managed server software.
Install the Microsoft Windows or Linux operating system on the PowerEdge server using virtual media and then install the Dell Open Manage software on the managed PowerEdge server and set up the last crash screen feature.
Console redirection — see Installing the Software on the Managed Server
iVM‐CLI: see Using the Virtual Media Command Line Interface Utility
12. Configure the managed server’s Last Crash Screen feature.
Set up the managed server so that the iDRAC can capture the screen image after an operating system crash or freeze. See Configuring the Managed Server to Capture the Last Crash Screen, Disabling the Windows Automatic Reboot Option.
13. Configure networking using the CMC Web interface.
Note: You must have Chassis Configuration Administrator privilege to set up iDRAC network settings from the CMC.
The default CMC user is root and the default password is calvin.
The CMC IP address can be found in the iDRAC Web interface by clicking System Remote Access CMC. You can also launch the CMC Web interface from this page.
1. Use your Web browser to log in to the CMC Web user interface using a Web address of the form https://czCMC‐IP‐eddress> or https://<CMC‐DNS‐name>. 2. Enter the CMC username and password and click OK. 3. Click the plus (+) symbol next to Chassis in the left column, then click Servers. 4. Deploy iDRAC. 5. Enable the LAN for the server by checking the checkbox next to the server beneath the Enable LAN heading. 6. Enable or disable IPMI over LAN by checking or unchecking the checkbox next to the server beneath the Enable IPMI over LAN heading. Note: By default, DHCP is disabled and a default IP address, netmask and gateway are provided. 7. Check the box to enable DHCP for the blade server. 8. If DHCP is disabled, enter the static IP address, netmask, and default gateway for the server. 9. Click Apply at the bottom of the page.
108 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Step Procedure
14. Update the iDRAC firmware as needed.
Updating the iDRAC firmware installs a new firmware image in the iDRAC flash memory. You can update the firmware using any of the following methods:
SM‐CLP load command
iDRAC Web interface
Dell Update Package (for Linux or Microsoft Windows)
DOs iDRAC Firmware update utility
CMC Web interface (only if iDRAC firmware is corrupted)
Configure the Managed Server
The following are the tasks to set up the managed server to enhance your remote management
capabilities.
Task Details
Install the management software on the managed server
The Dell management software includes the following features:
Local racadm CLI: allows you to configure and administer the iDRAC from the managed system. It is a powerful tool for scripting configuration and management tasks.
Server Administrator: a Web interface that allows you to administer the remote system from a remote host on the network.
Server Administrator Instrumentation Service o Provides access to detailed fault and performance
information gathered by industry‐standard systems management agents and allows remote administration of monitored systems.
o Displays logs of commands issued to or by the system, monitored hardware events, POST events, and system alerts. You can view logs on the home page, print or save them as reports, and send them by e‐mail to a designated service contact.
Server Administration Storage Management Service: provides storage management information in an integrated graphical view.
Use the Dell Systems Management Tools and Documentation DVD to install Server Administrator. For instructions on installing this software, see your Quick Installation Guide.
109 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Task Details
Configure the managed server to capture the last crash screen.
The iDRAC can capture the last crash screen so that you can view it in the Web interface to help troubleshoot the cause of the managed system crash. Follow these steps to enable the last crash screen feature. 1. Install the managed server software. For more information about
installing the managed server software, see the Server Administrator User's Guide. 2. If you are running a Microsoft windows operating system, ensure that the Automatically Reboot feature is deselected in the Windows Startup and Recovery Settings. See Disabling the windows Automatic Reboot Option. 3. Enable the Last Crash screen (disabled by default) in the iDRAC web interface. To enable the Last Crash Screen in the iDRAC web interface, click System Remote Access ‐ iDRAC Network/Security Services, then check the Enable checkbox under the Automatic System Recovery Agent Settings heading. To enable the Last Crash Screen using local racadm, open a command prompt on the managed system and type the following command: racadm config ‐g clgRacTuning ‐o cfgRacTuneAsrEnable 1 4. In the Server Administrator Web‐based interface, enable the Auto Recovery timer and set the Auto Recovery action to Reset, Power Off, or Power Cycle. For information about how to configure the Auto Recovery timer, see the Server Administrator User’s Guide. To ensure that the last crash screen can be captured, the Auto Recovery timer should be set to 60 seconds. The default setting is 480 seconds. The Last Crash screen is not available when the Auto Recovery action is set to Shutdown or Power Cycle if the managed server is powered off.
Disable the Windows Automatic Reboot option.
To ensure that the iDRAC can capture the Last Crash Screen, disable the Automatic Reboot option on managed servers running Microsoft Windows Server. 1. Open the Windows Control Panel and double‐click the System icon. 2. Click the Advanced tab. 3. Under Startup and Recovery, click Settings. 4. Deselect the Automatically Reboot check box. 5. Click OK twice.
110 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Configuring the iDRAC Using the Web Interface
Use the iDRAC Web interface for everyday systems management tasks. You can:
Configure the iDRAC properties and users
Perform remote management tasks
Troubleshoot a remote (managed) system for problems
Most Web interface configuration tasks can also be performed with local racadm commands or with SM‐
CLP commands.
Local racadm commands are executed from the managed server. For more information about
local racadm, see Using the Local racadm Command Line Interface.
SM‐CLP commands are executed in a shell that can be accessed remotely with a Telnet or SSH
connection. For more information about SM‐CLP, see Using the iDRAC SM‐CLP Command Line
Interface.
Configuring the iDRAC Using the Web Interface
The following list explains the tasks you can accomplish with the iDRAC Web interface.
Task Details
Access the iDRAC Web interface To access the DRAC Web interface: 1. Open a supported web browser window. 2. Type the following text string in the Address field:
If the default port has not been changed, Type: https://cziDsQ4c‐IP‐address> and press <Enter>.
The iDRAC Login window appears. You must log in to the DRAC with valid credentials.
Configure the iDRAC NIC. Assuming that the iDRAC has already been configured and is accessible on the network, you can:
Configure the network and IPMI LAN settings
Configure IP Filtering and IP Blocking
Configure platform events. Platform event configuration provides a mechanism for configuring the iDRAC to perform selected actions on certain event messages. The actions include:
No action
Reboot system
Power cycle system
Power off system
Generate an alert
111 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Task Details
Configure platform event filters (PEFs).
When a platform event occurs (for example, a battery warning assert), a system event is generated and recorded in the System Event Log (SEL). If this event matches a platform event filter (PEF) that is enabled and you have configured the filter to generate an alert, a PET or e‐mail alert is sent to up to four configured destinations. If the same platform event filter is also configured to perform an action (such as rebooting the system), the action is performed. Note: Configure platform event filters before you configure the platform event traps or e‐mail alert settings. To configure PEFs: 1. Log in to the iDRAC Web interface. 2. Click System. 3. Click the Alert Management tab. 4. On the Platform Events page, do either of the following:
Place a check next to the Generate Alert heading to group select all events or to check each individual event
5. Click the radio button below the action you would like to enable for each event. Only one action can be set for each event. 6. Click Apply. Note: Generate Alert must be enabled for an alert to be sent to any valid, configured destination (PET or e‐mail).
Configure platform event traps (PETs).
1. Log in to the remote system using a supported web browser. 2. Ensure that you followed the procedures in Configuring Platform Event Filters (PEFs). 3. Configure your PET destination IP address:
a. Click the Enable checkbox next to the Destination Number you would like to activate. b. Enter an IP address in the Destination IP Address box. Note: The destination community string must be the same as the iDR4C community string. c. Click Apply.
Note: To successfully send a trap, configure the Community String value on the Network Configuration page. The Community String value indicates the community string to use in a Simple Network Management Protocol (SNMP) alert trap sent from the iDRAC. SNMP alert traps are transmitted by the iDRAC when a platform event occurs. The default setting for the Community String is Public.
d. Click Send to test the configured alert (if desired). e. Repeat for any remaining destination numbers.
112 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Task Details
Configure e‐mail alerts 1. Log in to the remote system using a supported Web browser. 2. Ensure that you followed the procedures in Configuring Platform Event Filters (PEF). 3. Configure your e‐mail alert settings.
a. On the Alert Management tab, click E‐mail Alert Settings. 4. Configure your e‐mail alert destination.
a. In the E‐mail Alert Number column, click a destination number. There are four possible destinations to receive alerts. b. Ensure that the Enabled checkbox is selected. c. In the Destination Email Address field, type a valid e‐mail address. d. Click Apply.
Note: To successfully send a test e‐mail, the SMTP Server Address must be configured on the Network Configuration page. The IP address of the SMTP Server communicates with the DRAC to send e‐mail alerts when a platform event occurs.
e. Click Send to test the configured e‐mail alert (if desired). f. Repeat step a through step e for any remaining e‐mail alert settings.
Using Console Redirection
The iDRAC console redirection feature enables you to access the local console remotely in either graphic
or text mode. You do not have to sit in front of each server to perform all the routine maintenance. You
can manage servers from your desktop or laptop computer wherever you are.
Console Redirection Considerations
The Console Redirection page enables you to manage the remote system by using the keyboard, video,
and mouse on your local management station to control the corresponding devices on a remote
managed server. This feature can be used in conjunction with the Virtual Media feature to perform
remote software installations.
The following conditions apply:
A maximum of two simultaneous console redirection sessions are supported. Both sessions view
the same managed server console simultaneously.
The first logged‐in user will be notified if a second user connects.
113 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
A console redirection session should not be launched from a Web browser on the managed
system for its own console redirection.
The managed system should be set to a screen resolution and refresh rate listed in the table.
The screen resolution on the management station should be set higher than the resolution of
the managed system.
A supported Web browser must be installed on the management station.
If you are using Firefox or want to use the Java Viewer when using Internet Explorer, you must
install a Java Runtime Environment (JRE) on the management station.
Screen Resolution Refresh Rate (Hz)
640 x 480 60, 72, 75, 85
720 x 400 70
800 x 600 60, 70, 72, 75, 85
1024 x 768 60, 70, 72, 75, 85
1280 x 1024 60
Configuring Console Redirection in the iDRAC Web Interface
To configure console redirection in the iDRAC Web interface, perform the following steps:
Access the iDRAC’s Web interface from the management station.
Click System and then click the Console tab.
Click Configuration to open the Console Redirection Configuration page.
Using the Video Viewer
The Video Viewer provides a user interface between the management station and the managed system.
It enables you to see the managed server's desktop and control its mouse and keyboard functions from
your management station. The Video Viewer provides various control adjustments. The Help menu
provides for more information on these functions.
When you connect to the remote system, the Video Viewer starts in a separate window. You may need
to adjust the color mode and synchronize the mouse pointers. The bar across the top of the viewer
contains a menu where you can adjust the viewer settings.
Using the Video Viewer
The following table describes the menu options to adjust the viewer settings.
View
Pause Temporarily pauses console redirection.
Resume Resumes console redirection.
Refresh Redraws the viewer screen image.
Capture Current Screen Captures the current remote system screen to a .bmp file on Windows or a .png file on Linux. A dialog box is displayed that allows you to save the file to a specified location.
Full Screen To make the Video Viewer expand into full screen mode, select Full Screen from the Video menu.
114 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Exit When you have finished using the Console and have logged out (using the remote system’s logout procedure), select Exit from the Video menu to close the Video Viewer window.
Mouse
Synchronize Cursor The Mouse menu enables you to synchronize the cursor so that the mouse on the client is redirected to the mouse on the server.
Options
Color Mode Allows you to select a color depth to improve performance over the network. For example, if you are installing software from virtual media, you can choose the lowest color depth (3‐bit gray), so that less network bandwidth is used by the console viewer, leaving more bandwidth for transferring data from the media. The color mode can be set to:
15‐bit color
7‐bit color
4‐bit color
4‐bit gray
3‐bit gray Media
Virtual Media Wizard The Media menu provides access to the Virtual Media Wizard, which allows you to redirect to a device or image such as a:
Floppy drive
Floppy image
CD
DVD
Image in ISO format
USB flash drive For information about the Virtual Media feature, see Configuring and Using Virtual Media. You must keep the Console Viewer window active when using Virtual Media.
Help
Activates the Help menu.
Summary
Review the summary of this module before taking the self‐check to see how well you mastered the
topic.
The iDRAC implements several management and security features.
115 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
o The iDRAC provides both in‐band and out‐of‐band management solutions such as blade
power management, logging critical events, and remote management.
o M‐series iDRACs provide security features such as:
Active Directory authentication and authentication using local users
40‐bit and 128‐bit encryption
IP address restriction when login fails
Since the iDRAC is part of the blade’s system board, no installation is required, but some
configuration is.
o To set up the management station, the administrator should set up Web browsers,
TFTP, and SSH clients.
o When configuring the iDRAC, configure:
Network settings
Users
Active Directory
Event filters
The vKVM
vMedia
o You can also enable services such as Web, SSH, and Telnet.
o The iDRAC network can also be configured using the CMC Web interface.
o When configuring the managed server, the System administrator must:
Install OpenManage System software
Configure the Auto Recovery feature
Enable the Last Crash Screen capture
o System administrators can configure the settings of iDRAC NIC, Platform Event Trap, E‐
mail alerts, and Platform Event Filters through iDRAC Web interface. Most Web
interface configuration tasks can also be performed with local racadm commands or
with SM‐CLP commands.
116 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Module 7: Network Fabrics and IOMs This module provides information on Network
Fabrics A, B and C.
After completing this module, you will be able to:
Describe the features of Fabric A and the TOE
key
Describe the features and use restrictions of
mezzanine Fabrics B and C
Compare the features of the three types of
mezzanine cards compatible with Dell
M1000e blade servers
Explain how InfiniBand technology functions and list its advantages
Describe the features and functionality of InfiniBand mezzanine cards and IOM
Chassis Overview
The M1000e chassis can house 6 x I/O
modules. This allows for a greater diversity
of roles for all of the enclosed blade servers.
The 6 x I/O slots are classified as three
separate fabrics. Each fabric contains two
slots, 1 and 2, which relate to the two ports
found on the server side I/O cards. The
fabrics can be used independently of each
other, though each fabric must contain the
same technology. For example, Fabric A is
hardwired to the two LANs On Motherboard
(LOMs) on the blade server mainboards, so
the I/O modules in Fabric A must be
Ethernet. Fabrics B and C can be used for
Ethernet, Fibre Channel and InfiniBand.
117 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Fabric A Broadcom Ethernet Controllers
Fabric A consists of two embedded Broadcom Ethernet controllers. The controller was designed as a
Convergence Network Interface Card (CNIC) that combines network functionality into a single chip.
Fabric A includes:
LAN On Motherboard (LOM)
Fabric A Features
The bidirectional data for a single Broadcom Gb LOM is carried over a PCI Express x4 bus.
SerDes (serial to parallel and parallel to serial) transceivers transmit data across the midplane on
a single lane to the I/O module bays A1 and A2
Support for 10/100/1000Gb Ethernet speeds
iSCSI boot
Wake‐On‐LAN (WOL)
PXE
TOE (TCP Offload Engine)
o Requires optional TOE license key (all systems ship with the TOE key populated)
Dell LOM TOE License Key
Optional LOM features are activated with
the population of the TOE License Key.
TOE License Key
118 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Expansion Cards Overview
The M‐series blade servers support optional Fabric expansion Mezzanine cards designed to support high
speed network communication technologies including:
InfiniBand
10Gb Ethernet
Fibre Channel
Mezzanine Cards
Mezzanine cards supported on the PowerEdge 1000e
Blade Servers include:
Emulex LPe1105‐M4 FC HBA
QLogic QME2472 FC HBA
Dell BCM57086S Ethernet Mezzanine
Mezzanine cards installed in the Fabric B or Fabric C card slot require a matching I/O module installed in
the corresponding I/O bay to support data flow for that fabric.
The AMD Chipset connection to Mezzanine card slot C supports a single PCIe x8 link, which cannot be
split into two PCIe x4 links.
Dell BCM57086S
The Dell BCM57086S Ethernet Mezzanine card requires two PCIe x4 links.
As a result, the M605, M805, and M905 blade servers cannot support the Dell BCM57086S Ethernet
Mezzanine card in Mezzanine card slot C (slot restriction).
The Dell BCM57086S Ethernet Mezzanine card is the only Mezzanine card restricted from slot C;
all other existing Mezzanine cards are supported.
The Dell BCM57086S Ethernet Mezzanine card is supported in Mezzanine card slot B.
This limitation only applies to the M605; the PowerEdge M600 is not affected.
Emulex LPe1105‐M4 FC HBA
The following list details the functionality of the Emulex LPe1105‐M4 FC HBA.
Provides 4, 2, or 1Gb FC connectivity among up to 16 Dell blade servers and a SAN fabric
Comprehensive virtualization capabilities with support for N_Port ID Virtualization (NPIV) and
Virtual Fabric
Administration via HBAnyware® for Emulex HBAs anywhere in the SAN
Common driver with all generations and types of Emulex HBAs
Firmware releases deployed independently from the driver
119 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Support for Industry Standard Fibre Channel Security Protocol (FC‐SP)
QLogic QME2472 FC HBA
The QME2472 HBAs is an "Enterprise Class HBA" specifically designed for the Dell 4 Gbps M1000e Blade
Server Chassis blades M‐series blades.
Provides 4, 2, or 1Gb FC connectivity among up to 16 Dell blade servers and a SAN fabric
Robust interoperability simplifies deployment and upgrades of SAN hardware and software
Administration via SANSurfer
Capability of booting from SAN
120 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Qlogic SANBlade QNE247M Fibre Channel HBA Technical Specifications
Host Bus Specifications Feature Description
Bus Interface X4 PCI Express
Memory 1 MB SRAM per port
1 MB FLASH (SPI)
2 KB NVRAM (SPI)
Compliance Conforms to PCI Express Base Specification rev. 1.0a
PCI Express Card Electromechanical Specification rev. 1.0a
PCI Bus Power Management Interface Specification revision 1.1
Fibre Channel Specifications Feature Description
Data Rate 4/2/1 Gbps auto‐negotiation (4.25/2.125/1.0625 Gbps)
Performance 150,000 IOPS per port
Throughput 800 Mbps per port
Topology Point‐to‐Point (N_Port)
Arbitrated Loop (NL_Port)
Switched Fabric (N_Port)
Logins Support for F_Port and FL_Port login. 2,048 concurrent logins and 2,048 active exchanges
Class of Service Class 2 and 3
Protocols FCP (SCSI‐FCP)
IP (FC‐IP)
FICON (FC‐SB‐2)
FC‐TAPE (FCP‐2)
Buffer Credits 3 credits available (2,112 Byte frame payload)
Compliance SCSI‐3 Fibre Channel Protocol (SCSI‐FCP)
Fibre Channel Physical and Signaling Interface (FC‐PH)
Fibre Channel 2nd Generation (FC‐PH‐2)
Third Generation Fibre Channel Physical and Signaling Interface (FC‐PH‐3)
Fibre Channel Arbitrated Loop (FC‐AL‐2)
Fibre Channel Fabric Loop Attachment Technical Report (FC‐FLA)
Fibre Channel‐Private Loop Direct Attach Technical Report (FC‐PLDA)
Fibre Channel Tape (PF‐TAPE) profile
SCSI Fibre Channel Protocol‐2 (FCP‐2)
Second Generation FC Generic Services (FC‐GS‐3)
Third Generation FC Generic Services (FC‐GS‐3)
Fibre Channel Framing and Signaling (FC‐FS)
121 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
IOMs
Information on the IOMs available for Dell
PowerEdge M1000e blade servers follows.
Dell/Emulex PT‐1016 Fibre Channel Pass‐Through
The Dell PT‐1016 by Emulex is designed for use in
situations where a direct connection between each
blade server and external SAN switches is required.
Dell/Emulex PT‐1016
The Dell PT‐1016 by Emulex provides investment
protection for data centers with pre‐existing networking infrastructures by providing a direct connection
between the servers and the SAN. The PTM provides up to 16 4Gb Fibre Channel connections with 1:1,
non‐switched connections between the server blade and the SAN. The PTM design is hot pluggable and
provides for high‐availability failover when deployed in pairs within the server chassis.
122 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Dell/Emulex PT‐1016 Fibre Channel Pass‐Through
Capabilities Provides 4, 2, or 1Gb connectivity between up to 16 Dell blade servers and a SAN fabric
Each port provides a dedicated connection for guaranteed bandwidth to each server port
With 16 external ports in a single blade, the PTM provides the highest port density available in
Dell’s connectivity options
Provides uncompromising scalability to the fullest capability of the Dell chassis
Hot‐plug capability and redundancy provide true high‐availability for 24x7 data center operation
Investment protection: uses existing SAN infrastructure with no interoperability issues
Key Features
16 ports of 4Gb Fibre Channel connectivity between servers and SAN
Transparent connection between SAN and Server
Simplicity of unmanaged design speeds deployment
Simplicity of unmanaged design eliminates configuration hassles
Eliminates interoperability issues with transparent, pass‐through protocol
Compact, ultra‐dense form factor
4, 2 or 1 Gb Fibre Channel connectivity on each port to match existing server and SAN
installation
Hot‐pluggable design eliminates downtime during service actions
Intended to be installed in pairs for redundancy and high availability
Firmware can be upgraded in the field, eliminating downtime (future, not in initial release)
Completely dedicated connection for each server blade
Up to 256Gb full‐duplex bandwidth
Technical Specifications
The Dell PT‐1016 by Emulex Pass‐Through Module has the following technical specifications:
Product Specification Summary Item Description
Physical interface Hot‐pluggable industry‐standard copper and optical SFP transceivers at all ports
Blade type Fibre Channel Pass‐Through
Performance 4Gb Fibre Channel per port, non‐blocking
Port Configuration Sixteen 4Gb Fibre Channel internal connections to Server Blades
Sixteen 4Gb Fibre Channel external uplinks
Port Type 4Gb Fibre Channel optical SFPs, shortwave
123 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Brocade 4424 Fibre Channel Switch
Switch Overview The Brocade 4424 is a 24‐port Fibre Channel module with 8 external ports and 16 internal ports
designed for installation in a blade server chassis. The product ships with 12 ports enabled and is
upgradable to 24 ports through Ports on Demand (POD) licensing. These ports support link speeds up to
4Gbit/sec.
Technical Specifications The following table describes the technical specifications for the Brocade 4424 Fibre‐Channel I/O
modular switch.
Brocade 4424 Features Item Description
Scalability Full fabric architecture with maximum of 239 switches
Certified maximum 32 switches, 7 hops
Performance 1.063 Gbit/sec line speed, full duplex 2.125 Gbit/sec line speed, full duplex
Fabric latency (Fabric Switch Mode only)
<2.1 sec with no contention
Maximum frame size 2112‐byte payload
Class of Service Class 2 Class 3 Class F (interswitch frames)
Port Types FL_port
F_port
E_port
N_port
Fabric services (Fabric Switch Mode only)
Simple Name Server, Registered State Change Notification
PowerPC 440GP processor 333MHz
SDRAM memory 256 MB – for system memory at 64 bits wide with 8 ECC bits
124 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Brocade 4424 Features Item Description
Boot flash memory 4 MB
Compact flash memory 256 MB
InfiniBand Technology
The InfiniBand standard is a key element in the speed and efficiency of Dell
blade server I/O systems.
About InfiniBand
In 1999, two competing input/output (I/O) standards called Future I/O
(developed by Compaq, IBM and Hewlett‐Packard) and Next Generation I/O (developed by Intel,
Microsoft and Sun) merged into a unified I/O standard called InfiniBand. InfiniBand is an industry‐
standard specification that defines an input/output architecture used to interconnect servers,
communications infrastructure equipment, storage and embedded systems. InfiniBand is a true fabric
architecture that leverages switched, point‐to‐point channels with data transfers up to 120 gigabits per
second, both in chassis backplane applications as well as through external copper and optical fibre
connections.
How InfiniBand Works
InfiniBand is a pervasive, low‐latency, high‐bandwidth interconnect that requires low processing
overhead and is ideal to carry multiple traffic types (clustering, communications, storage, management)
over a single connection. As a mature and field‐proven technology, InfiniBand is used in thousands of
data centers, high‐performance compute clusters and embedded applications that scale from two nodes
up to a single cluster that interconnect thousands of nodes.
125 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Advantages of InfiniBand
Powerful multi‐core CPUs, server virtualization, blade architectures, increasing storage demands, and
the need to reduce power usage are all placing increased demands on I/O solutions. InfiniBand can
address these demands while delivering low‐latency in a high‐bandwidth interconnect. Advantages of
InfiniBand include:
Superior performance ‐ InfiniBand is the only shipping solution that supports 20Gb host
connectivity and 60Gb switch to switch links. InfiniBand also provides world‐class application
latency with measured delays of 1µs end‐to‐end.
High efficiency ‐ InfiniBand offloads all data transport mechanisms to the adapter cards to
relieve CPUs so they can be fully used on customer workloads. In addition, InfiniBand uses
Remote Direct Memory Access (RDMA) to optimize data transfer and reduce internal memory
usage to further enable CPUs to focus on application processing.
Fabric consolidation and low energy usage ‐ With less than 5W per 20Gb port InfiniBand can
consolidate networking, clustering, and storage data over a single fabric, significantly lowering
the overall power, real estate, and management overhead required for servers and storage.
Reliable and stable connections ‐ By enabling fully redundant I/O fabrics, with automatic path
failover and link layer multi‐pathing abilities to meet the highest levels of availability, InfiniBand
is perfectly suited to meet the mission critical needs of today’s enterprise data center.
126 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Data integrity ‐ Mission‐critical applications require that data has not been corrupted as it
travels across the fabric. InfiniBand enables the highest levels of data integrity by performing
cyclic redundancy checks (CRCs) at each fabric hop and end‐to‐end across the fabric to ensure
the data is correctly transferred.
These advantages combine to make InfiniBand the industry’s most powerful interconnect.
Cabling
InfiniBand uses a base 2.5‐Gbps clock rate that necessitates that InfiniBand ports and cables are thicker
and bulkier than Unshielded Twisted Pair or UTP, commonly found in Ethernet applications. Generally,
the faster a link is clocked, the more crosstalk is generated. This is because of the rapid rise and fall of
the carrier voltage as bits are transmitted. Additionally, because of the signal attenuation, the receiver
must be able to interpret the signal correctly when it is received. At higher InfiniBand speeds, the effects
of crosstalk and data skew limit cable lengths and consequently limit the number of nodes that can be
supported within a cluster. See the table below.
For InfiniBand 4X installations, although the copper cabling lengths are relatively limited when
compared to Gigabit Ethernet, very large clusters can be constructed by carefully planning cabling runs
and equipment placement. As an example, the world’s largest InfiniBand‐based HPC cluster, built using
Cisco SFS 7000 Series InfiniBand Server Switches, consists of more than 4,500 dual‐processor servers. By
contrast, using copper cabling only, the largest InfiniBand 12X HPC cluster that can be built is 256 nodes
using standard “side‐by‐side” rack configuration. By arranging the racks in a “U” shape, it is possible to
build a 512‐node cluster, although this complicates the cabling and maintenance aspects of the cluster.
Link Speeds
These variables determine at what speed the InfiniBand fabric will run and its capable bandwidth:
Link Width: The number of bonded channels (send/receive pairs) used for an InfiniBand link.
Defined IB link widths are 1X, 4X and 12x.
Signaling Rate: The simplex, raw, bit‐rate of an InfiniBand channel. Defined IB signaling rates are:
o SDR (Single Data Rate ‐ 2.5 Gbps/channel)
o DDR (Double Data Rate ‐ 5 Gbps/channel)
o QDR (Quad Data Rate ‐ 10 Gbps/channel)
Link Speed: The total simplex capacity of an InfiniBand link calculated as the signaling rate of
each channel multiplied by the number of channels present in the link. For example, a 4X DDR IB
Link has a simplex link capacity of 5.0 Gbps X four channels = 20 Gbps in each direction.
QDR signaling rate standard has been defined, but silicon that is able to drive a signal at this rate is not
yet in production by any vendor.
127 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Defined IB Link Speeds
SDR
1X SDR 2.5 Gbps
4X SDR 10 Gbps
12X SDR 30 Gbps
DDR
1X DDR 5 Gbps
4X DDR 20 Gbps
12X DDR 60 Gbps
QDR
1X QDR 10 Gbps
4X QDR 40 Gbps
12X QDR 120 Gbps
InfiniBand Physical‐Layer Characteristics
The InfiniBand physical‐layer specification supports three length widths, designated 1X, 4X, and 12X,
over both copper and fiber optic media, as stated earlier in this module. The base data rate, 1X single
data rate (SDR), is clocked at 2.5 Gbps and is transmitted over two pairs of wires—transmit and
receive—and yields an effective data rate of 2 Gbps full duplex (2 Gbps transmit, 2 Gbps receive). The 25
percent difference between data rate and clock rate is due to 8B/10B line encoding that dictates that for
every 8 bits of data transmitted, an additional 2 bits of transmission overhead is incurred.
InfiniBand 4X and 12X SDR interfaces use the same base clock rate that uses multiple pairs, referred to
as lanes, to increase bandwidth capacity. Therefore, an InfiniBand 4X interface realizes a data rate of 8
Gbps using 8 pairs of wires and a 12X interface realizes 24 Gbps using 24 pairs. Although it is possible for
InfiniBand switches to forward frames between an InfiniBand 1X and 4X, or 4X and 12X link, there is a
performance penalty because of the way that InfiniBand transmits packets.
InfiniBand switches use cut‐through switching to provide low‐latency transport between devices, which
does not require the entire packet to be buffered before being forwarded. This enables InfiniBand to
provide low‐latency network transmission, but at the same time requires that the ingress and egress link
speeds are the same and that the packet is received and transmitted in the same format.
However, when different link speeds are mixed in InfiniBand networks, InfiniBand transmits portions of
the packet in parallel. Therefore, for an InfiniBand 1X link, a packet is transmitted serially; for an
InfiniBand 4X link, the packet is divided into four chunks and transmitted in parallel across the four 2.5‐
Gbps lanes. For a 12X link, a packet is divided into 12 chunks and transmitted in parallel across twelve
2.5‐Gbps lanes.
Parallel Transmission in InfiniBand Links
Because the packet is transmitted in parallel, if a packet is switched between a 12X and a 4X interface, a
store‐and‐forward penalty is incurred to change transmission formats. To maintain the performance of
128 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
cut‐through switching, most InfiniBand fabrics consist of a single link speed. For example, all links are 4X
SDR with multi‐path forwarding used to provide additional bandwidth within the InfiniBand network.
However, if an application can tolerate the small store‐and‐forward delay, mixing 12X and 4X in the
same InfiniBand network may be an acceptable solution.
Mellanox ConnectX IB MDI InfiniBand Host Channel
Adapter (HCA) Mezzanine Card
The Mellanox ConnectX IB MDI InfiniBand Host Channel
Adapter (HCA) mezzanine card for the Dell PowerEdge M‐
series blade servers delivers low‐latency and high‐bandwidth
for performance‐driven server and storage clustering
applications in Enterprise Data Center and High‐Performance
Computing environments. Clustered databases, parallelized
applications and transactional services applications will
achieve significant performance improvements resulting in
reduced completion time and lower cost per operation.
ConnectX IB adapters simplify network deployment by
consolidating clustering, communications, storage, and
management I/O and by providing enhanced performance in
virtualized server environments.
129 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Benefits These are the main benefits of the Mellanox ConnectX Dual‐Port InfiniBand Mezzanine Card:
World‐class cluster performance
High‐performance networking and storage access
Guaranteed bandwidth and low‐latency services
Reliable transport
End‐to‐end storage integrity
I/O consolidation
Virtualization acceleration
Scales to tens‐of‐thousands of nodes
Key Features Key features of the Mellanox ConnectX Dual‐Port InfiniBand Mezzanine Card include:
1.2us MPI ping latency
20Gb InfiniBand ports
CPU offload of transport operations
End‐to‐end QoS and congestion control
Hardware‐based I/O virtualization
TCP/UDP/IP stateless offload
Additional Features Additional features of the Mellanox ConnectX Dual‐Port InfiniBand Mezzanine Card include:
World Class Performance and Scalability: Clustered applications running on multi‐socket servers using
multi‐core processors benefit from the reliable transport connections and advanced multicast support
offered by ConnectX lB. End‐to‐end Quality of Service (QoS) enables partitioning and guaranteed service
levels while hardware‐based congestion control prevents hot spots from degrading the effective
throughput. ConnectX lB is capable of scaling to tens‐of thousands of server and storage nodes.
Hardware Offload Architecture: Clustered and client/server applications achieve maximum
performance over ConnectX IB, because CPU cycles are available to focus on critical application
processing instead of networking functions. Network protocol processing and data movement overhead
such as RDMA and Send/Receive semantics are completed in the adapter without CPU intervention.
Applications using TCP/UDP/IP transport can achieve industry‐leading throughput when run over
ConnectX lB and its hardware‐based stateless offload engines.
I/O virtualization: ConnectX lB support for hardware‐based I/O virtualization is complementary to Intel
and AMD virtualization technologies. Virtual machines (VM) within the server are enabled with
dedicated I/O adapter resources and guaranteed isolation and protection. Hypervisor offload features
remove software‐based virtualization overheads and free up CPU cycles enabling native operating
system performance for VMs and higher server utilization by supporting more VMs per physical server.
130 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Storage Accelerated: A unified InfiniBand cluster for computing and storage achieves significant cost‐
performance advantages over multi‐fabric networks. Standard block and file access protocols leveraging
InfiniBand RDMA result in high‐performance storage access. Data reliability is improved through the use
of Tb0‐compliant Data Integrity Field (DIF). Fibre Channel (FC) over InfiniBand (FCoIB) features enable
the use of cost‐effective bridges for connecting to FC SANs.
Software Support: All Mellanox adapter cards are compatible with legacy TCP/IP and OpenFabrics‐based
RDMA protocols and software. They are also compatible with InfiniBand and cluster management
software available from OEMs. The adapter cards are supported with major operating system
distributions.
Technical Specifications The following table shows the technical specifications for the Mellanox ConnectX Dual‐Port InfiniBand
Mezzanine Card.
PowerEdge M1000e (InfiniBand Interconnect) Specifications Feature Description
InfiniBand IBTA Specification 1.2 compliant
10 or 20Gb per port
RDMA, Send/Receive semantics
Hardware‐based congestion control
Atomic operations
16 million I/O channels
256 to 4Kbyte MTU
2GB messages
9 virtual lanes: 8 data + 1 management
Enhanced InfiniBand Hardware‐based reliable transport
Hardware‐based reliable multicast
Scalable Reliable Connected transport
Enhanced Atomic operations
Service oriented I/O
Fine‐grained end‐to‐end QoS
Hardware‐Based I/O Virtualization Address translation and protection
Multiple queues per virtual machine
Native OS performance
Complimentary to Intel and AMD I/OMMU
Additional CPU Offloads TCP/UDP/IP stateless offload
Intelligent interrupt coalescence
Full support for Intel I/OAT
Compliant to Microsoft RSS and NetDMA
Storage Support T10‐compliant Data Integrity Field support
Fibre Channel over InfiniBand (FCoIB)
CPU Compatibility AMD X86, X86_64
Intel X86, EM64T, IA‐32, IA‐64
131 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
PowerEdge M1000e (InfiniBand Interconnect) Specifications Feature Description
PCI Express Interface PCIe Base 2.0 compliant, 1.1 compatible
2.5GT/s link rate x8 (20+20Gb bidrectional bandwidth)
Fits Dell PowerEdge M1000e server blades
Support for MSI/MSI‐X mechanisms
Connectivity Works with Dell PowerEdge M600 and PowerEdge M605 blades
Management and Tools OpenSM
Interoperable with third‐party subnet managers
Firmware and debug tools (MFT, IBADM)
Operating Systems/Distributions Novell SLES, Red Hat Enterprise Linux (RHEL), Fedora, and other Linux distributions
OpenFabrics Enterprise Distribution (OFED)
Protocol Support Open MPI, OSU MVAPICH, HP MPI, Intel MPI, MSMPI, Scali MPI
IPoIB, SDP, RDS
SRP, iSER, FCoIB and NFS RDMA
uDAPL
InfiniBand IOM: Cisco SFS M7000E IB
Switch
Cisco SFS7000E Series InfiniBand Switches
provide an ideal server interconnect for
distributed application environments,
such as high‐performance computing. The
Cisco SFS7000E Series meets the
performance demands of distributed
applications by offering:
The unprecedented bandwidth of InfiniBand
Superior latency characteristics
Enterprise‐class high availability and management features
The latest evolution of the Cisco SFS 7000 Series supports dual‐speed InfiniBand 4X double data rate
(DDR) and single data rate (SDR) interfaces that deliver 20 Gbps and 10 Gbps bandwidth respectively,
per port. The low latency and high bandwidth offered by Cisco SFS 7000D InfiniBand Server Switches
enable a new class of distributed applications and systems that deliver greater business agility and
competitive advantage.
Overview The Cisco SFS M7000E Switch delivers an ideal combination of price and performance with the Dell
M1000e Blade platform. The switch provides DDR InfiniBand connectivity to up to 16 blade server
132 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
modules and provides 8 DDR InfiniBand uplink ports. Each DDR InfiniBand port can support 20 Gbps of
fully non‐blocking bandwidth to each port. The port‐to‐port latency through the switch is 140 ns.
When fully populated, the Dell M1000e can have four Cisco SFS M7000E blade modules to drive 1.28
terabits of nonblocking traffic for I/O‐intensive applications. The blade switch is also a hot‐pluggable
expansion module, allowing on‐the‐fly capacity expansion without application disruption. The Cisco SFS
M7000E is designed to be a compelling high performance computing (HPC) solution in the blade market.
The Cisco SFS M7000E has been certified with the Cisco SFS 7000 Series InfiniBand Server Switches and
Cisco SFS 3000/3500 Multi‐fabric Server Switches. Cisco provides a complete DDR InfiniBand switching
family to complement the Cisco SFS M7000E switch to deliver HPC solutions. For example, by using the
Cisco SFS‐7024D Switches in the core (288‐port, 4x DDR InfiniBand switches) to aggregate multiple blade
chassis, a customer can build clusters with tens of thousands of processors using Dell M1000 blade
systems.
Because the Cisco SFS M7000E is an unmanaged switch, a subnet manager is required for the operation
of the InfiniBand fabric. Cisco recommends running the Cisco High Performance Subnet Manager on a
blade server module in the InfiniBand fabric. The Cisco SFS M7000E is also qualified with other SFS
management applications such as the Fabric Analysis and Correlation Toolkit.
133 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Technical Specifications
The following tables show the technical specifications for the Cisco SFS M7000E InfiniBand Dual‐Port
Switch.
Cisco SFS M7000E InfiniBand DualPort Switch Features Feature Description
20‐Gbps (DDR), 4x InfiniBand Ports The Cisco SFS M7000E offers 24 4x DDR IB ports of 20‐6bps links for the most demanding high‐performance computing (HPC) applications. The switch has full bisectional bandwidth of 960 Gbps. Each port is autosensing in both single‐data rate (SDR) and double‐data rate (DDR) modes. 16‐ports provide connectivity to server modules and 8 uplink ports are available for external connectivity.
Low Latency The Cisco SFS M7000E is a cut‐through switch with less than 140 nanoseconds of port‐to‐port latency. The switch is ideal for applications that have large inter‐process communication (IPC) or simply depend on quick movement of data for competitive advantage in the market.
Powered Ports All ports are powered to enable connectivity with passive and active copper cables as well as optical cables.
Hot‐Plug Swappable The Cisco SFS M7000E Switches are hot‐plug swappable into any of the four I/O slots in the rear of the Dell M1000E blade chassis. The administrator can reconfigure and maintain the fabric without powering down the server nodes.
Scale‐out Fabric Topologies Each server blade may contain up to 2 HCA cards. When the blade chassis is fully populated with the Cisco SFS M7000E Switches, a customer can build multiple, independent fabrics with the same server nodes. This type of configuration offers extremely high throughput.
Unmanaged Switch The Cisco SFs M7000E is an unmanaged switch to provide a power‐efficient, cost‐effective solution for HPC customers.
Complete Ecosystem Cisco offers an InfiniBand platform that has been certified and extensively tested to provide optimal productivity. The Cisco SFS M7000E is part of a greater InfiniBand offering from Cisco.
Cisco SFS M7000E InfiniBand DualPort Switch Technical Specifications Feature Description
Size (H x W x D) 1.15 x 12.10 x 10.75 in.
System Weight 3.75 lb.
Temperature Operating: 40 to 120°F (5 to 50°C)
Nonoperating: ‐40 to 140°F (‐40 to 60°C)
Power Consumption 56W
Environmental Standards ROHS‐5 compliant, WEEE compliant
Mean time between Failure (MTBF)
>100,000 hours
134 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Diagnostic Indicators
Diagnostic LEDs The following table shows the diagnostic indicators to troubleshoot the Cisco SFS M7000E InfiniBand
switch.
LED Type Indicator Description
Port Status LEDs Diagnostic Status (DIAG) Blinking Green: Traffic Active
Solid Green: Logical link exists
Off: Link error or Subnet Manager is not running
System Status LEDs Diagnostic Status (DIAG) Solid Blue: System is operating normally
Blinking Blue: POST is in progress
Amber: System is receiving power but is not functioning properly
Off: System power is off
Power Status LEDs Diagnostic Status (DIAG) Solid Green: Power is applied to the switch
Off: Power is not applied to the switch
Power‐On Indicators
The following LED sequence occurs after installation into the chassis I/O bay and the switch is functional.
1. Switch Status Power LED turns GREEN after a few seconds. CMC module in Blade system verifies
that the Switch is correctly connected to an InfiniBand HCA.
2. Port Status LEDs turn GREEN after approximately 30 seconds.
135 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
3. Switch Status LED turns BLUE.
After installation into the chassis I/O bay, if the switch is not operational, the following are true:
1. The Switch Status Power LED does not turn GREEN.
2. The Switch Status LED blinks AMBER.
Summary
Fabric A consists of two embedded Broadcom Ethernet controllers, which are designed as a
(CNIC) that combines network functionality into a single chip.
o The bidirectional data for a single Broadcom Gb LOM is carried over a PCI Express x4 bus
connected to the ESB2‐D Southbridge on the M600 or the MCP55 on the M605.
o Optional LOM features are activated with the population of the TOE License Key.
Optional Fabric expansion Mezzanine cards designed to support high‐speed network
communication technologies in Fabrics B and C include:
o 10Gb Ethernet
o Fibre Channel
o InfiniBand
The data for each mezzanine fabric is carried over a PCI Express x8 interface connected to the
Northbridge.
Mezzanine cards supported include:
o Emulex LPe1105‐M4 FC HBA
o QLogic QME2472 FC HBA
o Dell BCM57086S Ethernet Mezzanine
Mezzanine cards (which require a matching I/O module installed in the corresponding I/O bay)
installed in the Fabric B or Fabric C card slot require a matching I/O module installed in the
corresponding I/O bay to support data flow for that fabric.
o The PowerEdge M605 cannot support the Dell BCM57086S Ethernet Mezzanine card.
IOMs in Dell PowerEdge blade servers include:
o Dell/Emulex PT‐1016 Fibre Channel Pass‐Through, designed for use in situations where a
direct connection between each blade server and external SAN switches is required.
o Brocade 4424 24‐port Fibre Channel switch
InfiniBand is an industry‐standard specification that defines an input/output architecture used
to interconnect servers, communications infrastructure equipment, storage and embedded
systems.
o InfiniBand addresses the needs of today’s complex and powerful servers by providing:
Superior performance
High efficiency
Fabric consolidation and low energy usage
Reliable and stable
Data integrity
136 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
o The Mellanox ConnectX IB MDI InfiniBand Host Channel Adapter (HCA) mezzanine card
delivers low‐latency and high‐bandwidth for performance‐driven server and storage
clustering applications.
o Cisco SFS7000E Series InfiniBand Switches offer:
The unprecedented bandwidth of InfiniBand
Superior latency characteristics
Enterprise‐class high availability and management features
137 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Module 8: Troubleshooting
Common Configuration Issues for Blades
Objectives
In this section you will learn about common configuration issues that can avoid the need for parts
replacement.
Receive Message stating that the CMC’s are not redundant
Solutions
Plug both CMC’s into management network via Ethernet cable. Primary CMC will have a blue status indicator led.
CMC firmware does not match. Make sure both CMC’s are running the same firmware version.
Update the Firmware
Firmware Updates are critical for M1000e and blades. Update all components to current versions at
the same time. If this is not done, unexpected errors or behavior may occur from the CMC, iDRAC,
and/or blade. Plan for maintenance windows to perform updates.
138 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
CMC
iKVM
iDRAC
Bios
M6220 firmware
LOM BIOS/firmware
HBA Mezzanine BIOS/firmware
FLEXADDRESS INFORMATION: See http://www.delltechcenter.com/page/FlexAddress for information
on FlexAddress.
An example of why it is important to update all firmware/bios to current versions across all components
in the enclosure and blades as mentioned in item 2. Even if FlexAddressing is not enabled, the CMC
requires all pertinent component firmware be at a version that supports FlexAddressing. See items 3
and 4.
Message received in the iDRAC sel log
When the message “Link tuning sensor failed” is received:
Do not replace hardware. Update the firmware for the appropriate device listed in the error message. This could be the iDRAC, LOM, or HBA Mezzanine card.
Example of error: LinkT/FlexAddr: Link tuning sensor failed to get link tuning or flex address data from BMC/iDRAC
FC (QLGC or EMLX) Mezzanine would experience "F1/F2" during post.
Do not replace hardware. Update the QLGC or EMLX mezzanine BIOS.
QLogic QME2472 – BIOS 2.04 or later
Emulex LPe1105‐M4 – BIOS 3.03a3 and firmware 2.72A2 or later
Server non responsive at power up
A new installation or configuration change, the most common problem is fabric mismatch. The blade status/identification indicator will blink amber if there is an installed mezzanine card that does not match the I/O module installed in the M1000e enclosure.
139 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Explanation of fabrics: See CMC get dc info and get io info output below. Notice that the font color matches the fabric types.
Fabric A will always be Ethernet – Switch 1 and switch 2 correspond to the LOM’s in each of the blades. Note that the LOM’s are not listed under the server Daughtercard (DC1 type, DC 2 type) information below. (the term daughtercard is used instead of mezzanine card)
Fabric B can be either Ethernet or fibre – switch 3 and switch 4 correspond to daughtercard (DC1 type). Note that both switch 3 and switch 4 fabric must match. Example of unsupported configuration: switch 3 = Ethernet and switch 4 = fibre. Also note that the daughtercard fabric must match the switch fabric. In the example below, fabric B is Ethernet.
Fabric C can be either Ethernet or fibre – switch 5 and switch 6 correspond to the daughtercard (DC2 type) – Note that both switch 5 and switch 6 fabric must match. The daughtercard (DC2 type) must match the switch fabric. In the example below, fabric C is Fibre.
It is possible to install one switch for a fabric and leave the second switch blank. This is a valid configuration.
‐ If a switch is not installed, no mezzanine cards can be installed for that fabric. ‐ If a switch is installed, not all servers must have the corresponding mezzanine card installed.
Troubleshooting Resource
Dell Enterprise Technology Center ‐ http://www.delltechcenter.com
1. Use menu on left side of page to choose Blades:
140 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
2. At the bottom of the page choose from several information resources:
Support.dell.com
Firmware can manually be downloaded from this site. Information is located under servers.
1. Choose Drivers and Downloads from the main page:
141 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
2. Choose method for finding information as appropriate.
Ftp.dell.com
FTP site where updates are available. This site is automatically accessed by the servers.
142 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
Module 9: Navigating Dell Information and Tools Dell Enterprise is committed to offering customers information and support through many channels. Our
engineers and subject matter experts are dedicated to bringing you experience and guidance, technical
“how to” tips, tools, best practices and solutions.
Upon completion of this module, you will be able to:
Navigate through many of the support sites and tools designed and offered by Dell Enterprise.
Recognize the benefits of these tools.
Relate the relevancy of this information to your job.
Dell System E‐Support Tool (DSET)
The Dell System E‐Support Tool (DSET) provides the ability to collect hardware, storage and operating
system information from a Dell PowerEdge server. This information is consolidated into a single System
Configuration Report that can be used for troubleshooting or inventory collection of a system. The
browser user interface provides a convenient means to view specific data through hierarchical menu
trees.
Navigate and explore the Dell System E‐Support Tool (DSET) at this link:
http://support.dell.com/support/topics/global.aspx/support/en/dell_system_tool?c=us&l=en&s=gen
Support.dell.com
This comprehensive support site offers a wide‐variety of technical support options. You may research
your issues using Dell manuals and forums as well as check on order status and download drivers you
need. The site offers comprehensive information on all Dell products and features links to many tech
support, customer care, and chat options.
Navigate to the Dell’s Support page at this link: http://support.dell.com/
1. Find the Troubleshooting and FAQs section and click the link. Follow these steps to use the troubleshooting wizard:
a. Choose Select a Model.
b. With Select your Product Model highlighted, select Servers, Storage, Networking.
c. With Servers, Storage, Networking highlighted, select PowerEdge Server.
d. With PowerEdge Server highlighted, select the model of your choice, then confirm your choice.
143 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
e. Spend 10 minutes using the wizard.
2. Support.dell.com offers all customers options for signing up and managing Dell Technical Update information. The options include Dell Technical Updates and Management of Your Updates. The site requires that you sign up initially for an account after which updates concerning your equipment will automatically be e‐mailed to you.
a. Click this link, then review the web page: http://support.dell.com/support/topics/global.aspx/support/ notifications /en/index?c=us&l=en&s=gen&~ck=ln&lnki=0
b. Sign up for this service now or save the link for future reference.
Learndell.com
Dell offers comprehensive training and certification programs for Dell hardware and popular software
applications. Training can be customized to meet your unique requirements.
Navigate to the LearnDell site at this link:
http://www.learndell.com/Dell/
Spend five minutes reviewing the learning opportunities offered through Dell Learning.
Enterprise Support Services
Each level of Dell’s Enterprise Support incorporates different options, allowing Dell and you to create the
right match for your unique needs. Dell also offers help to ensure on‐going performance of your EMC
technology.
Navigate and explore the Enterprise Support site at this link:
http://www.dell.com/content/topics/global.aspx/services/ent_support/ent_support?c=us&cs=555&l=e
n&s=biz
Spend five minutes reviewing the different support options offered by Dell Enterprise.
Thank You!
This concludes the Dell Blade Server Configuration Course. Dell invites you to consider the following
related courses:
Dell SAN Data Protection.
2 day course
Dell SAN Management.
3 day course
Dell OpenManage and System Management.
2 day course
144 | P a g e B l a d e S e r v e r C o n f i g u r a t i o n & T r o u b l e s h o o t i n g
PowerEdge Configuration and Troubleshooting.
2 day course
Dell Desktop and Laptop Configuration and Troubleshooting.
2 days
Go to www.learndell.com to register!