Data Center Management Illinois Institute of Technology ... · • Many Data Centers and Server Rooms have raised floors. • Raised floors can be –Tripping hazards if the tiles

Data Center ManagementIllinois Institute of Technology

ITM 576

Lecture 10William F. Slater, III, M.S., MBA, PMP, CDCP

Spring Semester 2014

March 27, 2014 1IIT ITM 576 – Data Center Management - Lecture 10 -

Copyright 2014 - William Favre Slater, III

DATA CENTER DANGERS AND SAFETY

March 27, 2014 IIT ITM 576 – Data Center Management - Lecture 10 2

Data Center Dangers

William F. Slater, III (Getronics)

Data Center Manager / Change Management Coordinator

May 2005

March 27, 2014 3IIT ITM 576 – Data Center Management - Lecture 10


Data Center Safety Agenda

Introduction

The Dangers

Safety Signs

Policies

Additional Developments

Conclusion


Introduction

• Data Centers and Server Rooms often appear to be sterile, computer-filled environments that look boring and safe

• They represent many opportunities for accidents, if they are not approached with Safety in mind.

• Policies and Safety education are necessary to enforce safety.


The Dangers

• Raised Floors

• Lifting Hazards

• Electrical Shock Hazards

• Static Discharge Hazards

• Cutting Hazards

• Pinching / Amputation Hazards

• Fire Hazards

• Accidental Triggering an FM 200 Fire

Retardant System Dump

• Accidental Unplugging Network Cables

or Power From Servers

• Accidental power shutdown via the

Emergency Shutdown Button


Raised Floors

• Many Data Centers and Server Rooms have raised floors.

• Raised floors can be

– Tripping hazards if the tiles are not level

– Falling hazards if a tile is open

– Asbestos hazards if you try to cut them

– Sub-Floor Electrical shock hazards (see slide on Electric Shock)

– Pinching / Amputation Hazards (see slide on Pinching / Amputation)


Lifting Hazards

• Back injury may result from lifting heavy equipment.

• Never lift more than 40 pounds

• Size up the equipment. If it weighs 120 pounds, divide that by 40, and get two additional people.

• Ensure that equipment is not dropped – or else it could be damaged.


Electrical Shock Hazards

• It is possible to get shocked if you are plugging in new equipment or unplugging old equipment, or even moving existing equipment.

• It is also possible to get shocked if you are plugging in a cable into a large amp circuit under the raised floor. This could result in death.


Static Discharge Hazards

• When handling sensitive electrical components, a

static discharge from your body may damage or

render the component useless.

• Always ground yourself to ensure that you don’t have

any static electricity built up on your person.


Cutting Hazards

• Parts of certain equipment have sharp edges

• Prevention: Use work gloves


Pinching / Amputation Hazards

• Could get fingers caught

– in Tape Vault Door

– In Server Rack Door


Fire Hazards

• In the rare instance that electrical equipment has a severe malfunction, a fire could result.

• There is a fire retardant system in most Data Centers and Server Rooms

• Fire Extinguishers can avoid having to activate a fire retardant system dump.

• Use a Class A extinguisher for normal combustible fires, like paper.

• Use a Class B extinguisher for flammable liquid fires

• Use a Class C extinguisher for electrical fires

• Use a Class D for exotic metal fires, like magnesium


Accidental Triggering

an FM 200 Fire Retardant System Dump

• It is possible for a Fire Retardant System to

get a false positive, thus triggering a

countdown sequence. This can and should

be overridden.

• It is also possible to set off a manual Fire

Retardant System dump, so be careful and

avoid doing this.

Actual dump

Of an FM 200

System

http://www.greenfiresystems.com/FM200_discharge_sm.jpg



Pop Quiz 1

• FM 200 gas is dangerous and could be deadly

if you inhale it.

– True

– False


Pop Quiz 1

• FM 200 gas is dangerous and could be deadly

if you inhale it.

– True

– False




Pop Quiz 2

• You are in the Data Center. Where do you find

the real detailed information on FM 200 in the

shortest time possible?

– PDR

– MSDS posted in the Data Center

– The Web

– American Chemical Society


Pop Quiz 2

• You are in the Data Center. Where do you find

the real detailed information on FM 200 in the

shortest time possible?

– PDR

– MSDS posted in the Data Center

– The Web

– American Chemical Society

(MSDS = Material Safety Data Sheet)

March 27, 2014 19

Fire Protection – An Ongoing Debate• Two Fundamental Types

– Water, Of Which There Are Two Types

• Conventional Wet Pipe

o Potential For Pipe Leaks

o Broken Heads Start Water Flow

o Water Flows Until Shut Off

• “Dry Pipe” Pre-Action

o Pipes Filled With Compressed Air

o Air Pressure Change Detects Leaks

o Smoke Detection Charges Pipes With Water

o “Override” Delay Switches Usually Used

o Heads Still Activate With Heat

o Water Still Flows Until Shut Off

o Requires Pre-Action Valve Room Nearby

IIT ITM 576 – Data Center Management - Lecture 10

Robert E. McFarlane, Facilities Management Course

Marist College, Poughkeepsie, NY, 2006

March 27, 2014 20

Fire Protection – An Ongoing Debate• Two Fundamental Types (Cont.)

– “Inert” or “Clean Agent” Gas

• Facilities Requirements

o Sealed Room

• Required To Maintain Gas Concentration

• All Ducts Must Have Automatic Dampers

• Ceiling Tiles Must Be “Clipped” In Place

o Automatic Power Shutdown

• All Power Cut Before Discharge

• Includes UPS and Batteries

• All Air Conditioners Stopped Before Discharge

• Annunciator Panel, Overrides & Protection

o Visual Annunciator Panels Readily Visible

• Requires Two Sensors To Activate

o Audible Warning Alarms

o Override Buttons At Every Door

o Breathing Safety Systems Outside Each Door




March 27, 2014 21

Fire Protection – An Ongoing Debate• Types and Trade Names

– “Inert” or “Clean Agent” Gas Types

• “Halon” (DuPont)

o Original Gas Suppression System

o Hydrocarbon Agent, Not Legal Since Jan. 1, 1994

• Ozone Depletion Potential

• No Manufacture or New Installations Allowed

• Existing Installations Can Remain

• Modifications Require Removal

• Re-Charge Must Be From “Reclaim” Market

• Must Be Disposed Of Properly

• “HFC-227” (Generic Ident. of Common Halon Replacement)

o Heptafluoropropane, A Halocarbon, Not A Hydrocarbon

o No Chlorine or Bromine, No Ozone Depletion

o Requires 7% Concentration To Extinguish Fire

o Operates By Rapidly Reducing Fire Temperature

o No Residue Or Oily Deposits - Removed By Ventilation

o Higher Pressure & Volume Than Halon – Different Piping




March 27, 2014 22

Fire Protection – An Ongoing Debate

• HFC-227 Trade Names

– “FM-200” (Great Lakes Chemical, Now “Chemtura”

• Often Considered The “Original Halon Replacement”

– “FE-227 (DuPont)

• Main DuPont Replacement For Halon 1301

– “FE-25” (DuPont)

• Easiest Retrofit For Existing Halon 1301 Systems)

• Potential Toxicity Concerns




March 27, 2014 23


• Non-Halocarbon Types

– Characteristics

• Nitrogen, Argon & CO2 Mixture

o No Ozone Depletion Concern

• Reduces Oxygen Level To Below 15%

• High Pressure Tanks

o Should Be Stored In “Hardened” Room

• Requires Much More Gas Than Halocarbons

o Requires Several Times The Tank Space

• Often Marketed As “Direct Halon Replacement”

• Used More In Europe Than In US

– Trade Names

• “Intergen” (Ansul)

• “Argonite” (Ginge-Kerr)




March 27, 2014 24


• Carbon Dioxide (“CO2”)

– Early and Common Fire Extinguishing Agent

• Very Effective On Fires

• Dangerous To People

• Residue Is Hard To Remove




March 27, 2014 25

Fire Protection – An Ongoing Debate• Which Is Better???

– Most Common Assertions

• Water Protects Buildings and People

• Gas Protects Equipment

– Realities

• Water

o Requires Significant Fire To Activate Heads

o Water Only Ruins Equipment:

• If Power Is Still On

• If Fans Pull It Through

• If Its Not Filtered

o Water Can’t Get To Equipment Fire

• If Equipment Is Stacked In Cabinet

• If Anything Is In The Way (Cable,Cable Trays, Etc.)




March 27, 2014 26

Fire Protection – An Ongoing Debate• Which Is Better???

– Realities (Cont.)

• Inert Gas

o Will Invade Equipment Cabinet, Extinguishing Fire

o Will Act Much Earlier Than Sprinklers

o Will Not Extinguish Re-Ignition Unless:

• Backup Tanks Are Provided

• Backup Sprinkler System Is Provided

o Sprinklers May Be Required, Regardless, By:

• Local Jurisdiction

• Insurance Carrier

o All Air Conditioners Stopped Before Discharge

• Automatic “EPO” Switch

• Essentially An NEC Article 645 Room

– There Is No Absolute Answer

o Cost Is Often The Driving Factor

o Everyone Has A Preference Or Opinion




March 27, 2014 27

Other Hazards• Physical Hazards

– Tripping and Slipping Hazards

• Uneven Floor Tiles

• Cables On Floor

• Wet Floors

• Wire Cuttings

• Storage and Trash In Aisles

– Environmental Hazards

• Water and Other Liquids

o Leaking Pipes

• Provide Drip Pans Below

o Flooding On Floor Above

• Do Not Locate Under Wet Facilities

• Plug All Holes In Slab Above

• Provide Waterproof Membrane

o Air Conditioner Condensate

• Provide Pans Below Air Conditioners

• Provide Leak Detectors




March 27, 2014 28

Other Hazards• Physical Hazards

– Environmental Hazards (Cont.)

• Dirt and Fibers

o Seal Walls Around Data Center

o Provide Foot Wipes At Entrances

o Maintain Strict Clean-up Policy

• Require Pickup Of All Wire Cuttings

• No Drilling Without Adjacent Vacuum

• Clean Cartons Before Entering Machine Room

• Change Air Conditioner Filters Regularly

• Employ Professional Data Center Cleaning Service

o Maintain Positive Pressure In Machine Room

o Maintain Negative Pressure In Print Rooms

o Check For “Zinc Whisker” Growth Under Raised Floors

• Gas and Vapor Contamination

o Chemicals Used Or Stored In or Outside Building

o Chemicals Used In Building Cleaning

o Chemicals Used In Building HVAC Systems




March 27, 2014 29

Other Hazards• Other Disaster Exposures

– Flooding

• Locate Data Center Above Lowest Basement Level

o If Not Possible

• Provide Full Water Barriers – All Six Sides

• Provide Special Drainage

• Provide Generator-Backed-up Pumps & Sumps

• Build “Room Within Room”

• Ensure That Power Is Not Exposed To Flooding

o Avoid “Chicago River” Disaster

– Earthquakes, Hurricanes, Tornados

• Take All Local Precautions

• Abide By All Codes

• Have Backup Facility

– Terrorism

o Locate In “Unlikely Target” Building and Environment

o Do Not Identify Building As “Important”




March 27, 2014 30

Other Hazards• Further Considerations

– Excessive Vibration

• Usually Caused By Adjacent Construction

o Rarely A Problem For Modern Disk Drives

o Mitigate With Isolating Slab Cuts

o Prohibit Materials Dumping and Pile Driving

o Monitor Vibration At Critical Devices

o Physically Isolate Critical Devices If Necessary

• If Caused By Something Permanent

o Must Isolate Cause At Source

o Will Likely Require Professional Assessment

– Internal Noise

• Becoming A Personal Health & Safety Concern

o Large Numbers Of Server Fans

o Large Numbers Of Air Conditioners

• Difficult To Abate With Presently Available Equipment

• May Require Personal Ear Protection

• Liquid Cooling Should Eventually Reduce Noise





Accidental Unplugging

Data Cables or Power From Servers

• Remember to observe where the data cables and the power cables are when working around operational equipment.

• Be careful not to disturb the configuration of any existing equipment when adding, removing, or upgrading equipment.


Accidental Power Shutdown


Accidental Power Shutdown

• The accidental power shutdown is the single most common threat to a typical data center’s operation.

• Often, when this occurs, the spineless individual who did it will often blame it on someone else, like a cleaning lady.

• Don’t press the Emergency Shutdown button unless you really mean to.

Safety Signs Posted in theNaperville Data Center



Call Out ListThe following people should be notified in case of an

Emergency at the Naperville Data Center. Please call in

the order listed:


BP Naperville Data Center Manager

[email protected]

630-420-5765 – Office

312-758-0307 – Cellular

773-235-3080 -- Home

James Smith (BP)

GO DCT

Host Services

[email protected]

630-836-5482- Office

630-605-2541 - Mobile

Nancy Tall man (BP)

[email protected]

Area Operations Manager

Northeast and Central U.S.

Digital & Communications Technology

630-420-4581 - Office

404-713-1675 - Mobile

Jane Jones(BP)

BP Naperville DCT Services Manager

[email protected]

630-420-4397 - Office

630-292-0451 - Mobile

1

4

2

5

Bob Schoen y


Data Centers

DCT Group Operations -

Americas

[email protected]

Office Phone:

630.836.5598

Mobile Phone:

630.699.7395

3

Sample


Safety Notice

Then one of the following people should be notified in case of an Emergency at the

Naperville Data Center. Please call in the order shown.

If there is an Emergency please call



[email protected]

630-420-5765 – Office

312-758-0307 – Cellular

773-235-3080 -- Home

James Smith (BP)

GO DCT

Host Services

[email protected]

630-836-5482- Office

630-605-2541 - Mobile

Nancy Tall man (BP)

[email protected]


Northeast and Central U.S.

Digital & Communications Technology

630-420-4581 - Office

404-713-1675 - Mobile

Jane Jones (BP)

BP Naperville DCT Services Manager

[email protected]

630-420-4397 - Office

630-292-0451 - Mobile

1

4

2

5

Bob Shoney


Data Centers

DCT Group Operations -

Americas

[email protected]

Office Phone:

630.836.5598

Mobile Phone:

630.699.7395

3

Sample


Safety Notice

Updated on March 30, 2005

These are safety glasses and safety

gloves. Please use these when

performing Data Center work where

there is a danger of injuries to your

eyes and hands.

Did you remember to complete a BP

GCMS Change Request and a Job

Safety Analysis Form for the work

you are about to do? If not, please

contact the Data Center Manager right

now.

Sample


Safety NoticeIf there is an Emergency please call

This is a First Aid Kit.

Please use it to try to

assist an injured

person until

Emergency

Medical

Professionals

are available.

Sample


Safety Notice

This is a data center floor tile

lifter

Please use it carefully when

lifting data center floor tiles to

perform sub-floor Data Center

work. Works best if used on a

tile side, not the center.

Did you remember to complete a

BP GCMS Change Request

and a Job Safety Analysis

Form for the work you are about

to do? If not, please contact the

Data Center Manager right now.

Sample


ATTENTION

EMERGENCY Power Shutdown Button

Do not touch this button unless you

deliberately plan

to kill all electrical power in the

Naperville Data Center!

William F. Slater, III


[email protected]

630-420-5765 – Office

312-758-0307 – Cellular

773-235-3080 -- Home

Sample

ATTENTIONThe Rules of the BP Naperville Data Center

1. No food or drinks allowed at any time.

2. All servers and related equipment must be labeled with DNS name and IP address.

3. All visitors must be pre-approved by a member of the Site Management team and escorted at all times.

4. All Naperville Data Center hardware changes including additions, removals, and/or reconfigurations must

follow Change Management Procedures of the Downstream – GBC Change Board, and must be coordinated with

the Data Center Manager - William Slater.

5. Both electric and signal cables connected to any device in this area must be maintained in a safe and

orderly fashion.

6. No material is to be stored on top of any server rack, and a minimum of 18" clearance between racks and

the ceiling must be maintained.

7. All papers, packing materials, cardboard, boxes, plastic, etc. must be removed from the premises (including

the Tape Vault) when work is complete.

8. The Tape Vault door will be closed and locked at all times, when not in use. Contact William Slater if you

are authorized to have the combination to the Tape Vault.

9. Any non-essential or personal item left in the Data Center may be confiscated.

10. Any person with access to the Data Center must acquire familiarity with the FM200 Fire Suppression

system that is installed in this area.

Report Violations to --

William F. Slater, III


[email protected]

630-420-5765 – Office

312-758-0307 – Mobile

James Smith (BP)

BP Naperville DCT Service Manager

[email protected]

630-836-5482- Office

630-605-2541 - Mobile

Sample



Pop Quiz 3

• What is the most common number one threat to most data centers?

– Fire

– Tornados

– Weapons of Mass Destruction

– Accidentally pushing the Emergency Button that kills all power

– Floods

– Lawyers

– Viruses


Pop Quiz 3

• What is the most common number one threat to most data centers?

– Fire

– Tornados

– Weapons of Mass Destruction

– Accidentally pushing the Emergency Button that kills all power

– Floods

– Lawyers

– Viruses


Pop Quiz 4

• What is the emergency number for the Naperville Campus and Data Center?

– 911

– 5555

– 2222

– 1111

– 4444

– 1-800-EMERGENCY


Pop Quiz 4

• What is the emergency number for the Naperville Campus and Data Center?

– 911

– 5555

– 2222

– 1111

– 4444

– 1-800-EMERGENCY

Sample


Pop Quiz 5

• What is the best way to lift a

data center floor tile?– Steady yourself, place tile lifter in the

center, push to form grip, and lift

– Steady yourself, place tile lifter near

the edge of a side, push to form grip,

and lift

– Carefully

– Find a cut-out tile, steady yourself,

carefully reach in and lift


Pop Quiz 5

• What is the best way to lift a

data center floor tile?– Steady yourself, place tile lifter in the

center, push to form grip, and lift

– Steady yourself, place tile lifter near

the edge of a side, push to form grip,

and lift

– Carefully

– Find a cut-out tile, steady yourself,

carefully reach in and lift


Policies • The Rules of the Naperville Data Center must be understood and complied

with

• All work involving the addition of new equipment must be preceded by a Power Survey conducted by Meade Electric.

• All work must be coordinated with the Data Center Management with necessary paperwork completed. Ask the Data Center Manager for a Equipment Document.

• All work must have a JSA (Job Safety Analysis) Completed and Reviewed, and Signed Off Before the Work Begins.

– We have JSA templates, please ask me, and I can send via e-mail.

• Signs will be posted with Safety and Other Information

• All work must be under Change Management (in BP GCMS)

– Presently, general Data Center Change Requests go through the Downstream – GBC Change Board. CR’s involving other streams or business units may also require presentation at the Downstream – GBC Change Board as well as the Americas Regional Change Board.

• All Personnel assigned to offices on the Naperville Campus will be on an Evacuation Roll Call List and will also sign-in and sign-out using a notification board.

• In Naperville, dial 5555 on the phone in an Emergency

• Quarterly Safety Inspections

– Done by the Data Center Manager

Sample

Additional Developments


ATTENTION

The New DCT Host Services Manager, James Smith,

must approve all Changes Requests and Work Done

at Naperville Data Center. He should also be the

Change Owner on all these Change Requests

Sample



Pop Quiz 6

• Who has responsibility to

approve Change Requests and

work in the Naperville Data

Center?– Jane Jones

– Larry Scott

– William Slater

– Lord John Browne

– Naperville Fire Department Chief

– James Smith


Pop Quiz 6

• Who has responsibility to

approve Change Requests and

work in the Naperville Data

Center?– Jane Jones

– Larry Scott

– William Slater

– Lord John Browne

– Naperville Fire Department Chief

– James Smith


Conclusion

• Be Safe in the Data Center!


Questions?

Documents

Data Center Management Illinois Institute of Technology ... · • Many Data Centers and Server Rooms have raised floors. • Raised floors can be –Tripping hazards if the tiles