Data Center Design

Fiber Connect, Inc.

Fiber Optic Overview & Training

Fiber Connect ©2003

Data Center Design and the

ANSI/TIA-942 Standard

BICSI Regional Meeting – Ellicott, MD

Rick Foster RCDD,TLTCommunications Infrastructure Group

July 27, 2006

Fiber Connect, Inc.



What is ANSI/TIA-942?

• Published by TIA in April 2005

• “Telecommunications Infrastructure Standard for Data Centers”

• Relationship to other standards

• Excellent source of information

Data Center Design and the ANSI/TIA-942 Standard

Fiber Connect, Inc.



Codes & StandardsCodes & Standards

Fiber Connect, Inc.




CODES vs STANDARDS

• CODE: A regulation that is enforceable.

It is the Law !

• STANDARD: A recommend practice and performance requirements.

It is our Profession !

Fiber Connect, Inc.



Data Center Standards – TIA-942– NFPA – 70 National Electric Code – Article 645

– NFPA-75 – Data Centers– NFPA-76 – Central Office– International Standards & BICSI


Fiber Connect, Inc.



Historical Perspective

• Mainframes had their own unique requirements

• Separate Room Utilities – separate space

• Little movement after initial installation

• Vendor Specific equipment and cabling


Fiber Connect, Inc.



Today

No longer a static environment

– New technologies = new electronics

– Convergence = multiple systems

– Room utilities are constantly challenged


Fiber Connect, Inc.



Data Center is the Information Traffic CopData Center is the Information Traffic Cop

• Voice over IP * (10G)• IP-based distance learning• Streaming digital audio and video over IP, • Scientific modeling• SAN to SAN trafffic * (10G)• Online gaming• Unified Messaging• Telemedicine and Teleradiology• WAN/MAN interface * (10G)

Fiber Connect, Inc.



Business Risk

Proper Design = Redundancy = Continuity

Business Risk

Proper Design = Redundancy = Continuity• Room Layout• Equipment Types• Environmental - HVAC & Power• Change Control• Fire & Safety• Security Issues• IT Infrastructure – Physical Layer

Structured Design approach


Fiber Connect, Inc.



Enterprise CablingEnterprise Cabling

Data Center Cabling

Are they really any different?

TR’s CP Backbone WAO EF’s Admin

Horizontal

EDA ZDA Backbone MDA DATA CENTER CR

HDA Horizontal


Fiber Connect, Inc.




Why another standard?Why another standard?“The purpose of this standard is to provide

requirements and guidelines for the design and installation of a data center or computer room. It is intended for use by designers who need a comprehensive understanding of the data center design including the facility planning, the cabling system and the network design.”

Source: ANSI/TIA-942 2005

Fiber Connect, Inc.




TIA 942 Design Considerations


Fiber Connect, Inc.




ANSI/TIA-942 StandardANSI/TIA-942 Standard

– Defines new terms

– Recommends media selections

Correlation to existingANSI/TIA/EIA 568 & 569

– Defines the physical

environmentSource: ANSI/TIA-942 2005

Fiber Connect, Inc.





Spaces

– Computer Room– Telecommunications Room– Entrance Room– Main Distribution Area (MDA)– Horizontal Distribution Area (HDA)– Equipment Distribution Area (EDA)– Zone Distribution Area (ZDA)


Fiber Connect, Inc.





Cabling & Topology

– Backbone– Horizontal– Main cross-connect– Horizontal cross-connect– Cross-connect in the entrance room or MDA– Zone outlet– EDA outlet


Fiber Connect, Inc.





Spaces

Computer Room

Data Center

MDA EDA

HDA ZDA


Fiber Connect, Inc.




Entrance Room

- Space where the carrier circuits, access providers and campus facilities enter the building

- Small / Medium Centers could be located in the data center itself

Data Center Telecommunications SpacesData Center Telecommunications Spaces

Could be the Entrance Facility

Fiber Connect, Inc.




Telecommunications Room (TR)

- Space dedicated to the office and operations of the Data Center

- NOC center

- May be combined with an HDA/MDA


Not to be confused with a enterprise TR

Fiber Connect, Inc.




Main Distribution Area (MDA)

- It’s the central space where the point of distribution for the structured cabling system in the data center is located

- The data center shall have at least one MDA

- The core routers and core switches for the data center are often located in or near the MDA


Could be a Equipment Room

Fiber Connect, Inc.




Horizontal Distribution Area (HDA)

- It’s the space that supports cabling to the equipment distribution areas

- LAN, SAN and KVM switches that support the end equipment are also typically located in the HDA

- There should be a minimum of one horizontal distribution area per floor


Could be a Telecom Room (TR)

Fiber Connect, Inc.




Equipment Distribution Areas (EDA)

- Are spaces allocated for the end equipment, including computer systems and communications equipment

-End equipment is usually standing equipment or equipment mounted in cabinets or racks

- Horizontal cables are terminated in equipment distribution areas or connecting hardware mounted in cabinets or racks

- Point-to-point cabling is permitted between equipment located in the equipment distribution area. Cable should be greater than15 m and should connect equipment in adjacent racks or cabinets in the same row


Could be a Work Area

Fiber Connect, Inc.




Zone Distribution Area (ZDA)

- Should be limited to serving a maximum of 288 coaxial or twisted pair connections

- Cross-connection shall not be used in the ZDA

- No more than one ZDA shall be used within the same horizontal cable run

- There shall be no active equipment in the zone distribution area, except DC powering equipment


Could be a Consolidation Point

Fiber Connect, Inc.



Ability to connect each system without disrupting the work flow

MDA or HDA

ZDA

HDA or EDA

Fiber Connect, Inc.



HDA or EDA MDA or HDA

Structured Approach

Entrance Room

Telecom Room

Support Rooms

Fiber Connect, Inc.




Recognized Media

100Ω ohm twisted-pair copper cable• Cat-5e• Cat-6 recommended

Multimode fiber optic cable• 62.5/125 µm or 50/125 µm• 50/125 µm 850 nm laser optimized MMF recommended

Single Mode Fiber Optic Cable

75 Ω ohm Coax Cable

Fiber Connect, Inc.




Data Center RedundancyData Center Redundancy•Tiers - Annex G of ANSI/TIA-942

Tier 1 – Basic data centerTier 2 – Redundant componentsTier 3 – Concurrently maintainableTier 4 – Fault tolerant

Corresponds to the tier ratings defined by the Uptime Institute

Fiber Connect, Inc.




Data Center Tiers


Fiber Connect, Inc.




Summary – ANSI/TIA- 9421. Standard specifies data center planning and design

procedures. Structured approach.

2. Provides guidelines and recommendations for data centersimplementation, embraces much more thantelecommunications infrastructure (power, pathways, fireprotection, doors, walls treatment, and others)

3. Determines four tiers for data center redundancy

4. Considers ‘568-B, ‘569-B, ‘606-A and ‘607 for the cabling, pathways, administration and earthing design, installationsand other procedures

Fiber Connect, Inc.



Additional Standards

International

CENELEC EN 50173-5“Information Technology –Generic Cabling

Systems – Part 5 Data Centers

CENELEC EN 50174-2 - Annex on Data Centerplanning

ISO/IECProposed “Generic Cabling or Data Centers”

Data Center sub-committee to standards group

Best Practices Document

Data Center Design - Codes and Standards

Fiber Connect, Inc.



Source: NEC 2005 NFPA 70



Fiber Connect, Inc.




National Electric CodeNational Electric Code• Protects people and property from hazards

arising from the use of electricity.• It is not a “DESIGN SPECIFICATION” or

training manual for the untrained individuals.


Article 645.1 Scope. This article covers equipment, power-supply wiring, equipment interconnecting wiring, of information technology equipment and systems, including terminal units, in an information technology equipment room.

FPN: For further information, see NFPA 75-2003,Standard for the Protection of Electronic Computer/Data Processing Equipment.

Fiber Connect, Inc.



645.2 Special Requirements for Information TechnologyEquipment Room. This article shall apply, providedall the following conditions are met:

(1) Disconnecting means complying with 645.10 are provided.

(2) A separate heating/ventilating/air-conditioning (HVAC)system is provided that is dedicated for informationtechnology equipment use and is separated from otherareas of occupancy. Any HVAC system that servesother occupancies shall be permitted to also serve theinformation technology equipment room if fire/smokedampers are provided at the point of penetration of theroom boundary. Such dampers shall operate on activationof smoke detectors and also by operation of thedisconnecting means required by 645.10.

FPN: For further information, see NFPA 75-1999, Standardfor the Protection of Electronic Computer/Data ProcessingEquipment, Chapter 8, 8-1, 8-1.1, 8-1.2, and 8-1.3.

(3) Listed information technology equipment is installed.

(4) The room is occupied only by those personnel neededfor the maintenance and functional operation of theinstalled information technology equipment.

(5) The room is separated from other occupancies by fireresistant-rated walls, floors, and ceilings with protectedopenings.

FPN: For further information on room construction requirements,see NFPA 75-1999, Standard for the Protectionof Electronic Computer/Data Processing Equipment, Chapter 3.

National Electric Code NFPA-70 Article - 645

Article 645 applies only if all


of the following requirements are met

• Disconnecting means at exits

• Separate HVAC equipment or dampers

• Listed equipment

• Restricted access to personnel

• Separated from other areas with fire-rated construction

Data Center Design - Codes and Standards

Fiber Connect, Inc.




NFPA – 75Standard for the Protection of Electronic Computer/Data

Processing Equipment.

NFPA – 75Standard for the Protection of Electronic Computer/Data

Processing Equipment. NFPA-70 National Electric Code

645.1 Scope. This article covers equipment, power-supply wiring, equipment interconnecting wiring, of information technology equipment and systems, including terminal units, in an information technology equipment room.

FPN: For further information, see NFPA 75- 2003, Standard for the Protection of Electronic Computer/Data Processing Equipment.

ANSI/TIA-942


Fiber Connect, Inc.




NFPA Standard NFPA - 75NFPA Standard NFPA - 751.1 Scope. This standard covers the

requirements for the protection of information technology equipment and information technology equipment areas.

1.2* Purpose. The purpose of this standard is to set forth the minimum requirements for the protection of information technology equipment and information technology equipment areas from damage by fire or its associated effects —namely, smoke, corrosion, heat, and water.Source: NFPA -75

How to build it – A systems approach

Fiber Connect, Inc.




NFPA Standard NFPA - 75NFPA Standard NFPA - 75Chapter 4 Risk Considerations

4.1* Risk Factors. The following factors shall be consideredwhere determining the need for protecting the environment,equipment, function, programming, records, and supplies:

(1) Life safety aspects of the function (e.g., process controls,air traffic controls)(2) Fire threat of the installation to occupants or exposedproperty(3) Economic loss from loss of function or loss of records(4) Economic loss from value of equipment

4.2 Telecommunications Risks. In assessing and evaluating the damage and interruption potential of the loss of information technology equipment room operations, attention shall be given to the impact of the loss of data and communications lines. If these functions are vital to the operation, rooms housing the services shall be constructed in accordance with Chapter 5 and protected in accordance with Chapter 8. These rooms shall be secured, locked, and free of extraneous combustibles.

Source: NFPA -75

Reduce Risk = Business Continuity

Fiber Connect, Inc.




NFPA Standards NFPA - 76NFPA Standards NFPA - 76

4.2 Communications Risks. In assessing and evaluating the damage and interruption potential of the loss of communication operations, attention should be given to the impact of the loss of data, voice, and video communications links. The complexity and scope of switching equipment operations can make it necessary to provide internal redundancy, alternative routing, and in some cases dual access terminal in order to prevent communication outages.Source: NFPA -76

Reduce Risk = Business Continuity

Central Office Space

Fiber Connect, Inc.



ReferencesReferences

• American Society of Heating, Refrigerating and Air-Conditioning Engineers (ASHRAE) - http://www.ashrae.org/

• ASHRAE Thermal Guidelines for Data Processing Environments

• Electronic Industries Association (EIA) – www.eia.org• EIA-310-D Cabinets, Racks, Panels, and Associated Equipment

• National Fire Protection Association (NFPA) - http://www.nfpa.org• NFPA 75 Standard for the Protection of Information Technology

Equipment • NFPA 70 National Electrical Code are and NFPA 101 Life Safety Code

also applicable

http://www.ashrae.org/

http://www.eia.org/

http://www.nfpa.org/

Fiber Connect, Inc.



ReferencesReferences

• Telecommunications Industry Association (TIA) -http://www.tiaonline.org/TIA-942 (Proposed) Telecommunications Infrastructure Standard for Data CentersOther TIA cabling standards (568, 569, 606, 607, 758) are also applicable.

• Telecordia Technologies - http://www.telcordia.com/Telcordia GR-63-CORE (NEBS) Requirements: Physical Protection A central office specification, but referenced by TIA-942 for seismic specifications, vibration testing, floor loadingcapacity measurement and test methods

http://www.tiaonline.org/

http://www.telcordia.com/

Fiber Connect, Inc.



• 7x24 Exchange - http://www.7x24exchange.org/• AFCOM – www.afcom.com• Institute for Data Center Professionals (IDCP) – www.idcp.org• International Facility Management Association (IFMA) –

www.ifma.org• Uptime Institute – www.upsite.org• Construction Specification Institute – www.csinet.org• The Data Center Journal - www.datacenterjournal.com

Data Center User Groups and

Information Exchanges

Data Center User Groups and

Information Exchanges

http://www.7x24exchange.org/

http://www.afcom.com/

http://www.idcp.org/

http://www.ifma.org/

http://www.upsite.org/

http://www.csinet.org/

Fiber Connect, Inc.



Today’s Objective

1. Overview of TIA-942

2. Introduction to other standards

3. Encourage you to utilize these documents as reference and quality assurance tools.

Stay current – no such thing as “grand-fathered”

Documents

Data Center Design