<Insert Picture Here>

Austin Data Center – 6Sigma DC CFD Model

Keith WardKeith WardData Center Facilities Engineering and Logistics Design

Austin Data CenterStatistics - Space

•Original AT&T Designed and built building, but never operational•Purchased 200290 D D i Fi l Fit O t d C i i i•90 Day Design, Final Fit Out and Commissioning

•72,000 sqft 30” Pressurized Raised Floor•2240 Rack Locations for Customer IT Equipment• Underfloor Structured Power and Signal Cable InfrastructureUnderfloor Structured Power and Signal Cable Infrastructure•94% of Data Center Floor has 25 foot Ceiling •144,000 sqft Building •Deployment of IT Equipment is a Modular IT Equipment Cabinet (Roll-In) Architecture with Access Layer Switches in each cabinet•Zoned Floor Space, Power and Network Topology

•Divided into 10 Customer IT Spaces (Quads)Divided into 10 Customer IT Spaces (Quads)•Each Quad has an Network Intermediate Distribution Facility (IDF)•All Network IDFs feed into a central Main Distribution Facility (MDF)Facility (MDF)

Austin Data Center

Power• 8.1MW Parallel Redundant UPS Systems• N+1 Generator Backup ( 7 – 2MW Diesel GenSets)Ge e ato ac up ( ese Ge Sets)• Structured power delivery to each Rack Location

– NEMA L21-20 in Low Density Area– NEMA L21-30 in Heat Containment Area

Cooling• N+1 Chilled Water System• Supply Side Temperature Controlled CRACs• 600 Rack Locations out of 2,240 leverage Heat Containment System

(2008 ASHRAE Best Practices, 2010 ASHRAE Technology Award)• Hot and Cold Aisle Configuration• Increasing Inlet Temperatures• Intuitive Cooling Management

Austin Data CenterStatistics – Installed Customer Equipment

Install Base• Current contents:

– ~25,000 Customer Compute Servers– 6 PetaByte Raw HDD Storage (SAN/NAS/DAS)

• Workflow-Roles Based Life Cycle Asset Management System• Average 30% Tech-Refresh annually• Houses Oracle Internal Production, On-Demand and Development

Organizations. (Diverse Needs)ff f• People – minimal on site personnel; Facilities, Logistics, small staff of

System and Network Administrators

Austin Data CenterCFD Modeling Software Evaluation Characteristics

• Library of Individually Characterized Vendor Equipment • Complex Cooling Strategies (VFD, Intelligent Heat Containment System)Co p e Coo g St ateg es ( , te ge t eat Co ta e t Syste )• Build the model in 3 Dimensions• In house control over solution build and processing• Integration OpportunitiesIntegration Opportunities• Enhancement Requests• Support

Austin Data CenterBuilding the Modelg

Phase I• Modular Approach• Building• Under Floor Cabling• CRACs w/turning vanes• PDU’s• Fixed 2-post racks• Drop Ceiling

6Sigma Virtual Facility Buildup: Shell

6Sigma Virtual Facility Buildup: Shell & UnderFloor

6Sigma Virtual Facility Buildup: Raised Floor

6Sigma Virtual Facility Buildup: Power and Cooling

Austin Data CenterBuilding the Model

Phase 2• Cabinets

E i t• Equipment

6Sigma Virtual Facility Buildup: Cabinets & Equipment

Austin Data CenterBuilding the Modelg

• Heat Containment System• Plenum• Straight Fan Units• Intelligent Fan Units

6Sigma Virtual Facility Buildup: Cabinets

• Chimney Cabinets for High-Density Equipment

• Passive Chimney CabinetsPassive Chimney Cabinets• Active Chimney Cabinets –Fixed Flow

• OpenGate Data Systems EC20’s used to transition out old chimneyused to transition out old chimney cabinets• 2-Post Racks to house switching equipment

6Sigma Virtual Facility Buildup: OpenGate Data Systems EC20

•EC20 Fan Unit

• Pressure Sensors placed ‘inside’ and ‘outside’ the cabinet• Fan tray tied to the sensors such that they ‘zero

t’ th bi t i th bi tout’ the pressure cabinet in the cabinet• Agnostic to any equipment placed within the cabinet• Can be configured to fit with any cabinetAbilit t 2000 CFM•Ability to move ~2000 CFM

6Sigma Virtual Facility Buildup: OpenGate Data Systems EC20

•

•Traditional Chimney Cabinet •OpenGate Data Chimeny Cabinet

6Sigma Virtual Facility Buildup: OpenGate Data System

Austin Data CenterA very large model

•Over 200 New Library Items Built

•20MB 6Sigma *.room model

•120 Million Cells

•Solution takes 72 hours with 8 process streams and a 80GB memory process footprintmemory process footprint

Austin Data CenterCFD Analysis and Continuing Operations

• Phase III– Calibrate– Analyze

• Phase IV– Continue Updates– What if Scenarios– Design Enhancements

6Sigma Virtual Facility Buildup: Air Temperature ½ Cabinet Height

• Overall air temperature is• Overall air temperature is between 68 and 75 deg-F •Higher air temperatures in some areas

6Sigma Virtual Facility Results: Equipment Inlet Temperatures

• Model calibration will reveal ‘true’ hot-spots• Inefficiency of the passive chimney racks – pressure build up leads to recirculation• Current overheat is at the Cisco 1U Switches mounted on rear rails –

• Oracle designed, Opengate Data built, 1U cooling duct solution has already been implemented (SA1), 10 to 15 degree drop in temperaturebeen implemented (SA1), 10 to 15 degree drop in temperature

6Sigma Virtual Facility Results: Pressure and Flow Rates

• High pressure buildup on the top half of the VF• Model needs to be calibrated to reflect the correct under-floor pressure

6Sigma Virtual Facility Results: ACU Operational Cooling Capacity

• Cooling capacities are quite balanced• Effect of VFDs’ needs to be examined in the high-density areas

FutureNew Data Center

FutureCFD Modeling

• Dynamic

• Asset Management

• Active Sensor InputActive Sensor Input

• Intelligent Alerts