Rensselaer Why not change the world? Rensselaer Why not change the world? 1

RensselaerWhy not change the world?


1


RensselaerWhy not change the world?Computational Center for Computational Center for

Nanotechnology InnovationsNanotechnology Innovations

A Computational and Research Center A Computational and Research Center dedicated to dedicated to

Computational Nanotechnology InnovationsComputational Nanotechnology Innovations

A University/Industry/State PartnershipA University/Industry/State Partnership


RensselaerWhy not change the world?Rensselaer OverviewRensselaer Overview

3

Educates the leaders of tomorrow for technologically based careers

Schools – Architecture, Engineering, Humanities and Social Sciences, Management and Technology, Science

6,200 resident students – 5,000 undergraduate, 1,200 graduate

Private institution founded in 1824 450 faculty, 1400 staff



4

CCNI Vision - 1CCNI Vision - 1

The Computational Center for Nanotechnology The Computational Center for Nanotechnology Innovations (CCNI) will Innovations (CCNI) will

bring together university and industry researchers bring together university and industry researchers toto

address the challenges facing the semiconductor address the challenges facing the semiconductor industry as devices shrink in size to the industry as devices shrink in size to the nanometer range.nanometer range.



5


To account for the interactions of atoms and To account for the interactions of atoms and molecules up to the behavior of a complete molecules up to the behavior of a complete device, the CCNI mustdevice, the CCNI must

develop a new generation of computational develop a new generation of computational methods tomethods to

support the virtual design of the next generation support the virtual design of the next generation of devices which will of devices which will

require the massive computingrequire the massive computingcapabilities of the CCNI. capabilities of the CCNI.



6


The resulting virtual design methods will further The resulting virtual design methods will further expand New York State’s leadership position in expand New York State’s leadership position in nanotechnology.nanotechnology.



7

Industry NeedsIndustry Needs

NeedsNeeds– Technical and cost constraints are limiting Technical and cost constraints are limiting

the growth of the semiconductor industry the growth of the semiconductor industry and nanotechnology innovationsand nanotechnology innovations

– Computational nanotechnology is essential Computational nanotechnology is essential for decreasing the time from concept for decreasing the time from concept creation to commercializationcreation to commercialization



8

CCNI GoalsCCNI GoalsGoalsGoals Provide leadership in the development and Provide leadership in the development and

application of computational nanotechnologies application of computational nanotechnologies Establish partnership to create world class Establish partnership to create world class

competencies on design-to-manufacturing competencies on design-to-manufacturing research capabilitiesresearch capabilities

Produce new integrated predictive design tools Produce new integrated predictive design tools for nano-scale materials, devices, and systemsfor nano-scale materials, devices, and systems

Spur economic growth inSpur economic growth inthe Capital district, the Capital district, NYS & beyondNYS & beyond



9

Facilities and CapabilitiesFacilities and Capabilities

Computational SystemsComputational Systems– 100 teraflops of computing100 teraflops of computing– Heterogeneous computingHeterogeneous computing

environmentenvironment

Rensselaer Technology ParkRensselaer Technology Park– 4300 sq. ft. Machine Room4300 sq. ft. Machine Room– Business OfficesBusiness Offices– Systems and Operations supportSystems and Operations support– Scientific SupportScientific Support



10

Layout of CCNILayout of CCNI



Design and Engineering

11

Partners to Build CCNIPartners to Build CCNI



12


Architect



Turner Construction CompanyTurner Construction Company

13




14

CCNI ConstructionCCNI Construction

Raised FloorRaised Floor



15


Cooling TowersCooling Towers



16


LobbyLobby



17

CCNI InstallationCCNI Installation

Blue Gene Racks and Inter-rack Cables



18


Blue Gene Racks Without Covers



19


Blade Racks, Storage Racks, and Network Cables



20

CCNI – Blue Gene/LCCNI – Blue Gene/L

Blue Gene/L SystemBlue Gene/L System– 16 rack IBM Blue Gene/L system16 rack IBM Blue Gene/L system– #7 on Top 500 Supercomputer List#7 on Top 500 Supercomputer List– 32,768 PowerPC 700 MHz processors32,768 PowerPC 700 MHz processors– 12 TB of memory total12 TB of memory total– Compute Node KernelCompute Node Kernel

Simple, flat, fixed-size, address space Simple, flat, fixed-size, address space Single threaded, no pagingSingle threaded, no pagingFamiliar POSIX interfaceFamiliar POSIX interfaceBasic file I/O Basic file I/O operationsoperations

– Two modes - coprocessor Two modes - coprocessor or virtual mode or virtual mode


RensselaerWhy not change the world?Blue Gene/L HardwareBlue Gene/L Hardware

21

Optimized Communications1) 3D Torus2) Collective Network3) Global Barrier/Interrupt4) Gigabit Ethernet (I/O & connectivity)5) Control (system boot, debug, monitoring)

5.6 GF/s4 MB

11.2 GF/s2 GB

180 GF/s16 or 32 GB

5.7 TF/s (peak)512 GB or 1 TB

91.7 TF/s (peak)12 TB

16 Racks


RensselaerWhy not change the world?Blue Gene ArchitectureBlue Gene Architecture

22



23

CCNI – Blade Servers CCNI – Blade Servers

Blade Server ClusterBlade Server Cluster– 462 IBM LS21 blades462 IBM LS21 blades– 1,848 Opteron 2.6 GHz cores1,848 Opteron 2.6 GHz cores– 5.5 TB of memory total5.5 TB of memory total– 4X InfiniBand interconnect 4X InfiniBand interconnect

(10 Gbps)(10 Gbps)– Red Hat LinuxRed Hat Linux



24

CCNI – Large Memory CCNI – Large Memory

AMD and Intel SMP ServersAMD and Intel SMP Servers 40 IBM x3755 servers40 IBM x3755 servers

– Each with 8 Opteron 2.8 GHz cores and 64 GB of memoryEach with 8 Opteron 2.8 GHz cores and 64 GB of memory 2 IBM x3755 servers2 IBM x3755 servers

– Each with 8 Opteron 2.8 Ghz cores and 128 GB of memoryEach with 8 Opteron 2.8 Ghz cores and 128 GB of memory 2 IBM x3950 servers2 IBM x3950 servers

– One with 64 Xeon 2.8 GHz cores and 128 GB of memoryOne with 64 Xeon 2.8 GHz cores and 128 GB of memory– One with 32 Xeon 2.8 GHz cores and 256 GB of memoryOne with 32 Xeon 2.8 GHz cores and 256 GB of memory

All with 4X InfiniBand interconnectAll with 4X InfiniBand interconnect All with Red Hat LinuxAll with Red Hat LinuxPower SMP ServerPower SMP Server

– IBM p590IBM p590– 16 Power 5+ 2.1GHz processors16 Power 5+ 2.1GHz processors– 256 GB of memory256 GB of memory– AIXAIX



25

CCNI – Disk StorageCCNI – Disk Storage

File StorageFile Storage– Common file system for all hardwareCommon file system for all hardware– IBM General Parallel File System, IBM General Parallel File System,

GPFSGPFS– 832 TB of raw disk storage832 TB of raw disk storage– 52 IBM x3655 file server nodes52 IBM x3655 file server nodes– 26 IBM DS4200 storage controllers26 IBM DS4200 storage controllers

GPFSGPFS– High performance parallel I/OHigh performance parallel I/O– Cache consistent shared accessCache consistent shared access– Aggressive read ahead, write behindAggressive read ahead, write behind


26

Local fiber:CCNI/Campus/NYSERNet

State

International

CCNI NetworkingCCNI Networking



27

– Nanoelectronics modeling Nanoelectronics modeling and simulationand simulation

– Modeling of material structure Modeling of material structure and behaviorand behavior

– Modeling of complex flowsModeling of complex flows– Computational biologyComputational biology– Biomechanical system modelingBiomechanical system modeling– Multiscale methodsMultiscale methods– Parallel simulation technologiesParallel simulation technologies

CCNI – Research AreasCCNI – Research Areas



28

– Functionality of new materials and devicesFunctionality of new materials and devices– Fabrication modelingFabrication modeling– Mechanics of nanoelectronic systemsMechanics of nanoelectronic systems

– Application to the design of new devicesApplication to the design of new devices

Nanoelectronics Modeling andNanoelectronics Modeling andSimulationSimulation

carbon nanotubeT-junctions (Nayak)

submicron to nano (Huang)



29

– Modeling and design of material systemsModeling and design of material systems– Modeling of energetic materials Modeling of energetic materials – Multiscale modeling of nanostructured polymer rheologyMultiscale modeling of nanostructured polymer rheology

Modeling of Material Structure andModeling of Material Structure andBehaviorBehavior

Multiscale modeling of polymer rheolgy

Matrix Compatible Block (MCB)Property Enhancing Block (PEB)

Controlled InhomogeneityDesigned Interfaces Macroscopic Layering



30

– Hierarchic modeling of turbulent flowsHierarchic modeling of turbulent flows– Modeling of biological systems flowsModeling of biological systems flows

Modeling of Complex FlowsModeling of Complex Flows



31

– Protein structure and interactions with small moleculesProtein structure and interactions with small molecules– Membranes and membrane protein structure and functionMembranes and membrane protein structure and function– Modeling cellular processes and communities of cellsModeling cellular processes and communities of cells

Computational BiologyComputational Biology

dppc

fentanyl

cholesterol



32

– Virtual biological flow facility for Virtual biological flow facility for patient specific surgical planningpatient specific surgical planning

– Distributed digital surgery Distributed digital surgery – Biomedical imaging via inverse Biomedical imaging via inverse

problem constructionproblem construction

Biomechanical System ModelingBiomechanical System Modeling



33

– Multiscale mathematics and Multiscale mathematics and modelingmodeling

– Adaptive simulation systems Adaptive simulation systems applied to applicationsapplied to applications

Multiscale Science and Engineering Multiscale Science and Engineering



34

– High-performance network modelsHigh-performance network models– Optimistic parallel approachesOptimistic parallel approaches– Multi-level parallel network modelsMulti-level parallel network models

Parallel Simulation TechnologiesParallel Simulation Technologies

Geometric model Partition model Partitioned mesh

Adapted mesh (23,082,517 tets)Initial mesh (1,595 tets)

RensselaerWhy not change the world?Thus Begins the CCNI OdysseyThus Begins the CCNI Odyssey

35

RensselaerWhy not change the world?QuestionsQuestions

36



37

Documents

Rensselaer Why not change the world? Rensselaer Why not change the world? 1