High Performance Computing for Real Time M & S

Randall Shumaker

Director, Institute for Simulation and Training

Growth in important IT technologiesOptical Fiber(bits/second)

Silicon Computer Chips(number of transistors)

Data Storage(bits/square inch)

Number of years

04 5321

Pe

rfo

rman

ce p

er D

olla

r Doubling Time(months)

Source: Scientific American Jan 2001“The Triumph of the Light” p81-86

High End Computing PowerHigh End Computing Power

The Top 10 MachinesNovember 2007

Rmax is in TeraFLOPS = One Trillion (1012) Floating Point Operations per second

Projected Top 500 computing power

Storage has more than kept pace

Circuit City external hard drive Circuit City external hard drive advertisement Jan 2008advertisement Jan 2008

750 Gbyte USB 2.0 drive $199.99750 Gbyte USB 2.0 drive $199.99

3.375 Gigabytes/Dollar!3.375 Gigabytes/Dollar!

Why can’t it go on forever?Source: Economist 10 May 2003

The heat dissipation issue: The heat dissipation issue: power density of Intel processors power density of Intel processors (watts/ sq Cm)(watts/ sq Cm)

IBM demonstration - 80 cores on a die

Shifting strategies - multiple processing cores on each die

What does all this mean for What does all this mean for less high end computing?less high end computing?

Comparable Trends in Nature

Time (days)0 10 20 30 40 50 60 70 80

Growth of a Weed in the Negev Desert

Hei

gh

t (in

ches

)0

5

10

1

5

20

2

5

30

3

5

40

Harvester Ant Colony Size

By these standards computers are probably late midlife in growth

When might this reasonably be expected to taper off?

(Prediction is hard, particularly about the future)

multicore

Where is the PC world now?

Apple has announced the Mac Pro dual quadTypical Configuration:

Two 3.0 GHz Quad core Intel Xeon (8 - cores)

16 G-bytes RAM

2 T-bytes Disc

Dual link DVI video 256 M bytes graphics memory

2- 20 inch flat panel graphics displays

Cost: $8,422 plus tax, free shipping

The good news: This is a highly capable system, theoretical peak performance is > 600 GigaFLOPS. Five years ago this would have been a supercomputer. In 5 years this will cost under $2000.

The bad news: Pretty much none of your current software can take effective advantage of this capability.

What are some implications of What are some implications of these numbers?these numbers?

Some perspective: Computing Power and CapabilitiesThe Hans Moravec vision

Computing Power and CapabilitiesThe Hans Moravec vision

Intel Pentium 4 EE (2005)Intel Pentium 4 EE (2005)9.726 MIPS @ 3.2GHz9.726 MIPS @ 3.2GHz

AMD Athlon FX-60 AMD Athlon FX-60 dual core (2006)dual core (2006)

22.150 MIPS @2.6 GHz22.150 MIPS @2.6 GHz

Blue gene L (2007)Blue gene L (2007)478 Tflops478 Tflops

Well Hans!, computing power isn’t really the problem

Visual metaphor for the state of the art in software developmentVisual metaphor for the state of the art in software development

Where might we like to apply Where might we like to apply more computing power other more computing power other

than building a monkey brain?than building a monkey brain?

Interactive Simulation• Needs

– Real time capability using fast processors and high-speed interconnects– High fidelity– Low latency/High bandwidth interconnects– Real time I/O– Connection to real world assets– Fixed frame rates (some apps)

• Strategies– Message Passing Interface (MPI) or Scalable Link Interface (SLI)– Ltd shared memory processing (SMP) or distributed processing

• Interfaces with sensory processors (e.g., interactive visualization, haptics, …)

• Scalability in terms of HPC architecture and simulation entities

Areas for Investigation• Extents of single image environments

– Terrain/Environment– Interacting entities

• Live, virtual, constructive experimentation– Scalable simulations– Multi-scale simulations– Control of propagating granularity

• HPC architectures for interaction– Map HPC types to applications

• Techniques for porting interactive applications to HPC platforms

• Tools for interaction

Areas for Investigation (con’t)

• Let’s remember the ‘human factor’– How will a user interact with an HPC?– How will multiple users interact with an HPC &

maintain coherence of I/O?– How will interim results be gathered?– How can timely and relevant HF experiments

be developed to influence the design?

• Get developers involved…

UCF/IST high performance UCF/IST high performance computing initiativecomputing initiative

The Purchase

• Competitive Procurement (7 Bids)

• IBM X-Series– 24 Node (192 Cores) ~1.8 TeraFLOPS– 16 GB Memory/Node (768 Gigabytes RAM)– 20 TB Spinning Storage (20,000 GigaBytes)– Red Hat Linux1– 3 Years Support

• Expect Delivery in mid Feb 2008

How this fits within the big HPC picture

1st increment

2nd increment

HPC Infrastructure Support

• Sys Admin and Parallelization Expertise• Training from IBM

– 15 Days On-Site– Tuned to Our Skill Set

• Support from IBM– 3 Year Warranty/Software Licenses

• Services– Storage/Network Config– Resource Mgmt Setup– …

Some slides we stole from Some slides we stole from Roger Smith, CTO PEOSTRI, to Roger Smith, CTO PEOSTRI, to

validate our viewsvalidate our views

HPC Applications

Batch Jobs• Computational Fluid

Dynamics• Computational

Chemistry• Protein Folding• Cryptanalysis

Interactive• JFCOM Urban

Challenge 2004• Joint SAF

Future Simulation Hardware

• HPC in its various forms may be an important part of the future of simulation … we intend to find out

• We need the support of our Team Orlando partners

• HPC offers the power to Create larger scenarios with higher fidelity Drive innovation in simulation software architecture Globally distribute training from a Simulation IT Center Coordinate different technologies for different problems:

Distributed Processing, Clusters, Shared Memory, Multicore, GPGPU, Cell

HPC in the Sim CenterShared Memory: Big ScenariosTight connection between Sims distributed across multiple CPU and memory. E.g. Very large single instance of WARSIM on 128 processors

Cluster: Multi-chunked WorldMany Sim instances on individual CPUs with local memory. E.g. Multiple geographically divided OneSAF instances.

GPU: Vector OperationsOff-loaded vector operations like rendering and LOS. E.g. Graphic heads for HPC or LOS for sims.

Shared Cluster GPU

Simulation Server

Multi-core: Multi-threaded SoftwareEvery CPU in the machine can support multiple threads.E.g. Movement, AI, Sensors parsed off to a separate core.

Training to Every Unit

C4I IT

E-BOX

Sim GUI, Web I/F, Google Earth[Analogy: GIAC, Tapestry, Phosphor]

Sim Center 21Shared Cluster GPU

Closed, Portable Sim Center

Sim IT

CTRL NET

Regional CtrCluster GPU

Network, Service Oriented Arch

HPC Competition• Suffolk - JFCOM

UR’04, Wright-Patt, Maui, Joint SAF Federation

1,000,000 Simple Entities Active

Getting their own 256 node HPC (generic cluster)

• Huntsville – MDA, SMDC

Advanced Research Center

Hypervelocity Missile Center

Redstone Technical Test Center

• Orlando – PEO-STRI, RDECOM OneSAF Chemical Plumes (SAIC) OneSAF LOS on GPU (RDECOM STTC, UNC Chapel Hill, SAIC) WARSIM on Multiple Cores (PM CONSIM, Lockheed, Northrop)

Team Orlando

• PEO-STRI, STTC, & UCF IST HPC

• $1M Congressional Earmark

• PM CONSIM Support

• Supercomputing 2007 Conference Panel

• … Bring Your Corporate Expertise Here

Competitive Summary

• Is HPC or specialized computing an important part of the future of Interactive Training Simulation?

• If so … Who are the competitors? How is Team Orlando positioned to compete? What can your organization contribute?

• Where will HPC-enabled simulation be based? Suffolk, Huntsville, Orlando, DC-area, Leavenworth

Questions?