Embed Size (px)
Citation preview
Heterogeneous Parallel ComputingAndrew David Targhetta
April 23, 2008
What to expect...I. Introduction to Heterogeneous Parallel Computing
A. What is HPC?
B. Why are we interested in HPC?
C. Problems solved in HPC environments
II. Programming in HPC environments
A. Challenges
B. Concerns
C. Techniques
III. Conclusion and Questions
Introduction
What is Heterogeneous Parallel Computing (HPC)?
Parallel computation using a collection of unlike computational machines
A HPC environment includes heterogeneous machines and high speed interconnections
Introduction
Heterogeneous parallel computing can be performed with machines of similar architecture but varying performance as seen in cluster computing
Introduction
Picture taken from Quinn's book.
Introduction
However, Heterogeneous Parallel Computing can also be performed by a network of computational machines which differ by architecture.
Obviously I drew this!
Introduction
Image courtesy of Khokhar, Prasanna, Shaaban, and Wang
Introduction
Why are we interested in HPC? HPC allows us to exploit the inherent capabilities of a
wide range of computational machines to solve difficult problems
Problems can be broken up into modules, which can be catagorized and distributed to machines built to handle specific types of problems
Introduction
Image courtesy of Khokhar, Prasanna, Shaaban, and Wang
Introduction
Problems solved in HPC environments
Driving force for HPC Includes image processing, vehicle navigation, and
environmental modeling and simulation
Programming in HPC Environments
What are some of the challenges which exist when programming in a HPC environment?
Different architectures require different compilations of code
Programmer must map portions of entire application code to various machines depending on type of computation and expected efficiency of machines
Debugging (do I need to say more???)
Programming in HPC Environments
Image courtesy of Khokhar, Prasanna, Shaaban, and Wang
Programming in HPC Environments
What are some of the major concerns involved in HPC programming?
Task synchronization and scheduling become more challenging in comparison to homogeneous systems
Load balancing problems are more defined Interconnectivity amongst heterogeneous machines adds
additional challenges (example: byte order) Modeling HPC environment is nontrivial
Programming in HPC Environments
What are some techniques that attempt to alleviate the challenges and concerns?
Code profiling is used to categorize portions of application code
Analytical benchmarking is used to determine appropriate machines
Several solution have been proposed for code mapping such as ClusterM
The Parallel Virtual Machine (PVM)
Programming in HPC Environments
Special programming environments attempt to provide an architecturally independent programming language which hides many of the concerns discussed on previous pages
High Performance FORTRAN PCN Linda PC++
Conclusion
HPC allows for best used of hardware Programming is further complicated in an HPC
environment Techniques exists which attempt to alleviate
challenges and concerns Further research includes Performance
Analysis of HPC Algorithms and Hardware Questons????
References
Khokhar, A.A., et al., Heterogeneous Computing: Challenges and Opportunities . Computer, 1993. 2626(6): p. 1827.
Quinn, M.J., Parallel Programming in C with MPI and OpenMP. McGrawHill, New York, NY, 2004.
Weems, C.C., et al., Linguistic Support for Heterogeneous Parallel Processing: A Survey and an Approach. Third Heterogeneous Computing Workshop, Canceen, Mexico, 26 Apr., 1994, pp. 8188.
Additional References
YU, W. AND COX, A. 1997. Java/DSM: A platform for heterogeneous computing. Concurrency: Pract. Exper. 9, 11 (Nov.), 1213–1224.
Sunderam V.S., Geist G.A. (1999) ”Heterogeneous Parallel and Distributed Computing.” Parallel Computing 25:16991721.
Lastovetsky, A., Reddy, R. ”On Performance Analysis of Heterogeneous Parallel Algorithms”. Parallel Computing, 30,2004.
