Upload
others
View
11
Download
0
Embed Size (px)
Citation preview
Introduction to Parallel Computing
Irene MoulitsasProgramming using theMessage-Passing Paradigm
MPI BackgroundMPI : Message Passing InterfaceBegan in Supercomputing ’92
VendorsIBM, Intel, Cray
Library writersPVM
Application specialistsNational Laboratories, Universities
Why MPI ?One of the oldest librariesWide-spread adoption. Portable.Minimal requirements on the underlying hardwareExplicit parallelization
Intellectually demandingAchieves high performanceScales to large number of processors
MPI Programming StructureAsynchronous
Hard to reasonNon-deterministic behavior
Loosely synchronousSynchronize to perform interactionsEasier to reason
SPMDSingle Program Multiple Data
MPI FeaturesCommunicator InformationPoint to Point communicationCollective CommunicationTopology SupportError Handling
Six Golden MPI FunctionsMPI is 125 functionsMPI has 6 most used functions
MPI Functions: Initialization
Must be called by all processesMPI_SUCCESS
“mpi.h”
MPI Functions: Communicator
MPI_CommMPI_COMM_WORLD
Hello World !
Hello World ! (correct)
MPI Functions: Send, Recv
sourceMPI_ANY_SOURCE
MPI_StatusMPI_SOURCEMPI_TAGMPI_ERROR
MPI Functions: Datatypes
Send/Receive Examples
Blocking Non-Buffered Communication
Send/Receive Examples
Blocking Buffered Communication
Send/Receive Examples
MPI Functions: SendRecv
MPI Functions: ISend, IRecv
Non-blockingMPI_Request
MPI Functions: Test, Wait
MPI_Test tests if operation finished.MPI_Wait blocks until operation is finished.
Non-Blocking Non-Buffered Communication
Example
Example
Example
MPI Functions: Synchronization
Collective CommunicationsOne-to-All BroadcastAll-to-One ReductionAll-to-All Broadcast & ReductionAll-Reduce & Prefix-SumScatter and GatherAll-to-All Personalized
MPI Functions: Broadcast
MPI Functions: Scatter & Gather
MPI Functions: All Gather
MPI Functions: All-to-All Personalized
MPI Functions: Reduction
MPI Functions: Operations
MPI Functions: All-reduceSame as MPI_Reduce, but all processes receive the result of MPI_Op operation.
MPI Functions: Prefix Scan
MPI Names
MPI Functions: Topology
Performance EvaluationElapsed (wall-clock) time
Matrix/Vector Multiply