Embed Size (px)
DESCRIPTION
Network Systems Lab. Korea Advanced Institute of Science and Technology No.3 Need for Parallel & Distributed Computation Restrict attention to numerical computation in this lecture Large-scale fast computation Partial differential eq.s (PDEs) e.g.) fluid dynamics, weather prediction, image processing Can be decomposed along a spatial dimension Each processor manipulates the variables associated with a small region in space Interaction between variables are local in nature Systems of equations, mathematical programming (optimization)
Citation preview
Network Systems Lab.Korea Advanced Institute of Science and Technology
No.1
Ch. 1 Introduction
EE692
Parallel and Distribution Computation | Prof. Song Chong
Network Systems Lab.Korea Advanced Institute of Science and Technology
No.2
Issues in Parallelization Task allocation to processors
The breakdown of the total workload in small tasks assigned to different processors
Proper sequencing of the tasks when some of them are interdependent and cannot be executed simultaneously
Communication of interim computation results between processors
Synchronization of computations of processors Synchronous: predetermined points for the completion of
computations or for the arrival of data Asynchronous: no such points
Performance measures Effects of system’s architecture on performance
Network Systems Lab.Korea Advanced Institute of Science and Technology
No.3
Need for Parallel & Distributed Computation Restrict attention to numerical computation in this lecture Large-scale fast computation
Partial differential eq.s (PDEs)e.g.) fluid dynamics, weather prediction, image processing Can be decomposed along a spatial dimensionEach processor manipulates the variables associated with a small
region in space Interaction between variables are local in nature
Systems of equations, mathematical programming (optimization)
Network Systems Lab.Korea Advanced Institute of Science and Technology
No.4
Need for Parallel & Distributed Computation (Cont’d) Analysis, simulation and optimization of large scale
interconnected systems e.g.) queueing system Information acquisition, extraction and control in
geographically distributed system e.g.) sensor networks, communication networks, wireless networks Synchronization, unreliable communication, absence of a central
control mechanism
Network Systems Lab.Korea Advanced Institute of Science and Technology
No.5
Distinction between parallel and distributed computing systems Parallel computing systems
Processors are located within a small distance of each other Designed in such a way that processors execute jointly a
computational task Communication between processors is reliable and predictable
Distributed computing systems Processors may be far apart (geographically distributed) Communication delay may be unpredictable Communication links may be unreliable Topology may undergo changes while operating Usually loosely coupled, there is very little central coordination
and control
Network Systems Lab.Korea Advanced Institute of Science and Technology
No.6
Parameters to classify parallel and/or distributed computing systems Type and number of processors Presence or absence of a global control mechanism Synchronous vs. asynchronous operation Processor interconnections
Tow extremes Shared memory system
• solve inter-processor computation problem by lacing a global memory, but introduce the problem of simultaneous accessing of different locations of the memory by several processors
Message-passing system• local memory with inter-connection networks
