View
21
Download
0
Category
Preview:
DESCRIPTION
Semantics of Minimally Synchronous Parallel ML. Myrto Arapini, Frédéric Loulergue, Frédéric Gava and Frédéric Dabrowski LACL, Paris, France. Outline. Introduction Pure functional minimally synchronous parallel programming High level semantics Distributed evaluation - PowerPoint PPT Presentation
Citation preview
Semantics of Minimally Synchronous Parallel ML
Myrto Arapini, Frédéric Loulergue, Frédéric Gava and
Frédéric DabrowskiLACL, Paris, France
F. Gava SNPD 2003 2
Outline
Introduction Pure functional minimally
synchronous parallel programming High level semantics Distributed evaluation Conclusions and future work
F. Gava SNPD 2003 4
Our previous work:Bulk Synchronous Parallelism + Functional Programming = BSML
Bulk Synchronous Parallelism : Scalability Portability Simple cost model
Functional Programming : High level features (higher order
functions, pattern matching, concrete types, etc.)
Programs proofs Safety of the environment
F. Gava SNPD 2003 5
What is MSPML ?Why MSPML ? = Minimally Synchronous Parallel ML BSML without synchronization barriers
Comparison of efficiency BSML/MSPML Further extensions restricted or
impossible in BSML (divide-and-conquer, nesting of parallel values)
Basis for an additional level to BSML for Grid computing: BSML on each nodes (clusters) + MSPML for coordination
F. Gava SNPD 2003 7
The MSPML Library Parallel ML library for the
Objective Caml language operations on a parallel data
structureAbtract type: par
access to the machine parameters:
p: unit int
p() = number of processes
F. Gava SNPD 2003 9
Pointwise Parallel Application apply: ( ) par par par
apply
=
fp-1…f1f0
vp-1…v1v0
fp-1 vp-1…f1 v1f0 v0
F. Gava SNPD 2003 10
Example (1) let replicate x = mkpar(fun pid->x)replicate: ’a -> ’a par
(replicate 5)
5…55
F. Gava SNPD 2003 11
Example (2) let parfun f v = apply (replicate f) vparfun: (’a->’b)->’a par->’b par
parfun (fun x->x+1) parallel_vectorf
vp-1…v1v0
f…ffapply
= (f vp-1 )…(f v1 )(f v0 )
F. Gava SNPD 2003 12
Communication Operation: get
get: parint par par
get
vp-1…v1v0
ip-1…i1i0(=1)
vip-1…vi1
vi0
=
F. Gava SNPD 2003 13
Example (3)let bcast_direct root parallel_vector =if not(within_bounds root)then raise Bcast_Error else get parallel_vector
(replicate root)
bcast_direct: int->’a par->’a par
vp-1…v1v0
vroot…vrootvroot
(bcast_direct root
=
)
F. Gava SNPD 2003 14
Global Conditional
if parallel_vector at n then … else …
if
at n then E1 else E2= E1
… true fp-1…b1b0
n
F. Gava SNPD 2003 15
Implementation of MSPML MSPML v 0.05 :
F. Loulergue Library for Ocaml (uses threads and
TCP/IP) October 2003 http://mspml.free.fr
F. Gava SNPD 2003 17
Terms Functional semantics:
can be evaluated sequentially or in parallel Terms :
F. Gava SNPD 2003 18
Values and judgements Values:
Judgement:e v
Term « e » evaluates to value « v »
F. Gava SNPD 2003 21
Informal presentation (1) MSPML programs seen as SPMD programs The parallel machine runs p copies of the
same MSPML program Rules for local reduction + rule for
communication For example at processor i the expression
(mkpar f) is evaluated to (f i)
F. Gava SNPD 2003 22
Informal presentation (2)
Proc. 0 1 2
empty 0,v’
Local computation
Communication environment
get v 1
0,v
request 0 1v’A BIT LATER0,v’’
F. Gava SNPD 2003 23
Informal presentation (3)
Proc. 0 1 2
empty 0,v’0,v’’1,w’2,w’’
Local computation
Communication environment
0,v0,v’1,w
request 2 0Not Ready !
!
F. Gava SNPD 2003 24
Terms and judgments New term: request i j
the processor asks the value stored at processor j during the ith step
Step = each step is ended by a call to get Judgment:
At process i, the term ed with communication
environment Ec is evaluate to e’d with new
communication environment E’c
F. Gava SNPD 2003 27
Conclusion and Future Work Conclusion
Minimally Synchronous Parallel ML:Functional semantics & Deadlock free
Two semantics: High level semantics (programmer’s view) Distributed evaluation (implementation’s view)
Implementation Future Work
MPI Implementation Comparison with BSMLlib Extensions: parallel composition, etc.
Recommended