Embed Size (px)
DESCRIPTION
Introduction
Citation preview
Semantics of Minimally Synchronous Parallel ML
Myrto Arapini, Frédéric Loulergue, Frédéric Gava and
Frédéric DabrowskiLACL, Paris, France
F. Gava SNPD 2003 2
Outline Introduction Pure functional minimally
synchronous parallel programming High level semantics Distributed evaluation Conclusions and future work
Introduction
F. Gava SNPD 2003 4
Our previous work:Bulk Synchronous Parallelism + Functional Programming = BSML Bulk Synchronous Parallelism :
Scalability Portability Simple cost model
Functional Programming : High level features (higher order
functions, pattern matching, concrete types, etc.)
Programs proofs Safety of the environment
F. Gava SNPD 2003 5
What is MSPML ?Why MSPML ? = Minimally Synchronous Parallel ML BSML without synchronization barriers
Comparison of efficiency BSML/MSPML Further extensions restricted or impossible
in BSML (divide-and-conquer, nesting of parallel values)
Basis for an additional level to BSML for Grid computing: BSML on each nodes (clusters) + MSPML for coordination
F. Gava SNPD 2003 7
The MSPML Library Parallel ML library for the
Objective Caml language operations on a parallel data
structureAbtract type: par
access to the machine parameters:p: unit int
p() = number of processes
F. Gava SNPD 2003 8
Creation of Parallel Vectors mkpar: (int )par
(mkpar f )f (p-1)…(f 1)(f 0)
F. Gava SNPD 2003 9
Pointwise Parallel Application apply: ( ) par par par
apply
=
fp-1…f1f0
vp-1…v1v0
fp-1 vp-1…f1 v1f0 v0
F. Gava SNPD 2003 10
Example (1) let replicate x = mkpar(fun pid->x)replicate: ’a -> ’a par
(replicate 5)
5…55
F. Gava SNPD 2003 11
Example (2) let parfun f v = apply (replicate f) vparfun: (’a->’b)->’a par->’b par
parfun (fun x->x+1) parallel_vectorf
vp-1…v1v0
f…ffapply
= (f vp-1 )…(f v1 )(f v0 )
F. Gava SNPD 2003 12
Communication Operation: get
get: parint par par get
vp-1…v1v0
ip-1…i1i0(=1)
vip-1…vi1
vi0
=
F. Gava SNPD 2003 13
Example (3)let bcast_direct root parallel_vector =if not(within_bounds root)then raise Bcast_Error else get parallel_vector
(replicate root)bcast_direct: int->’a par->’a par
vp-1…v1v0
vroot…vrootvroot
(bcast_direct root
=
)
F. Gava SNPD 2003 14
Global Conditionalif parallel_vector at n then … else
…if
at n then E1 else E2= E1
… true fp-1…b1b0
n
F. Gava SNPD 2003 15
Implementation of MSPML MSPML v 0.05 :
F. Loulergue Library for Ocaml (uses threads and
TCP/IP) October 2003 http://mspml.free.fr
High level semantics
F. Gava SNPD 2003 17
Terms Functional semantics:
can be evaluated sequentially or in parallel Terms :
F. Gava SNPD 2003 18
Values and judgements Values:
Judgement:e v
Term « e » evaluates to value « v »
F. Gava SNPD 2003 19
Some rules
Distributed evaluation
F. Gava SNPD 2003 21
Informal presentation (1) MSPML programs seen as SPMD programs The parallel machine runs p copies of the
same MSPML program Rules for local reduction + rule for
communication For example at processor i the expression
(mkpar f) is evaluated to (f i)
F. Gava SNPD 2003 22
Informal presentation (2)Proc. 0 1 2
empty 0,v’
Local computation
Communication environment
get v 1
0,v
request 0 1v’A BIT LATER0,v’’
F. Gava SNPD 2003 23
Informal presentation (3)Proc. 0 1 2
empty 0,v’0,v’’1,w’2,w’’
Local computation
Communication environment
0,v0,v’1,w
request 2 0Not Ready !
!
F. Gava SNPD 2003 24
Terms and judgments New term: request i j
the processor asks the value stored at processor j during the ith step
Step = each step is ended by a call to get Judgment:
At process i, the term ed with communication environment Ec is evaluate to e’d with new communication environment E’c
F. Gava SNPD 2003 25
Local computation rules
F. Gava SNPD 2003 26
Communication rule
F. Gava SNPD 2003 27
Conclusion and Future Work Conclusion
Minimally Synchronous Parallel ML:Functional semantics & Deadlock free
Two semantics: High level semantics (programmer’s view) Distributed evaluation (implementation’s view)
Implementation Future Work
MPI Implementation Comparison with BSMLlib Extensions: parallel composition, etc.
