Prograamção Paralela Baseada em Componentes (Introduzido o Modelo #)

Parallel Programming Parallel Programming with Componentswith Components

Prof. Dr. Francisco Heron de Carvalho JuniorProf. Dr. Francisco Heron de Carvalho Junior([email protected])([email protected])

Mestrado e Doutorado Mestrado e Doutorado em Ciência da Computaçãoem Ciência da Computação

Departamento de ComputaçãoDepartamento de Computação

Universidade Federal do CearáUniversidade Federal do Ceará

Presentation PlanPresentation Plan

• What are components ?What are components ?

• Components in high performance computingComponents in high performance computing

• Parallelism support in components infrastructuresParallelism support in components infrastructures

• The # component model and # programming systemsThe # component model and # programming systems

• ConclusionsConclusions

Software ComponentsSoftware Components

• What is a What is a software componentsoftware component ? ?

““A software components is a unit of composition with A software components is a unit of composition with contractually specified interfaces and explicit context contractually specified interfaces and explicit context

dependencies only. A software components can be dependencies only. A software components can be deployed independently and is subject to composition deployed independently and is subject to composition

by third parties” by third parties” (workshop on Component-Oriented Programming at ECOOP 1996)(workshop on Component-Oriented Programming at ECOOP 1996)..

Component TechnologyComponent Technology

• A A component infrastructurecomponent infrastructure is defined by is defined by

– a component modela component model

• What are the components, concretely, and how they are What are the components, concretely, and how they are

created;created;

– A connection modelA connection model

• What kind of connectors are supported for composing What kind of connectors are supported for composing

components in a high-level component or an application;components in a high-level component or an application;

– a deployment modela deployment model

• How a component is mapped to an execution platform How a component is mapped to an execution platform

(architecture), how it is launched with an application, and how (architecture), how it is launched with an application, and how

it coexists with other components in an environment, and so on.it coexists with other components in an environment, and so on.

Component TechnologyComponent Technology• Components Infrastructures:Components Infrastructures:

– Java Beans / EJB;Java Beans / EJB;

– COM / DCOM;COM / DCOM;

– CORBA;CORBA;

– .NET;.NET;

– OSGi (OSGi (Open Services Gateway InitiativeOpen Services Gateway Initiative))

– Web ServicesWeb Services;;

• Commercial applications;Commercial applications;

• Service interfaces (method signatures); Service interfaces (method signatures);

• A very natural paradigm for distributed computing;A very natural paradigm for distributed computing;

– Remote method calls (client-server relationships);Remote method calls (client-server relationships);

• Parallelism support is not explicit;Parallelism support is not explicit;

• High overhead for using in HPC (lots of indirections !) .High overhead for using in HPC (lots of indirections !) .

Components and HPCComponents and HPC

• Component models and frameworks have been developed for the Component models and frameworks have been developed for the specific needs of HPC applications;specific needs of HPC applications;

• CCA (CCA (Common Component ArchitectureCommon Component Architecture))– Inspiration in commercial component models (CORBA/COM);Inspiration in commercial component models (CORBA/COM);

– SIDL (Scientific Interface Definition Language);SIDL (Scientific Interface Definition Language);

– Client/server interfaces with method signatures;Client/server interfaces with method signatures;• Uses/Provides design pattern;Uses/Provides design pattern;

– Direct method calls whenever possible;Direct method calls whenever possible;

• FractalFractal– Hierarchical composition (primitive Hierarchical composition (primitive vsvs composite components); composite components);

– Comunicação assíncrona (Comunicação assíncrona (futuresfutures););

– ASP calculus (semantics);ASP calculus (semantics);

– Reflective facilities through controllers;Reflective facilities through controllers;

– ADL for describing configurations;ADL for describing configurations;

– Fractal has been used as the basis for GCM (Grid Component Model);Fractal has been used as the basis for GCM (Grid Component Model);

ProActive

Julia

CCAffeine

XCAT

MOCCA

SciRun2

LegionCCA

DCA

Parallelism and ComponentsParallelism and Components

• What is a parallel program (distributed case) ?What is a parallel program (distributed case) ?– A set of programs interacting through some A set of programs interacting through some

synchronization mechanismsynchronization mechanism to solve a problem that to solve a problem that could be solved by a sequential program;could be solved by a sequential program;

– WhyWhy ? To minimize “time to solution”; ? To minimize “time to solution”;

– HowHow ? Exploiting concurrency inherent to the problem ? Exploiting concurrency inherent to the problem and/or solution algorithms;and/or solution algorithms;

– Synchronization mechanism(s) must support Peer-to-Synchronization mechanism(s) must support Peer-to-Peer and Client-Server relations;Peer and Client-Server relations;

• message-passing or channel-based synchronization;message-passing or channel-based synchronization;

MPMD SPMD


• Parallelism encapsulated inside components;Parallelism encapsulated inside components;

– Parallelism in CORBA (PARDIS, PaCo++, GridCCM, Data Parallel Corba);Parallelism in CORBA (PARDIS, PaCo++, GridCCM, Data Parallel Corba);

• Data parallelism between a set of cooperating components;Data parallelism between a set of cooperating components;

• SCMD (SCMD (Single Component, Multiple DataSingle Component, Multiple Data))

– a a cohortcohort of components perform the same task over each part of a data structure; of components perform the same task over each part of a data structure;

– Components interact using any synchronization mechanism;Components interact using any synchronization mechanism;

– CCA “CCA “stays out of the way with components parallelism”stays out of the way with components parallelism”;;

– CCA community now looks for MCMD extensions;CCA community now looks for MCMD extensions;

• Extensions to client-server connectors Extensions to client-server connectors

– Supporting several clients and several several;Supporting several clients and several several;

– Distinguished cases: 1Distinguished cases: 1M, NM, N1, M1, MN;N;

– Fractal/ProActive: Fractal/ProActive: multicast / gathercastmulticast / gathercast interfaces; interfaces;

– Is this sufficient expressive compared to MPI ?Is this sufficient expressive compared to MPI ?


• The “weakness” of current component models for parallelism:The “weakness” of current component models for parallelism:

– Parallelism concerns orthogonal to the component and connection models; Parallelism concerns orthogonal to the component and connection models;

– Support for a fixed (normally unitary) set of connectors, emphasizing Support for a fixed (normally unitary) set of connectors, emphasizing

client/server relations;client/server relations;

• Useful for commercial applications (distributed systems);Useful for commercial applications (distributed systems);

• CCA and Fractal follow this approach;CCA and Fractal follow this approach;

– ““(...) stay of the way with component parallelism (...)”(...) stay of the way with component parallelism (...)”

– Further works looks for more expressive forms of parallelism (???)Further works looks for more expressive forms of parallelism (???)

– If not purely SCMD, common modularization principles are broken !If not purely SCMD, common modularization principles are broken !

• For example: parallel concerns scattered accross a set of components;For example: parallel concerns scattered accross a set of components;

Parallel programs involve a dynamically Parallel programs involve a dynamically

evolving set of connectors, most of them evolving set of connectors, most of them

emphasizing emphasizing peer-to-peer relationspeer-to-peer relations;;


• Trends in Trends in software architecturesoftware architecture

– Focus on connectors;Focus on connectors;

– Exogenous coordination;Exogenous coordination;

– Composing new connectors by combination of existing ones;Composing new connectors by combination of existing ones;

• Algebras of connectors: Reo, CommUnity, ...Algebras of connectors: Reo, CommUnity, ...

• A model of A model of parallel componentsparallel components needs to support ... needs to support ...

– ... The ability to encapsulate parallel concerns.... The ability to encapsulate parallel concerns.

– ... several primitive connectors for parallel synchronization;... several primitive connectors for parallel synchronization;

– ... user-defined (programmable) connectors;... user-defined (programmable) connectors;

– ... tuning implementation of connectors and components for architectures;... tuning implementation of connectors and components for architectures;

• Kinds of connectors appropriate for specific architectures;Kinds of connectors appropriate for specific architectures;

– Different Memory and parallelism hierarchies ???Different Memory and parallelism hierarchies ???

A component defined by its relations with other components in the environment

The # Component ModelThe # Component Model

• A notion of parallel component called #-component;A notion of parallel component called #-component;

• A #-component is like an usual component deployed in a set of A #-component is like an usual component deployed in a set of

computing nodes of a parallel architecturecomputing nodes of a parallel architecture

• A #-component address a (parallel) concern;A #-component address a (parallel) concern;

– Given a parallel program P, composed by N processes, that uses Given a parallel program P, composed by N processes, that uses

a #-component C in its composition, each unit a #-component C in its composition, each unit uuii, 1 , 1 << ii N, denote the role N, denote the role

of a process of P with respect to a given of a process of P with respect to a given concernconcern;;

– Bottom-upBottom-up perspective: we say that perspective: we say that unitsunits are are slices of processesslices of processes after after

slicing the slicing the parallel programparallel program by using the by using the concernconcern addressed by the addressed by the #-#-

componentcomponent as a as a slicing criterionslicing criterion;;

...units

#-component

P1 P2 P3 P4

u1

u2

v1

v2

v3

v4

t1

t2t3

The model abstracts away from a concrete

notion of role

1. Send a message to another process;

2. Receive a message from another process;

3. Calculate the contribution of a process in the parallel solution of a linear system

4. Informs the processor where the process will execute,

5. Informs the processor about execution time parameteres;

6. A CCA uses port;7. A CCA provides port;8. etc

#-components may be used to address non-functiona concerns !!!


• #-components may be composed by overlapping composition#-components may be composed by overlapping composition

... ...

... ...

joining

... ...

......



... ... ...

... ......

unification

... ...

... ...


fusion


......

......

...

...

......

...

The # Compoent ModelThe # Compoent Model

• Exogenous coordination:Exogenous coordination:

a b c

d

efg

h

par { seq {a; b; c}; alt {d; seq {e; f}; seq {g; h} } }

protocol

OpenMP directives

#omp paralell sections { a(); b(); c();} #omp section{ if (<condition>) { d(); } else if (<condition>) { e(); f(); } else { g(); h(); }}

Petri net

A

B

C

D

E

F

H

Ga1d1

h2

b2

f3

a3

d2b1

c1

e2

f2

g1

h1

e1

f1

C = {·, A, B, C, D, E, F, G, H}

a1 f3a3

d1 h2b2 d2b1 c1

e2 f2 e1 f1 h1g1

g

g

ggg

ggg

gg

gg

gg

unidade unidade unidade

the root is a distinguishednode, representing the #-component, whose subtrees denote each one of its units.

g

A

d1 d2 b1 b2 f3 c1 h2 d1 d2

e2 f2e1 f1 g1 h1

b1 f3 c1 d2

e1 f1 g1 h1b2

e2 f2

d1 h2

a1

c1

f3

b1 d2

g1 h1e1 f1

b2

e2 f2

d1 h2

a1 a2

b1

f3

c1 d2

f1g1 eh1

b2

eh2 f2

d1

a1 a2

D

A AA

A

(2) t º join {t1,t2,t3}

(3) t' º unify t {b2,h2,d1} (5) t’’’ º fuse t’’ {E,H}

(1) Three #-components to be overlapped

(4) t'' º unify t’ {b1,c1,d2}

t3

t

t’ t’’ t’’’Figure ??

c1 h2

g1 h1

Ct2b1 b2 f3

e2 f2e1 f1

Bt1r(e1)=r(h1)

r(e2)=r(h2)

requisite for fusion

Example of steps in the overlapping composition of three components

Component KindsComponent Kinds

• What defines a # Programming System ???What defines a # Programming System ???

– A A programming platformprogramming platform that materializes a notion of #-component in that materializes a notion of #-component in

terms of common terms of common software abstractionssoftware abstractions;;

– It defines a set of It defines a set of component kindscomponent kinds;;

• A A component kindcomponent kind includes a set of #-components that includes a set of #-components that

– are concretely described in terms of the same software abstractions;are concretely described in terms of the same software abstractions;

– have the same have the same restrictionsrestrictions for for overlapping compositionoverlapping composition of other of other

#-components according to their respective #-components according to their respective component kindscomponent kinds;;

– have the same have the same deployment modelsdeployment models in a platform; in a platform;

• ConnectorsConnectors may be treated as may be treated as kindskinds of #-components; of #-components;

Component KindsComponent Kinds

• The set of The set of Fractal componentsFractal components might form a component kind in a # might form a component kind in a #

programming system;programming system;

• The sets of The sets of CCA componentsCCA components, , portsports, and , and bindingsbindings each one might form a each one might form a

component kind in a # programming system;component kind in a # programming system;

• The set of connectors for The set of connectors for collective communicationcollective communication might form a component might form a component

kind in a # programming system;kind in a # programming system;

• The subroutines of a linear algebra parallel library, such as PETSc and The subroutines of a linear algebra parallel library, such as PETSc and

ScaLAPACK could be thought in terms of component kinds;ScaLAPACK could be thought in terms of component kinds;

• (...)(...)

Could an extensible # programming system be used to integrate different component infrastructures and frameworks, also including parallel synchronization connectors that are not found in most of them ?

topic to investigate:

Developers

Developers

ComponentsComponents

CoreCore

Bac

k-En

d

Bac

k-En

dFront-End

Front-End

Exe

cutio

n P

latf

orm

s

Exe

cutio

n P

latf

orm

s

Front-End

Core

Back-End

discovery

configuration

registration

retrieving

deploymentrunningmonitoring

HASH FRAMEWORK

ARCHITECTURE

LocationsLocationsCoreCore

Front-EndFront-End

Back-EndBack-End

LocationsLocationsCoreCore

Front-EndFront-End

Back-EndBack-End

HPE (The Hash Programming Environment)HPE (The Hash Programming Environment)

• It is a general purpose parallel programming platformIt is a general purpose parallel programming platform

– ... not a computational science framework !... not a computational science framework !

– ... not a problem solving environment !... not a problem solving environment !

• It is a # programming systemIt is a # programming system

– Materialization of a notion of #-component by specifying a set of component kinds Materialization of a notion of #-component by specifying a set of component kinds

appropriate to the application domain;appropriate to the application domain;

• It complies to the Hash Framework Architecture;It complies to the Hash Framework Architecture;

– Front-EndFront-End: Visual Configuratin Editor on the Eclipse Platform (GEF/EMF);: Visual Configuratin Editor on the Eclipse Platform (GEF/EMF);

– CoreCore: a Web-Service that acess a set of registered locations: a Web-Service that acess a set of registered locations

• LocationsLocations are also Web Services. are also Web Services.

– Back-EndBack-End::

• Clusters of multi-processors;Clusters of multi-processors;

• Extension of Mono/CLI (next slide);Extension of Mono/CLI (next slide);

• Forró (Forró (probably the first, and still unique, Brazilian CCA frameworkprobably the first, and still unique, Brazilian CCA framework) for distributed ) for distributed

parallel computations (workspaces, linked bindings, iterators, etc).parallel computations (workspaces, linked bindings, iterators, etc).

HPE (The Hash Programming Environment)HPE (The Hash Programming Environment)

#-components

Architectures

Environments

Computations Data Structures

Synchronizers

Applications

Qualifiers

The Back-End of HPEThe Back-End of HPE• Extending the Mono/CLI platform;Extending the Mono/CLI platform;

• Why Mono/CLI ?Why Mono/CLI ?– Language interoperability through a common type system;Language interoperability through a common type system;

– Support for native arrays;Support for native arrays;

– Improvements in just-in-time compilation (virtual machines performance);Improvements in just-in-time compilation (virtual machines performance);

– Support for unmanaged code;Support for unmanaged code;

– Side-by-side control of component versions;Side-by-side control of component versions;

• DGAC (Distributed GAC)DGAC (Distributed GAC)– GAC = GAC = Global Assembly CacheGlobal Assembly Cache;;

– Manage Manage deploymentdeployment of #-components; of #-components;

• The definition of a #-component dependes on its The definition of a #-component dependes on its component kindcomponent kind;;– ComputationComputation and and SynchronizationSynchronization: units are classes implementing the methods : units are classes implementing the methods

activate activate and and synchronizesynchronize, respectively;, respectively; – Data StructureData Structure: units are simple objects or abstract data types;: units are simple objects or abstract data types;– ApplicationApplication: units are static classes implementing the method : units are static classes implementing the method main. main. TheyThey

instantiate all slices of the process that they implement;instantiate all slices of the process that they implement;– ArchitectureArchitecture, , EnvironmentEnvironment, and , and QualifiersQualifiers: XML configuration files;: XML configuration files;

• Non-functional informationNon-functional information

Node 1 Node 2 Node 3 Node N

NETWORK

...Mono Mono Mono Mono

Hash-Mono Hash-Mono Hash-Mono Hash-Mono

Mono API Mono API Mono API Mono API

The Back-End of HPEThe Back-End of HPE

• Back-EndBack-End

cluster

Distributed Hash-Mono

Abstract ComponentsAbstract Components

• From the From the Front-EndFront-End, programmers may build , programmers may build

– abstract componentsabstract components specifications, and specifications, and

– concrete componentsconcrete components implementations that conform to a given abstract component; implementations that conform to a given abstract component;

• A A concrete componentconcrete component is an implementation of an abstract component that is optimized is an implementation of an abstract component that is optimized

to a specific execution platform;to a specific execution platform;

• A A concrete componentconcrete component is directly deployed in the architecture; is directly deployed in the architecture;

– The system knows the abstract component of each concrete component;The system knows the abstract component of each concrete component;

• Registered in the DGAC;Registered in the DGAC;

– Only Only oneone concrete component may be deployed for each abstract component; concrete component may be deployed for each abstract component;

– Several versions of a concrete component may exist in the environment;Several versions of a concrete component may exist in the environment;

• GAC/Mono support;GAC/Mono support;

• When an When an abstract componentabstract component CC built from inner abstract components built from inner abstract components BB11, , BB22, ..., , ..., BBnn is is

deployed, the system will look for deployed #-components deployed, the system will look for deployed #-components bb11, , bb22, ... , ... bbnn and compose them and compose them

to form to form CC using the configuration specification. using the configuration specification.

– Deployment procedure.Deployment procedure.

Domain Specific # Programming SystemsDomain Specific # Programming Systemsusing the using the Hash ArchitectureHash Architecture or or HPE FrameworkHPE Framework

• Define appropriate Define appropriate component kindscomponent kinds for the for the application domainapplication domain;;

– What What abstractionsabstractions scientists and engineers need to compose applications of their interest ??? scientists and engineers need to compose applications of their interest ???

– Computational Science FrameworksComputational Science Frameworks and and Problem Solving EnvironmentsProblem Solving Environments;;

• ForróForró: Components, Ports, Workspaces, Locations, Bindings, and Links !: Components, Ports, Workspaces, Locations, Bindings, and Links !

– Support for distributed paralel computations in HPE;Support for distributed paralel computations in HPE;

– Parallel progams running in distinct clusters accessed through the same Back-End;Parallel progams running in distinct clusters accessed through the same Back-End;

– Investigate interoperability between CCA frameworks ???Investigate interoperability between CCA frameworks ???

• A component with bindings in different CCA frameworks encapsulated by A component with bindings in different CCA frameworks encapsulated by disjoint workspacesdisjoint workspaces;;

• MPhyScasMPhyScas: Kernels, Groups, Blocks, Phenomena, Quantities, States ?: Kernels, Groups, Blocks, Phenomena, Quantities, States ?

– Front-EndFront-End for building simulations ? for building simulations ?

– Abstract componentsAbstract components for supporting implementations for different archictetural characteristics ? for supporting implementations for different archictetural characteristics ?

• Application to build Application to build Problem Solving EnvironmentsProblem Solving Environments ?? ??

– Towards the higher level of abstraction for scientists and engineers ....Towards the higher level of abstraction for scientists and engineers ....

ConclusionsConclusions

• Parallelism support in component models and frameworks Parallelism support in component models and frameworks

is still an evolving area of research;is still an evolving area of research;

• We think that focusing on appropriate connectors it is We think that focusing on appropriate connectors it is

possible to address paralelism in component models;possible to address paralelism in component models;

– See our CompFrame’2007 paper for examples of useful connectors See our CompFrame’2007 paper for examples of useful connectors

and its use with abstract components;and its use with abstract components;

• We have presented how we have tried to apply our ideas:We have presented how we have tried to apply our ideas:

– The # Component Model;The # Component Model;

– The Hash Architecture;The Hash Architecture;

– The Hash Programming Environment;The Hash Programming Environment;

– Integration to the Forró CCA Framework;Integration to the Forró CCA Framework;

Parallel Programming Parallel Programming with Componentswith Components

Prof. Dr. Francisco Heron de Carvalho JuniorProf. Dr. Francisco Heron de Carvalho Junior([email protected])([email protected])

Mestrado e Doutorado Mestrado e Doutorado em Ciência da Computaçãoem Ciência da Computação

Departamento de ComputaçãoDepartamento de Computação

Universidade Federal do CearáUniversidade Federal do Ceará