Alice Workshop, Aug 21, 20081 Recent experiences in Grid- enabling legacy software codes Dana Petcu [email protected] petcu Prof

Alice Workshop, Aug 21, 2008

1

Recent experiences in Grid-enabling legacy software codes

Dana Petcu

[email protected]://web.info.uvt.ro/~petcu

Prof. & Dir. of Computer Science Dep., West Univ. of Timisoara, www.uvt.roRes. & Dir. of research Institute e-Austria Timisoara, Romania, www.ieat.ro


2

Content

Context Romanian sites & Grid initiatives UVT & IeAT Grid-related activities Grid – where to?

Cases studies of Grid-enabling existing codes For Computational Grids (CFDs, DLs etc) For Service Grids (EOs, CASs etc)

Conclusions


3

RO sites in EGEE-II / Feb 08


4

RO sites in EGEE-III / 20 Aug 08

Locations (SEE-Grid monitoring):


5

UVT in SEE-Grid-Sci / 20 Aug 08


6

Current infrastructureUVT-CS Department’s cluster IeAT’s cluster

(28 CPUs, gLite 3) (30 CPUs, GT 4)


7

Recent history – UVT & IeAT team 2002

UVT’s cluster of 8 IBM PCs with Myrinet connection (2Gps) included in ASC facilities (NSF’s Astrophysics Simulation Collaboration)

2003 Test platform between UVT and RISC-Austria

2004 First prototypes of Grid-enabled codes (CFD + CAS) in

CompGrid project 2005

Started national collaborations MedioGrid (imag.proces.) http://mediogrid.utcluj.ro GridMOSI (optimization) http://www.gridmosi.ro


8

Recent history – UVT & IeAT team 2006:

Involved in FP6 SCIEnce – SymGrid (symb. Comp) http://www.symbolic-computation.org

Involved in national SINRED prj. Digital lib. & Grids Started Grid training and dissemination activities (ISPDC06 symp., GridCAD ws, GridTDT training, PhD programme)

2007: Participation at FP6 EGEE-II and SEE-Grid-II Accepted in RoGrid consortium Participation at national prj. PEGAF (workflow &Grid services)

2008: Participation at FP7 EGEE-3 and SEE-Grid-Sci Leading the ESA’s project: GiSHEO http://gisheo.info.uvt.ro


9

Application prototypes in the last year gLite oriented applications

DEMO/G – Distributed Evolutionary Multiobjective Optimization on Grid

GOC – portal for gLite jobs

GT4 oriented applications SymGrid-Services – Web and Grid services for

symbolic computations components of MedioGrid platform – Web and

Grid services for satellite image processing


10

Back to the international context (1)

European Commission programs: FP6 funds:

Infrastructure (e.g. EGEE-II) IST – Grid Unit F2

FP7 funds: E-Infrastructure (e.g. EGEE-3) ICT – Service Oriented

Architectures (Grid Unit -> Service-Oriented Archs. Unit)

The business pressure companies promoting Web

services & SOA E.g. Aug’08 GridToday newsletter -> On-demand Entreprise Daily news EU projects promoting SOA in

FP7-ICT coming from enterprise-driven consortiums like NESSI


11

Back to the international context (2) NSF’s Cyberinfrastructure Vision – March 07

Grid infrastructure = world-wide Cyber-utility populated by self-* cooperating services which interact as in a complex gigantic software ecosystem

European NGG3 expert group – Jan 2006 Grid services + Knowledge technologies = SOKU

Services are knowledge-assisted + Architecture comprises services + Utility: immediately useable service

Different Grids: Computational Grids Service Grids Instrument Grids …

New-related buzzwords: cloud computing, on demand computing (ODC) Service oriented computing (SOC) Service oriented infrastructures (SOI) Software as a Service (SaaS) …


12

Legacy codes & Grids

Legacy system = a mission-critical software system developed sometime in the past that has been around and has changed for a long time without undergoing systematic remedial actions

Modernization techniques are applicable to legacy systems:1. Redevelopment, from scratch, using a new hardware platform and modern

architectures tools and database; 2. Migration that moves the system to a new platform, while retaining the

original system data and functionalities3. Wrapping, which surrounds existing data, programs, applications and

interfaces with new interfaces

Grid-related solutions Migration towards Grids in order to increase their ability to solve bigger

problems by (“gridification” for Computational Grids) Wrapping as Grid services is a cost-effective and quality solution for evolving

them in order to meet new requirements (“wrapping” for Service Grids)


13

Gridification for Computational Grids1. Identify parallelism degree

2. Manage the issue related to the resources heterogeneity

3. Deal with intra-cluster and inter-cluster communications

4. Create interfaces for easy-to-use


14

Example for 1. Parallelism degree Embarrassing gridification in analogy with

embarassing parallelism – no communication Digital libraries [in our case for SINRED project, developments on

RoGrid infrastructure based on gLite] Multimedia conversions for large collections of

documents (e.g. PDF -> audio) – considerable requirements in what concerns response time & storage

Applying OCR techniques on large collections of documents in order to create indexes (keywords, references etc), extract images etc

Used available free convertors


15

Example for 2. Manage heterogeneity Stage 1 – 2003 parallel version of STHAMAS 3D

commercial CFD simulator for crystal growing

three-dimensional finite-volume code

based on a block-structured finite-volume Navier-Stokes solver Stage 2 – 2004 (CompGrid project)

adapt the parallel code to use heterogeneous computing resources improved efficiency through avoiding load balancing problems


16

Example for 3. Intra/Inter-cluster comm.DEMO/G (in GridMOSI project)Distributed Evolutionary Multiobjective

Optimization on Grid= implementation of evolutionary algorithms

for multiobjective optimization (EMOA) based on a layered structure First layer: sequential EMOA executed on

one processor Second layer: parallel EMOA based on a

colony of inter-communicating populations

Third layer: multiple colonies which evolves separately

Used for multiobjective multidisciplinary optimization (case study: airfoil design) Evaluation of objective functions is based

on numerical simulations; coupling between EMOA and other codes

e.g. CFD


17

Example for 4. Create interfaces easy-to-use

GOC Grid Operations Center for gLite 3 Modular Approach

Authentication Application

interface Demo/G,

STHAMAS, OCR etc

Grid Engine

Application Interface / DEMO/G


18

Wrapping codes for Service Grids Non-interactive legacy systems1. Example 1: Service for CFD

simulation

Interactive legacy systems*1. Example 2: Service for image

processing2. Example 3: Services for

symbolic computations3. Example 4: Service for an

expert tool

*systems where any transaction is completed by means of a dialogue between the user and the system


19

Example 1: service for CFD simulation Legacy code: GERRIS

open source free software library for the solution of the PDEs describing fluid flow

Inputs: a file describing the simulation

parameters; the number of processes used by

Gerris; the name(s) of the resulting file(s)

to be transferred to the client side.

Similar experiences with other codes E.g. for optimization DEME - library

for sequential, parallel & distributed metaheuristic algorithms

INPUT – SIMULATION PARAMETERS:

# The classical lid-driven cavity test case.

1 0 GfsSimulation GfsBox GfsGEdge {} {

Time { end = 300 } Refine 6

SourceDiffusion {} U 1e-3

SourceDiffusion {} V 1e-3

EventStop { istep = 10 } U 1e-4 DU

OutputScalarNorm { istep = 10 } du { v = DU }

OutputPPM { start = end } { v = Velocity }

{ convert -colors 256 ppm:- velocity.eps } }

GfsBox {

top = Boundary { BcDirichlet U 1 BcDirichlet V 0 }

bottom = Boundary { BcDirichlet U 0 BcDirichlet V 0 }

right = Boundary { BcDirichlet U 0 BcDirichlet V 0 }

left = Boundary { BcDirichlet U 0 BcDirichlet V 0 } }

OUTPUT:


20

Example 1: service for CFD simulationThe service exposes: Simple client interface:

setType(type) Set the execution type: use gerris2D, sequential (type=1) or in parallel (type=2), or gerris3D, sequential (type=3) or in parallel (type=4); by default type=1

exec(simFileName) Launch the simulation described in the simFile

getExample(type) Get an example of simulation file

getHelp() Get a user manual for problem description

getStatus() Get the status (Active, Done or Error) of the simulation; by default the status is Done


21

Example 2: Service for image processing A Grid service for remote image processing based on GIMP

GIMP accept command lines! despite its classical interactive mode

A distributed DB of images is available to a virtual organization (VO) as a shared facility.

A user apply a sequence of image processing standard procedures to extract a particular information from specific images

Usage in MedioGrid platform

Follows: ESA project GiSHEO! Combining ESA’s G-POD with MedioGrid platform services for educational purposes

The service exposes:

createServInstance <FactoryRef> <EPRFile>

startGIMPServer <EPRFile>

<DatabaseRef>

clientGIMPAction <EPRFile>

<GIMPcommand>

clientGetFile <EPRFile>

<FileName> <Destination>

Simple client interface


22

Example 3: Services for Symbolic Computations Computer Algebra Systems

(CAS) are wrapped Create an instance for each

client Gradually interpret

commands Examples CAS: GAP, Kant,

MuPad, Maple, Yacas, Singular, Pari, CoCoA, Macaulay etc

Allows interoperability between CASs

Only one part of SymGrid- (EU SCIEnce project) – another part: access Grid/Web/gLite services within CAS


23

Example 4: an expert tool for ODEs EpODE (1997) @ UVT EpODE (2008) @PEGAF prj.


24

Conclusions

The process of Grid-enabling legacy codes depends at least the followings: Grid type (e.g. Computational/Service etc) Code type (e.g. Interactive/Non-interactive, GUI oriented/or

not etc) Despite there are several existing tools for Grid-

enabling codes, those are useful for particular conditions, not for the general case

An automatized process for Grid-enabling any legacy code is far for being possible at this moment

Documents

Alice Workshop, Aug 21, 20081 Recent experiences in Grid- enabling legacy software codes Dana Petcu [email protected] petcu Prof