The use of Cluster and parallelism to support the ...€¦ · Workshop Open Modeller - Cluster...

Workshop Open Modeller - Cluster

The use of Cluster and parallelism to support the execution of openModeller.

Workshop Open Modeller - Cluster

� Agenda:

− LAHPC;

− Cluster;

− Activities;

− Experiments;

− Results;

− Future Work;

Workshop Open Modeller - Cluster

� Agenda:

− LAHPC;

− Cluster;

− Activities;

− Experiments;

− Results;

− Future Work;

Workshop Open Modeller - Cluster

� LAHPC (Laboratory of Architecture and High Performance Computing)

− Tools for parallel and distributed programming, respectively to shared memory machine and homogeneous and heterogeneous clusters,

− Performance analysis to parallel and distributed programs,

− Tools to parallel database,

− Memory management techniques,

− Parallel and distributed architectures;

− Grid computing.

Workshop Open Modeller - Cluster

� Agenda:

− LAHPC;

− Cluster;

− Activities;

− Experiments;

− Results;

− Future Work;

Workshop Open Modeller - Cluster

� Cluster Hardware:

− 11 nodes: 2 quad intel Xeon 2.0 GHz and 8 GB ram memory;

− Network interconnection: Gigabit and Infiniband;

− Local File System and Network File System.

� Cluster Software:

− Operational System: SUSE Linux Enterprise 10, 64 bits, XFS;

− Compilers: C, C++ and Fortran;

− MPI: LAM, Voltaire, Intel;

− Condor job scheduler;

− Ganglia cluster monitor;

− Apache Server;

− Tomcat;

Workshop Open Modeller - Cluster

Workshop Open Modeller - Cluster

Workshop Open Modeller - Cluster

Workshop Open Modeller - Cluster

Workshop Open Modeller - Cluster

� Agenda:

− LAHPC;

− Cluster;

− Activities;

− Experiment;

− Results;

− Future Work;

Workshop Open Modeller - Cluster

� Improve the performance of openModeller

− Parallelism;

− Cluster (node: multi-core architecture);

− Infiniband;

� Enable the use of cluster power processing on openModeller Desktop;

Workshop Open Modeller - Cluster

� Condor:

− Specialized workload management system for compute-intensive jobs.

� LAM-MPI:

− LAM (Local Area Multicomputer) an open source implementation of the Message Passing Interface (MPI) standard. The MPI standard is the de facto industry standard for parallel applications.

� SOA (Service Oriented Architecture):

− Architectural model to enable the integration of loosed coupling systems.

Workshop Open Modeller - Cluster

� Integration with openModeller desktop using web services;

� openModeller execution as a job:− openModeller Library build on Front Node (NFS);− Layers replicated in all the nodes;− Condor support;

� openModeller parallelization:− openModeller components parallelization using LAM;− Condor support;

Workshop Open Modeller - Cluster

� Agenda:

− LAHPC;

− Cluster;

− Activities;

− Experiment;

− Results;

− Future Work;

Workshop Open Modeller – Cluster

Workshop Open Modeller – Cluster

Workshop Open Modeller – Cluster

Workshop Open Modeller – Cluster

Workshop Open Modeller – Cluster

Workshop Open Modeller - Cluster

� Experiment:− Species: Furcata boliviana

− Layers:� WorldClim (bioclimáticos);

− Output Mask: /application/scratch/layers/WORLDCLIM/bioclimaticos/bio_10min/bio_1

− Algorithm: BioClim

− Format: GreyTiff100

− Average time 1 min 55 segundos. (serial)

− Average time 55 segundos. (parallel) 10 nodes;

Workshop Open Modeller - Cluster

� Agenda:

− LAHPC;

− Cluster;

− Activities;

− Experiment;

− Results;

− Future Work;

Workshop Open Modeller - Cluster

� Experiments:

− Furcata Boliviana:

� Algorithms:

− Bioclim

− DG GARP

− GARP

− Envelope Score

− Support Vector Machines

− Environmental Distance

− Environmental Distance (Chebyshev)�

− Environmental Distance (Mahalanobys)�

� Executions:

− From 1 to 15 process running 5 times each.

Workshop Open Modeller - Cluster

• Average Execution Time (om_project):

1 3 4 5 6 7 8 9 10 11 12 13 14 15

0.00

2.00

4.00

6.00

8.00

10.00

12.00

14.00

16.00

18.00

20.00

BioclimDG GARPGARPEnvelope ScoreSupport Vector M achinesEnvironmental DistanceEnvironmental Distance (Chebyshev)Environmental Distance (M ahalanobys)

Workshop Open Modeller - Cluster

� Discussion:

− Degradation level;

− Network communication;

− Map Generation;

− Layer file reading.

Workshop Open Modeller - Cluster

� Agenda:

− LAHPC;

− Cluster;

− Activities;

− Experiment;

− Results;

− Future Work;

Workshop Open Modeller - Cluster

� Future Work:

− Performance evaluations;

− Port new algorithms to openModeller: AdaptGarp, Pgarp, ... ;

− Implement an authorization infra-structure based on credentials;

− Enforcement of web services invocation by known users identified by credentials;

Workshop Open Modeller - Cluster

� Questions ?