VL-e PoC:What it is and what it isn’t

Jan Just KeijserVL-e P4 Scaling and Validation Team

TU Delft Grid Meeting, December 11th, 2008

The VL-e vision (aka “Bob's stoomboot”)

What is the VL-e PoC Environment?(from http://poc.vl-e.nl)

The Proof-of-Concept Environment (PoC) is the shared, common environment for e-Science of the Virtual Laboratory for e-Science. In the PoC, the different tools and services used by and provided by the project are available, and bound together in a service oriented approach. The PoC covers three distinct areas:

• A software Distribution, to be installed by any-one in VL-e interested in participating in the PoC

• A PoC Environment, the ensemble of systems that run the current PoC Distribution

• The PoC Central Facilities, those systems running the PoC Distribution that are centrally managed by the P4 Scaling and Validation Programme on behalf of the Project

The VL-e PoC R3 Distribution

VL-e PoC R3 Contents:Scientific Linux 4 32bit

gLite 3.1

Sun Java JDK 1.5.0_16

Plusfsl 4.0 gat 1.8.2 globus-toolkit 4.0.7graphviz 2.18 ibis 1.4 itk 3.4javagat 1.7.1 kepler-1.0.0rc1 lam 7.1.2lucene 2.3.1 MatlabMPI 1.2 Mesa3D 6.4.2modules 3.2.3 mpitb 2.1.73 mricro 1.3.9octave 2.1.73 paraview 3.2.1 swi-prolog 5.6.53R 2.6.2 Rmpi-0.5 sesame-client 2.0.1SRB client 3.4.2 taverna 1.7.1 vlet/vbrowser 0.8.1vtk 4.4.2 & 5.0.4 weka 3.4.12

Contributed packagesmono (C#) 1.2.5

see http://poc.vl-e.nl/distribution/R3contents.html for full list

The VL-e PoC Environment

Application development

NL-Grid production clusterCentral mass-storage facilities

+SURFnetInitial compute

platform

Stable, reliable, testedCert. releases Grid MW & VL-

software

VL-e Proof of Concept EnvironmentVL-e Rapid Prototyping

Environment

DAS-2/3,local resources

VL-e Certification Environment

NL-Grid FabricResearch Cluster

Test & Cert.Grid MW & VL-software

Compatibility

Flexible, test environment

Environments

Usage

Characteristics

Virtual Lab.rapid prototyping

(interactive simulation)

Flexible, ‘unstable’

PoC Release n Release Candidaten+1

Developers environment

is a shared, common environment, where different tools and services are both used and provided by the VL-e community

PoC / BigGrid CTB RPE (DAS-3)

The VL-e PoC Central Facility topology

● Rapid Prototyping Environment (DAS-3) System load is usually low Users have a single userid + home directory Some people say it's not a grid, but a

Cluster of Clusters● Proof-of-Concept Environment

Typically high load (90+ %) Users get access using X509 certificates and are

assigned pool accounts. Pool accounts are unique to each cluster and do not

share home directories, not even within a cluster Group rights are handled using Virtual Organisations

and VOMS No direct user logins allowed No inbound connectivity (listeners) on worker nodes

Differences between RPE and PoC

● Scientific Linux/Centos 4 32bit● Migration to Centos 5 64bit in 2009● Runs gLite grid middleware● Usually Torque/PBS as the batch system● Usually not allowed to run software on the

cluster headnode● Runs PoC software and lots of non-PoC

software● Job allocation is based on 1 job per core● Typically 2GB of RAM per job ● Multicore/MPI jobs not well supported (yet)

Typical PoC cluster characteristics

1. VO Software Group area VO has full control Must be installed per cluster Not supported at all by the P4 team or site

administrators

2. VL-e 'contrib track'• Contributed packages are not part of the regular PoC

release cycle

• Certified only to pass some basic installation tests

• Not supported by the P4 team, but exclusively by the original contributor

• Most sites will install contributed packages, but no guarantees

• See http://poc.vl-e.nl/distribution/contrib/ for details

How to get software to run on the PoC

3. Part of the next PoC release Nirwana/Walhalla/Heaven? Must pass 'contrib track' first Must be stable, reliable and tested software PoC Releases are scheduled every ~9 months NO major updates possible in between (except critical

security fixes)

How to get software to run on the PoC

3. Part of the next PoC release Nirwana/Walhalla/Heaven? Must pass 'contrib track' first Must be stable, reliable and tested software PoC Releases are scheduled every ~9 months NO major updates possible in between (except critical

security fixes)

There are some good points: Eternal glory Will be installed at all VL-e PoC clusters Supported by VL-e P4 Scaling and Validation team

How to get software to run on the PoC

VL-e User Applications seen on thePoC Infrastructure

• Data Intensive Science: DANS (KNAW/KB) Sciamachy (KNMI) eNMR (UU)

• Food Informatics: Bitterbase web service (Unilever)

• Medical Diagnosis & Imaging: fMRI, Vbrowser, Moteur (AMC) JavaGAT (VUmc)

• Bio Informatics

Questions?