Www.eu-eela.org E-science grid facility for Europe and Latin America How to run GILDA tutorials Riccardo Bruno INFN.CT 30.06.2008 – 04.06.2008

www.eu-eela.org

E-science grid facility for Europe and Latin America

How to run GILDA tutorials

Riccardo Bruno INFN.CT

30.06.2008 – 04.06.2008

2www.eu-eela.eu Catania (Italy) , Joint EELA/EGEEIII Tutorial for Trainers, 30.06.2008 – 04.07.2008

GILDA t-Infrastructurehttps://gilda.ct.infn.it

Reasons why people should need a t-Infrastructure:

•Introduce the Grid paradigm to new scientific communities

•Provide expertise on application porting

•Test of new releases of Grid services

•Development on Grid (new services)

•Test of middleware co-existence and interoperability

•Provide support

•Deliver training

•Provide training materials


GILDA Tutorials

• Tutorial for Users

• Tutorial for Site Administrators

• Grid Schools/Tut. for App. developers

• Customized events

• Tutorial for Trainers


Some concepts …• The virtualization

– GILDA uses VMWare to virtualize PCs/Grid Services VMWare – (http://info.vmware.com/)

– An alternative software is: Xen – (http://www.xen.org/)

• GILDA Virtual Services– Several virtual services are ready to use at:

https://gilda.ct.infn.it/VirtualServices.html

– The most important Virtual Service is the virtual User Interface https://gilda.ct.infn.it/GILDAVM/gLite-UI-VM3.0.tar.gz

• The UI PnP (https://gilda.ct.infn.it/UIPnP.html)

– Sometimes it is useful to install a UI on an existitng Linux distro No root privileges are necessary Easy to install and configure

• Grid2Win (http://grid2win.gilda-forge.ct.infn.it/new/index.php)

– Recently GILDA supports/tests a Windows porting of gLite


Tutorial for users …

• Provides mostly Training on Grid middleware

• But GILDA also supports the multi-middleware co-existence: gLite, GTK4, UNICORE, Condor, OMII-UK (OURGrid, GTRS)

• GILDA offers tools developed by supported projects like: Grid2Win, SecureStorage, GSAF, gLibrary/DRI

• The aim of the tutorial is to let student APPRECIATE the service offered by the GRID and let him able to operate on it

We will focus on training for gLite


Tutorial for Users Requisites• Audience Requisites:

– Basic Unix/Linux commands knowledge (command line)– Basic concepts of networking (ssh, scp, etc.)

• Technical Requisites:– A computing room equipped with computers that can be used by students

or a simple class room where students can bring their own laptops – To access to the Grid, 3 possibilities:

Use of a dedicated UI server Use of UIs running on virtual machines Use of the UI PnP on Linux Accounts Use of the GILDA glite-tutor and glite-tutor2 UI servers

– A network connection with the outside Internet bandwidth of at least 8 Mbit/s

– a FastEthernet (10/100 Mbit/s) and/or a Wi-Fi (IEEE802.11b/g802.11b/g at 11/54 Mbit/s) local area network

– a big screen and a video-projector; – a sound system (microphone + amplifier + loud speakers) if the room is

big.

Sites with outside connections smaller than 2 Mbit/s are strongly discouraged to candidate themselves to host grid tutorials.


Requesting tutorial for users 1/3

• From GILDA web site: https://gilda.ct.infn.it





• From GILDA site admin. a email will come containing:– A tarball file containing usercertificates

[location]1/ [location]1_userkey.pem

[location]1_usercert.pem [location]1_userkey.p12 [location]2/

[cityname]2_userkey.pem [location]2_usercert.pem[location]2_userkey.p12

… [location]n/

[location]n_userkey.pem [location]n_usercert.pem

[location]n_userkey.p12

– The account names and passwords associated to students Normally follow simple rules related to the account numbers

ATTENTION!The number of users is limited to max 80 persons

Accounts will be created on GILDA UI servers:

glite-tutor.ct.infn.it

glite-tutor2.ct.infn.it

Specified IP address in the request form will be enabled for incoming connections to both machines

The PASSPHRASE is common to ALL generated user certificates

It is: [location] (all capital letters)

GILDA tutorial certificates normally expire a week after the end of the tutorial.

Users might want to continue to use GILDA have to follow the instructions:

https://gilda.ct.infn.it/users.html


Tutorial for Users: The UI

• A single UI server

• Usage of UIs running on Virtual Machines

The student room

The UI server The Grid

The student room

The Grid


Tutorial for Users: glite-tutor

• GILDA relies on two UI servers:– glite-tutor.ct.infn.it

– glite-tutor2.ct.infn.it

glite-tutor.ct.infn.it

The Grid

The student room

glite-tutor2.ct.infn.itGENIUS

Web portal

Users’ /home/…


Tutorial for Users: Exercises

• Students need to make practice on – GILDA maintains a tarball file containing lot of exercises ready to

be used by students

– It’s a good practice to copy exercises on ALL students home directories.

– The tarball of exercises is available on:https://gilda.ct.infn.it/RPMS/GILDA-TAR-APPLICATIONs.tar.gz

• For each student account just execute:– wget https://gilda.ct.infn.it/RPMS/GILDA-TAR-

APPLICATIONs.tar.gz --no-check certificate– tar xvfz GILDA-TAR-APPLICATIONs.tar.gz

ATTENTION!

Install the tarball file into the $HOME directory

The classic and simplest JDL example among exercises is under:

gLite/Other/hostname.jdl

Other examples come from supported application portings made by the GILDA TeamOhter examples show different gLite middleware capabilities, like:

MPI JobsUse of APIs

Use of well known software packages like: OpenFoam, Matlab, etc.


Tutorial for Users: Testing the UI

• In case the students´ UI is centralized, it is a good practice to make stress tests on the UI in order to discover potential problems.

• GILDA developed several stress tests for WMS and Storage and Cataloguing services.

• Stress tests may discover problems on resource overloading, network latencies, certificates, etc.

• It’s a good practice to perform at least once a stress test over the UI hosting students´ accounts.

ATTENTION!

To request the stress test scripts we kindly invite you to send a email togrid-prod mailing list: grid-prod.ct.infn.it


Tutorial for Users: Other scenarios

• A local RB serving the site

The local UI server The Grid

The student room

The local RB server

The local Grid Portal


Tutorial for users: Student PCs• Two different solutions:

– PCs running Windows Office suite + PDF viewer (Adobe) SCP Client SSH Client

– PCs running Linux X Interface Office suite + PDF viewer (xpdf)

• Suggestions (available free software):– Windows:

MS Office (if licenced) or OpenOffice (http://www.openoffice.org/) SCP: WinSCP (http://winscp.net/eng/docs/introduction) SSH: Putty (http://www.chiark.greenend.org.uk/~sgtatham/putty/) LinuxLike solution with CygWin

– Linux: Office package: OpenOffice (http://www.openoffice.org/)

In this tutorial followin SSH client will be used:


User tutorial: Running the tutorial• Every day ask students to sign a list containing at

least:– Name, Surname, Institute, Country, email

• Tutors will describe theory first• For practice sesssions Tutors may choice among:

– Use of dedicated slides to explain exercises– Use of the GILDA Wiki Pages (recommended):

https://grid.ct.infn.it/twiki/bin/view/GILDA/WebHome

– During practice sessions ALL participating tutors will support students requesting help

• At the end of the tutorial don’t forget to ask students to fill the Feedback Form! (see later)

• At the end of the tutorial provide the Training Task Leader with the list of students and the feedback values.


Tutorial for site administrators ..…

• Provides mostly Training on Grid Middleware

• GILDA actually supports the multimiddleware co-existence: gLite, GTK4, UNICORE, Condor, OMII-UK (OURGrid, GTRS)

• The aim of this tutorials is to explain how to INSTALL and MAINTAIN grid sites.

We will focus on training for gLite


Tutorial for S.Adm: Requisites• Audience Requisites:

– Advanced Unix/Linux commands knowledge (command line)– Unix/Linux administration knowledge– Good knowledge of networking and relative services (DNS, Firewall, etc.)

• Technical Requisites:– A network connection having outside Internet bandwidth of at least 8 Mbit/s – a FastEth. (10/100 Mbit/s) and/or a WiFi (IEEE802.11b/g - 11/54 Mbit/s) LAN – A computing room equipped with desktops that can be used by students or a

simple class room where students can bring their own laptops – At least one machine (either real or virtual, with Xen or Vmware) each 2

users. Machines must have direct and inverse resolution, so either : - Machines with public IP, direct/reverse solved by public DNS (or) - Machines with private IP solved by a local DNS ("pointed" by students)

– Real machines should have at least 1 GB of memory per processor (dual processor machines are preferred) and at least 40 GByte of internal hard-disk

– The machines must be at the disposal of tutors at least 1 week before the tutorial – A big screen and a video-projector (a 2nd video-projector for the machine to

install would be better); – A sound system (microphone + amplifier + loud speakers) if the room is big.

Sites with local network slower than 100 Mbit/s are strongly discouraged to candidate themselves to host grid tutorials.


Tutorial for S.Adm: Request

• The requesting procedure is the same as for users …A further step is necessary after the form submission.It is necessary to send a list of hostnames to the:

GILDA-CA mailing list ([email protected])This list of hostnames will be used to generate GILDA HOST certificates.

ATTENTION!The number of users is limited to max 30 persons

Specified IP addresses in the request form will be enabled for incoming connections to both GILDA UI servers

GILDA host certificates for tutorials normally expire after the end of the tutorial.Users might want to continue to use GILDA and have to follow this procedure:

https://gilda.ct.infn.it/sites.html

Requested GILDA host certificates have one year of validity


Tutorial for S.Adm: Preparation• Dedicate a PC hosting a web server and make it accessible by

students

• Create a mirror of the INFN Grid software repository (instructions will follow on this)

– http://igrelease.forge.cnaf.infn.it/doku.php?id=doc:tips:mrepo

• Create a directory seen by the web server and copy on it:

– GILDA Host certificates

– GILDA User certificates– The java JDK 1.5.0 RPM (Java Development Kit 5.0 Update XX)

– The tarball of GILDA exercises

– A directory containing all slides (sometime indico is slow or unreachable or just contain last minute slides not yet uploaded into the Agenda).

– VM Image with BASIC installation (SL4 + pre-requisites)


Tutorial for S.Adm: Scenario

• A Dedicated PC for each service

• Usage of Virtual Machines

The student room

Any Grid ServiceThe student room


Tutorial for S.Adm: Final Exercise

• A key point in tutorial for site administration:– Each student installs a service storing the VM image file

– Let installed machine cooperate as a Local grid site ( )

The student room


Tutorial for S.Adm : Run the Tut.• Every day ask students to sign a list containing at least:

– Name, Surname, Institute, Country, email

• Tutors will describe theory first, then the installation of the grid service will follow

• Students will install the service in a dedicated and separated slot in the Agenda.

• Students may also have a look on GILDA wiki pages:

– https://grid.ct.infn.it/twiki/bin/view/GILDA/GILDASiteInstall – During practice sessions ALL participating tutors will support students

requesting help

• At the end of the tutorial don’t forget to ask students to fill the Feedback Form! (see later)

• At the end of the tutorial provide the Training Task Leader with the list of students and the feedback values.


Grid schools …

• Aims to support students in porting their own applications

• Requires a big effort in terms of human and physical resources

• A survey of potential application must be done in advance

• A selection of posted application must be done in accordance of well defined selection and evaluation criteria


Grid Schools: Requisites• Audience Requisites:

– Researchers having User tutorial requisites– The Application responsible must partecipate to the GridSchool

• Technical Requisites:– A network connection having outside Internet bandwidth of at least 8 Mbit/s – a FastEth. (10/100 Mbit/s) and/or a WiFi (IEEE802.11b/g - 11/54 Mbit/s) LAN – A computing room equipped with desktops that can be used by students or a

simple class room where students can bring their own laptops – At least 10 real or virtual (with Xen or Vmware) machines. These machines

must have public IP addresses and must be registered on a local DNS with direct and inverse name resolution.

– Real machines should have at least 1 GB main memory per processor (dual processor machines are preferred) and at least 40 GByte of internal hard-disk

– The machines must be installed by tutors at least 1 week before the tutorial – A big screen and a video-projector (a 2nd video-projector for the machine to

install would be better); – A sound system (microphone + amplifier + loud speakers) if the room is big.– Appropriate Surveillance, Fire estinguishing systems, Cooling fans and

Electricity power must be granted during the school

Sites with outside connections smaller than 2 Mbit/s are strongly discouraged to candidate themselves to host grid schools.

Sites with local network slower than 100 Mbit/s are strongly discouraged to candidate themselves to host grid tutorials.


Grid Schools: Request• There is a standardized way to organize a GridSchool,

like tutorials for users and site admin tutorials

• Normally Grid Schools are planned at the beginning of a project due to their huge request of resources

• Planning are normally done on TB Meetings or during Workshops.

• In order to collect applications a survey must be submitted to the community

• The list of potential applications must be provided to NA2 and the TB in order to be evaluated

ATTENTION!The number of application is limited to max 15

The number of days shouldn’t be less than 10 (working days)

Applications will be installed on GILDA or other Organizations (EELA-2)


Grid Schools: Preparation• A local grid site must be installed by tutors one week

before the school

• Schools requires 2 rooms

– One room dedicated to Grid Services

– One room dedicated to Students

• Computer room should be deployed far from the student room in order to reduce heating and noise

• Student room can be arranged in different fashions (U/O-shape, classroom shape, etc.)

• Student PCs must be installed exaclty like in a user tutorial


Grid Schools: Scenario

Computers roomThe student room

The Local Grid


Grid Schools: ROC on Duty• Normally Grid Schools are a very good opportunity to

train site managers to the ROC on Duty tutorial

• The Regional Operation Center (ROC) has the responsability for providing the Grid Services. This responsibility includes:

– Middleware certification– Middleware deployment– Day to day operations – User support

• This tutorial AIMS to explains the ROC activities and Policies adopted by the GILDA’ supported project.


Customized events …

• Sometime a generic tutorial does not fit training requests coming from the hosting institute

• In such case GILDA has to provide a focused training:

– Examples: Training focused on HPC and MPI Jobs Application developers days Training on GILDA services: SecureStorage,

GSAF, gLibrary/DRI…


Customized events: Requisites

• Due to the unhespected nature of the target topic to cover; requisites may be very different

• A good starting point maight be to refer to:

– Same requisites of: Tutorial for Users

– Same requisites of: Tutorial for SiteAdministrators


Customized events: Request

• There is a standardized way to organize a Customized event, like tutorials for users and site admin tutorials

• Normally Customized events are planned during the lifetime of the supported project

• Planning are normally done on TB Meetings or during Workshops.

• Customized events normally need more time to be organized due to the unhespected nature of the target topic


Training for Trainers …

• GILDA currently supports many projects: EELA-2, EGEEIII, COMETA, etc.

• Projects actually cover many continents: Europe, Asia, Latin America, etc.

• Training means costs for the project, then the need of: Local Tutors.

• GILDA ever encourages the creation of local team of tutors

• Training for Trainers is the opportunity to become a GILDA Local Tutor.

• The aim is to form Local Tutors


Training for Trainers: Requisites• Audience Requisites:

– Same requisites of Site Administration Tutorial– Knowledge of gLite or a previous participation to any GILDA Tutorial

• Technical Requisites:– A network connection having outside Internet bandwidth of at least 8 Mbit/s

– a FastEth. (10/100 Mbit/s) and/or a WiFi (IEEE802.11b/g - 11/54 Mbit/s) LAN

– A computing room equipped with desktops that can be used by students or a simple class room where students can bring their own laptops

• At least 10 real or virtual (with Xen or Vmware) machines. These machines must have public IP addresses and must be registered on a local DNS with direct and inverse name resolution – Real machines should have at least 1 GB of memory per processor (dual

processor machines are preferred) and at least 40 GByte of internal hard-disk

– The machines must be at the disposal of tutors at least 1 week before the tutorial

– A big screen and a video-projector (a 2nd video-projector for the machine to install would be better);

– A sound system (microphone + amplifier + loud speakers) if the room is big

Sites with outside connections smaller than 2 Mbit/s are strongly discouraged to candidate themselves to host grid schools.


Training for Trainers : Request

• There is a standardized way to organize a Training for Trainers event, like tutorials for users and site admin tutorials

• Normally Training for Trainers events are planned at the beginning of the supported project

• Planning are normally refined on TB Meetings or during Workshops.


Common operations

Following actions are common for any kind of Training Event …


Common: The Agenda

• The training event need to be published;• GILDA uses CERN’ Indico Agenda System

– EELA-2 Project: (http://indico.eu-eela.org/) – EGEEIII Project: (http://indico.cern.ch/categoryDisplay.py?categId=41)

ATTENTIONFor EELA-2 project the preferred timetable default style is: Indico Style

Please do not forget to ask Tutors to provide slides, Then upload them into the Agenda


Common: The feedback form

• The training event need a Feedback Form;

– Why the feedback form: Improve the quality of training material Improve the quality of training Obtain suggesions from students Report the quality of training to the funding institute

• The FF just collects student feedbacks– Normally one feedback per Talk– 6 degree of choice

Poor, Insufficient, Fair, Good, Very Good, Excellent

– 1 comment field

ATTENTION!

There is no actually a official request form for the Feedback FormTo request a FeedbackForm just send a email:

TO: [email protected] CC: [email protected]


Common: Images

• Few images of GILDA tutorials

User Tutorial

Grid SchoolStudent Room

Grid SchoolComputer Room

ATTENTION!

Please do not forget to bring your camera!

Photos are very useful for many cases like:

Creating project bookletsDesigning Web sites

Creating document reports….


Questions …

Documents

Www.eu-eela.org E-science grid facility for Europe and Latin America How to run GILDA tutorials Riccardo Bruno INFN.CT 30.06.2008 – 04.06.2008