UCY HPCL

Introduction to the CrossGrid Testbed

George TsouloupasUCY HPCL

UCY HPCL

– What is the “CrossGrid testbed” ?● A collection of distributed computing

resources● Supporting a “Grid environment”

– Objectives● Development, Testing and validation● Emphasis on interoperability

with EU-DataGrid (EDG)• Extension of GRID across Europe

UCY HPCL

Testbed Overview● A collection of distributed

computing resources– 16 sites (small & large) in 9 countries,

connected through Géant + NRN● 115 CPUs (Worker Nodes)● 4 TB (Storage Elements)

– + Grid Services: Middleware (based on Globus) RB, VO, RC…

● A group of people and resources make up the CrossGrid Virtual Organization (VO) and trust each other through national Certification Authorities.

UCY HPCL

Testbed resources

C o m3

ID C

Worker Nodes GatekeeperStorageElement

UserInterface

ConfigurationMachine(LCFG)

C o m3

ID C

Worker Nodes GatekeeperStorageElement

UserInterface

ConfigurationMachine(LCFG)

C o m3

ID C

Worker Nodes GatekeeperStorageElement

UserInterface

ConfigurationMachine(LCFG)

C o m3

ID C

Worker Nodes GatekeeperStorageElement

UserInterface

ConfigurationMachine(LCFG)

C o m3

ID C

Monitoring

ResourceBroker

ReplicaCatalogue

VirtualOrganization

D a ta G e n er al

MyProxyServer

latigid

Network

UCY HPCL

The Local Testbed We currently have: ● 1 CE (apelatis.grid.ucy.ac.cy) 2xPIII 1.2GHz 1GB● 1 SE (se001.grid.ucy.ac.cy) 2xPIII 1GB 40GB● 1 UI (thales.grid.ucy.ac.cy) P4 2GHz 1GB 40GB

● 4 WN (wn001-004.grid.ucy.ac.cy) P4 2GHz 1GB 40GB

● CA

● LCFG Installation server

UCY HPCL

local testbed

CURRENT local testbed status

CE

apelatis2xPIII 1.2GHz

1GB RAM18GB scsi100Mbit

SE

2xPIII 800MHz1GB RAM128GB scsi

WNWNWNWN

P4 2GHz1GB RAM

40GB 1000Mbit

CA

(certificate generation)

UI

thalesP4 2GHz

1GB RAM40GB scsi1000Mbit

LCFG

(installation server)

nfs

nfs

Submit

UCY HPCL

Grid Security● Security is a primary concern when it comes to Grids.● The Grid Security Infrastructure is based on Public Key

Infrastructure.● A user is authenticated based on possession of a private

key (known only to the user).● Authorization depends on whether the user's certificate

is signed by an accepted Certification Authority (In our case the CyGrid CA)

● The gridmap-file at the target resource will determine the if a resource will accept a specific user, and specifies a mapping to a local user

UCY HPCL

Crossgrid CA's

UCY HPCL

Obtaining a certificate1)Create a certificate request file:

Under Globus use command: grid-cert-request Or using openssl command:   

openssl genrsa -des3 -out user.key;  openssl req -new -x509 -days 365 -key user.key -out user-req.crt

DN Format :    person:   "/C=CY/O=CyGrid/O=HPCL/CN=user name"   machine:   "/C=CY/O=CyGrid/O=HPCL/CN=host/machinename.ucy.ac.cy"

2)Send by email your request file to: cygrid-ca@ucy.ac.cy and your RA: hpcl@ucy.ac.cy

DO NOT delete either the key file or the request file. It is the sole copy of your private key, Most importantly: keep it secure!

3)You will receive your certificate which is good for one year.4)Install your certificate on the User Interface machine.

UCY HPCL

Job Submission

A machine installed as a

EDG UI

A machine with globus

client

CE

CE

CEResource Broker

CrossGrid/DatagridServices

ex: dg-job-submitex: globus-job-submit

UCY HPCL

Running a Job● User logs in on the UI● User issues a grid-proxy-init and enters his certificate's password, getting a valid Globus proxy● User sets up his JDL file.

Example: JDL of the “hello world” job Executable = "/bin/echo";

Arguments = "Hello World";

StdOutput = "Messagge.txt";

StdError = "stderr.log";

OutputSandbox = {"Message.txt","stderr.log"};

UCY HPCL

Running a job● User issues a: dg-job-submit HelloWorld.jdl

and gets back from the system a unique Job Identifier (JobId)

● User issues a: dg-job-status <JobId>

to get logging information about the current status of his Job

● When the “OutputReady” status is reached, the user can issue a dg-job-get-output JobId

and the system returns the name of the temporary directory where the job output can be found on the UI machine.

UCY HPCL

Introduction to LCFG

George TsouloupasUCY HPCL

UCY HPCL

What is LCFG?

● Local ConFiGuration system

● A tool for installation and configuration of large scale, diverse, dynamically changing systems.

● Installs an RPM based Linux system and configures it.● Config of entire site held in source files on central server● Each machine defined by exactly one profile.● “Profile” constructed for each machine

– Utilise inheritance– Based on RPM lists, key-value pairs

UCY HPCL

LCFG Server Components

Server XML machine profiles.Web Server

Serves RPM repository.NFS Server

Manages the rpm repository and machine profiles.

Software (LCFG)

Provides network information.DHCP Server

UCY HPCL

How LCFG Works in One Slide.

A collection of agents read configuration parameters and either generate traditional config files or directly manipulate various services

Abstract configuration parameters for all nodes stored in a central repository

ldxprof

LoadProfile

Generic

Component

ProfileObject

rdxprof

ReadProfile

LCFG Objects

Local cache

Client nodes

Web Server

HTTP

XML Profile

LCFG Config Files

Make XMLProfile

Server

+inet.services telnet login ftp

+inet.allow telnet login ftp sshd

+inet.allow_telnet ALLOWED_NETWORKS

+inet.allow_login ALLOWED_NETWORKS

+inet.allow_ftp ALLOWED_NETWORKS

+inet.allow_sshd ALL

+inet.daemon_sshd yes

.....

+auth.users myckey

+auth.userhome_mickey /home/mickey

+auth.usershell_mickey /bin/tcsh

Config files

<inet>

<allow cfg:template="allow_$ tag_$ daemon_$">

<allow_RECORD cfg:name="telnet">

<allow>192.168., 192.135.30.</allow>

</allow_RECORD>

.....

</auth>

<user_RECORD cfg:name="mickey">

<userhome>/home/MickeyMouseHome</userhome>

<usershell>/bin/tcsh</usershell>

</user_RECORD>

XML profiles

ProfileObject

inet auth

/etc/services

/etc/inetd.conf

/etc/hosts.allow

in.telnetd : 192.168., 192.135.30.

in.rlogind : 192.168., 192.135.30.

in.ftpd : 192.168., 192.135.30.

sshd : ALL

/etc/shadow

/etc/group

/etc/passwd

....

mickey:x:999:20::/home/Mickey:/bin/tcsh

....

UCY HPCL

LCFG Installation

● Custom install Red Hat 6.2 including– NFS, web server

● Install LCFG RPMs and directory tree● Configure web server● Populate RPM repositories● Make root filesystem available over NFS● Configure DHCP server● Create client profiles

UCY HPCL

Profiles

● A profile contains a list of RPMs to be installed and configuration data

● Located in /var/obj/conf/profile/source.● Edit site-cfg.h => globus.conf , info-mds.conf, gdmp.conf and others.

● Files testbed001-testbed004 are sample UI, CE, SE and WN. Rename one, e.g. apelatis.

● Edit apelatis; hostname and hardware.#include "AIC_ComputingElement-cfg.h"...+update.modlist eepro100+update.mod_eepro100 alias eth0 eepro100+update.rootpwd hj%4aJkf\k45

UCY HPCL

Profiles (cont)

●Contents of "AIC_ComputingElement-cfg.h"

#include "ComputingElement-cfg.h"

+update.rpmcfg CE_AIC7xxx-rpm

+update.disks sda+update.bootdisk sda+update.dopartition_sda yes+update.partitions_sda sda2 sda1 sda3+update.pdetails_sda1 64 /boot+update.pdetails_sda2 free /+update.pdetails_sda3 1000 swap

UCY HPCL

Questions ?

...then more examples on prometheus.grid.ucy.ac.cy

(our LCFG server)