FP6−2004−Infrastructures−6-SSA-026409

www.eu-eela.org

E-infrastructure shared between Europe and Latin America

LFC Server Installation and Configuration

Tony CalanducciINFN CataniaFirst EELA Grid tutorial for users and system administrators Madrid, 20-24th February 2006

Madrid, EELA Kick-off Meeting, 30.01.2006 2FP6−2004−Infrastructures−6-SSA-026409

E-infrastructure shared between Europe and Latin America

Outline

• Introduction to LCG File Catalogue (LFC)

• How to install a LFC Server

• LFC References

Madrid, EELA Kick-off Meeting, 30.01.2006 3FP6−2004−Infrastructures−6-SSA-026409

E-infrastructure shared between Europe and Latin America

• User and programs produce and require data • Data may be stored in Grid datasets (files)

– Located in Storage Elements (SEs)– Several replicas of one file in different sites– Accessible by Grid users and applications from “anywhere”– Locatable by the WMS (data requirements in JDL)– Data may be copied from/to local filesystems (WNs, UIs) to the

Grid

• Why File Catalogs are used?– They keep track of the location of copies (replicas) of Grid

files

– They provide DM tools and APIs through which users and grid services can deal with files

File Catalog Motivations

Madrid, EELA Kick-off Meeting, 30.01.2006 4FP6−2004−Infrastructures−6-SSA-026409

E-infrastructure shared between Europe and Latin America

• Logical File Name (LFN) – An alias created by a user to refer to some item of data, e.g.

“lfn:cms/20030203/run2/track1”

• Globally Unique Identifier (GUID) – A non-human-readable unique identifier for an item of data, e.g.

“guid:f81d4fae-7dec-11d0-a765-00a0c91e6bf6”

• Site URL (SURL) (or Physical File Name (PFN) or Site FN)– The location of an actual piece of data on a storage system, e.g.

“srm://pcrd24.cern.ch/flatfiles/cms/output10_1” (SRM) “sfn://lxshare0209.cern.ch/data/alice/ntuples.dat” (Classic SE)

• Transport URL (TURL)– Temporary locator of a replica + access protocol: understood by a SE, e.g.

“rfio://lxshare0209.cern.ch//data/alice/ntuples.dat”

Name conventions

Madrid, EELA Kick-off Meeting, 30.01.2006 5FP6−2004−Infrastructures−6-SSA-026409

E-infrastructure shared between Europe and Latin America

• LFC features– Hierarchical namespace

– Integrated GSI Authentication + Authorization

– Access Control Lists (Unix Permissions and POSIX ACLs)

• LFN acts as main key in the database. It has:– Symbolic links to it

– Unique Identifier (GUID)

– System metadata

– Information on replicas

– One field of user metadata

The LFC

Madrid, EELA Kick-off Meeting, 30.01.2006 6FP6−2004−Infrastructures−6-SSA-026409

E-infrastructure shared between Europe and Latin America

LFC Server Installation

Madrid, EELA Kick-off Meeting, 30.01.2006 7FP6−2004−Infrastructures−6-SSA-026409

E-infrastructure shared between Europe and Latin America

Installation Pre-requisites

• Start from a fresh install of SLC 3.0.x• Verify that apt is installed and properly configured:

• rpm -qa | grep apt• Install apt if necessary:

• rpm -ivh http://linuxsoft.cern.ch/cern/slc30X/i386/SL/RPMS/apt-0.5.15cnc6-8.SL.cern.i386.rpm

• Java SDK is required to install almost any LCG/gLite grid service. Due to license issues, it is not found in the LCG repositories. So please download and install a release of Sun Java SDK >= 1.4.2_08 from http://java.sun.com/j2se/1.4.2/download.html

chmod +x j2sdk-1_4_2_10-linux-i586-rpm.bin./j2sdk-1_4_2_10-linux-i586-rpm.binrpm -ivh j2sdk-1_4_2_10-linux-i586.rpmPreparing... ########################################### [100%] 1:j2sdk ########################################### [100%]

Madrid, EELA Kick-off Meeting, 30.01.2006 8FP6−2004−Infrastructures−6-SSA-026409

E-infrastructure shared between Europe and Latin America

Security pre-requisites

• Request a X509 host certificate for the BDII Server you are going to install from a Certification Autority (CA). For example, to get host certificates from the GILDA CA, go to:– https://gilda.ct.infn.it/CA/mgt/restricted/srvreq.php

• Install host certificates (hostcert.pem and hostkey.pem) in /etc/grid-security.– chmod 644 hostcert.pem– chmod 400 hostkey.pem

• If you plan to use certificates released by unsupported EGEE CA’s, be sure that their public key, signing policy and CRLs (usually distributed with an rpm) are installed in /etc/grid-security/certificates.– For the VO GILDA, the RPM is available from

https://gilda.ct.infn.it/RPMS/ca_GILDA-0.28-1.i386.rpm

Madrid, EELA Kick-off Meeting, 30.01.2006 9FP6−2004−Infrastructures−6-SSA-026409

E-infrastructure shared between Europe and Latin America

Node Syncronization with NTP

• Syncronization among all LCG nodes is mandatory. It can be achieved by the NTP protocol with a time server

• Install ntp if not already available for your system:– apt-get install ntp

• Add your time server in /etc/ntp.conf– restrict <time_server_IP_address> mask 255.255.255.255 nomodify notrap

noquery – server <time_server_name> – (you can use ntp-1.infn.it – IP 193.206.144.10)

• Edit /etc/ntp/step-tickers adding your(s) time server(s) hostname

• If you are running a firewall, you will have to allow inbound comminication on the NTP port:-A INPUT -s <NTP-serverIP-1> -p udp --dport 123 -j ACCEPT

• Activate the ntpd service with the following commands: ntpdate <your ntp server name> service ntpd start chkconfig ntpd on

– You can check ntpd’s status with:

ntpq -p

Madrid, EELA Kick-off Meeting, 30.01.2006 10

FP6−2004−Infrastructures−6-SSA-026409

E-infrastructure shared between Europe and Latin America

YAIM Installation

• YAIM is a set of bash scripts used to install and easily configure the LCG middleware on the target machine

• Ensure you FQDN is properly set– hostname –f

• Find the latest YAIM version at http://www.cern.ch/grid-deployment/gis/yaim/

• Download it using wget – wget http://www.cern.ch/grid-deployment/gis/yaim/lcg-yaim-2.7.0-3.noarch.rpm

• Install using– rpm -ivh lcg-yaim-2.7.0-3.noarch.rpm

Madrid, EELA Kick-off Meeting, 30.01.2006 11

FP6−2004−Infrastructures−6-SSA-026409

E-infrastructure shared between Europe and Latin America

Site Configuration File

• All the configuration values to sites have to be configured in a site configuration file using key-value pairs.

• This file is shared among all the different LCG node types. So edit once and keep it in a safe place

• Create a copy of /opt/lcg/yaim/examples/site-info.def template (coming from the lcg-yaim RPM) to your reference directory for the installation (e.g. /root):– cp /opt/lcg/yaim/examples/site-info.def /root/my-site-info.def

• The general syntax of the file is a sequence of bash-like assignments of variables (<variable>=<value>, no spaces are allowed around =).

• A good syntax test for your site configuration file is to try to source it manually running the command:– source my-site-info.def

Madrid, EELA Kick-off Meeting, 30.01.2006 12

FP6−2004−Infrastructures−6-SSA-026409

E-infrastructure shared between Europe and Latin America

LFC Server related variables

• $MY_DOMAIN -> your domain (ciemat.es) • $LFC_HOST -> the hostname where the LFC will be

installed (gil07.ciemat.es)• $LFC_DB_PASSWORD -> the Mysql password for the

"lfc" user • $MYSQL_PASSWORD -> the root Mysql password • $LFC_CENTRAL -> list of VOs for which the LFC

should be configured as a central catalogue. If you leave this empty, the LFC will be set as a local catalog.

• $LFC_LOCAL -> If you want to limit the VOs for your local catalog, add them here.

Madrid, EELA Kick-off Meeting, 30.01.2006 13

FP6−2004−Infrastructures−6-SSA-026409

E-infrastructure shared between Europe and Latin America

Support for GILDA VO

VOS="dteam gilda“ add here the VOs you want to support

VO_GILDA_SW_DIR=$VO_SW_DIR/gildaVO_GILDA_DEFAULT_SE=$SE_HOSTVO_GILDA_STORAGE_DIR=$CE_CLOSE_SE1_ACCESS_POINT/gildaVO_GILDA_QUEUES="short long infinite"VO_GILDA_VOMS_SERVERS="vomss://voms.ct.infn.it:8443/voms/gilda?/gilda"

Add gilda poolaccount in /opt/lcg/yaim/examples/users.conf according the following format:UID:LOGIN:GID:GROUP:VO:FLAG:

example:…– 4401:gilda001:4400:gilda:gilda::– 4402:gilda002:4400:gilda:gilda::…

Add the following lines to /opt/lcg/yaim/examples/groups.conf– "/VO=gilda/GROUP=/gilda/ROLE=SoftwareManager":::sgm:– "/VO=gilda/GROUP=/gilda"::::

Madrid, EELA Kick-off Meeting, 30.01.2006 14

FP6−2004−Infrastructures−6-SSA-026409

E-infrastructure shared between Europe and Latin America

Installation and configuration scripts

• Start the download and installation of all the needed package using the following command:– /opt/lcg/yaim/scripts/install_node /root/my-site-info.def lcg-

LFC_mysql

• Start the configuration of the LFC according the information provided into the my-site-info.def with:– /opt/lcg/yaim/scripts/configure_node /root/my-site-info.def

LFC_mysql

Madrid, EELA Kick-off Meeting, 30.01.2006 15

FP6−2004−Infrastructures−6-SSA-026409

E-infrastructure shared between Europe and Latin America

LFC Server Testing

• Log into a LCG user interface where the LFC clients commands and lcg-utils are installed:– rpm -qa | grep LFC-client– rpm -qa | grep lcg_util

• Set the hostname of the LFC catalog you want to use:– export LFC_HOST=gil07.ciemat.es

• Be sure to have already initialized a voms proxy: voms-proxy-info

• Give a look inside the /grid/gilda folder with:– lfc-ls -l /grid/gilda

• Try to create a subdirectory with:– lfc-mkdir /grid/gilda/tcaland

Madrid, EELA Kick-off Meeting, 30.01.2006 16

FP6−2004−Infrastructures−6-SSA-026409

E-infrastructure shared between Europe and Latin America

LFC Server Testing (II)

• Check that LCG_GFAL_INFOSYS variable is correctly set up to your BDII server:– export LCG_GFAL_INFOSYS=gil03.ciemat.es:2170

• Use lcg-infosites to discover which SE are available to your VO:– lcg-infosites --vo gilda se

• Choose one SE and upload a file with:– lcg-cr -v --vo gilda -d <fadq of the choose SE> -l

lfn:/grid/gilda/tcaland/test.jdl file://$PWD/hostname.jdl

• Check if the file was correctly registered into the catalog and where it was stored:– lfc-ls -l /grid/gilda/tcaland– lcg-lr lfn:/grid/gilda/tcaland/hostname.jdl

Madrid, EELA Kick-off Meeting, 30.01.2006 17

FP6−2004−Infrastructures−6-SSA-026409

E-infrastructure shared between Europe and Latin AmericaReferences

• LCG File Catalog (LFC) administrators' guide– https://uimon.cern.ch/twiki/bin/view/LCG/LfcAdminGuide

• LFC Troubleshooting– https://uimon.cern.ch/twiki/bin/view/LCG/LfcTroubleshooting