IST-2006-026409 www.eu-eela.org

E-infrastructure shared between Europe and Latin America

Hands on session: the AMGA Metadata Catalogue

Domenico Vicinanza, CERN

EELA Tutorial, Santiago, September 2006

Location, Meeting, dd.mm.yyyy 2IST-2006-026409

E-infrastructure shared between Europe and Latin America

www.eu-eela.org

• Login to the GRID authenticating with VOMS extension:

$ voms-proxy-init –voms gilda• Start up the Metadata Catalog Client with

$ mdclient

Connected to amga.ct.infn.it:8822ARDA Metadata Server 1.2.0Query>

• Once logged in, you can list the available commands, typing help. • Commands are grouped by topic. You can get the list of valid

commands for each topic, typing: help [topic]• Example: help entry

(If mdclient is not working, copy a template of config file for the MDC:$ cp $GLITE_LOCATION/etc/mdclient.config \ $HOME/.mdclient.config)

Access the Metadata Catalog

Location, Meeting, dd.mm.yyyy 3IST-2006-026409

E-infrastructure shared between Europe and Latin America

www.eu-eela.org

• Valid topics:– help – metadata – metadata-optional – directory – replication – entry – group – acl – index – schema – sequence – user – view – ticket – commands

AMGA: List of topics (HELP)

Location, Meeting, dd.mm.yyyy 4IST-2006-026409

E-infrastructure shared between Europe and Latin America

www.eu-eela.org

>> createdir <path> [options] Make a new folder. It can inherit the schema assosciated to the upper level folder>> rm pattern Remove items corresponding to the given pattern>> link <file> Make a link to another file or to a external URL>> dir <directory> List the content of a directory>> listentries <directory> List the items (not the collections) of a directory>> stat <filepattern> Show the statistic information about a directory>> chown <file> <owner> Changhe the ownership of a file or a directory>> chmod <file> <rights> Change the access rights to a file or a directory>> rmdir <directory> Remove a directory>> dump <directory> Make a recursive dump startung from a given directory, (the default is: ‘/’)

mdclient: generic commands

Location, Meeting, dd.mm.yyyy 5IST-2006-026409

E-infrastructure shared between Europe and Latin America

www.eu-eela.org

• Browse the contents of a directory– dir [path]Returns the name of all subdirectories and files in the given path or in

the current directory if not specified• Print the current working directory

– pwd• Change the current working directory

– cd directoryExample: cd /gilda/santiago

• Directory creation– createdir /parentdir/dirCreates the directory dir if it does not yet exist but parentdir

already doesExample: createdir /gilda/santiago/vicinanza

• Directory removal– rmdir pathRemoves the (empty) directory given by path

MDC directory related commands

Location, Meeting, dd.mm.yyyy 6IST-2006-026409

E-infrastructure shared between Europe and Latin America

www.eu-eela.org

• Schema population– addattr dir attr typeAdds a new attribute to the schema of a directory. Type is the name of an SQL

datatype which will translated (if necessary) into a data type understood by the back end DB.

Examples of valid datatypes are int, float, varchar(n), timestamp, text, numeric(p,s)

Examples: addattr /gilda/santiago/vicinanza/ MovieTitle varchar(100)addattr /gilda/santiago/vicinanza/ Runtime intaddattr /gilda/santiago/vicinanza/ PlotOutline text

• Attribute listing– listattr pathReturns a list of all attributes of the given file/direcory

• Attribute Removal– removeattr dir attributeRemoves an attribute from a directory if it is not used by any entry in the directory

MDC: Handling attributes

Location, Meeting, dd.mm.yyyy 7IST-2006-026409

E-infrastructure shared between Europe and Latin America

www.eu-eela.org

• Entry creation– addentry entry (attribute value)+Add a new entry and initializes some attributesExample: addentry /gilda/santiago/vicinanza/madagascar.mov

MovieTitle Madagascar• Setting attribute values

– setattr entry (attribute value)+Sets one or more attributes of an entry to given valuesExample: setattr /gilda/…/madagascar.mov Runtime 86

• Getting attribute values– getattr pattern (attribute)+Returns the entries and all the attributes for every file matching patternExample: getattr /gilda/…/vicinanza/*.mov Title

• Entry deletion– rm patternRemoves all entries matching patternExample: rm /gilda/…/m*.mov

MDC: managing entries

Location, Meeting, dd.mm.yyyy 8IST-2006-026409

E-infrastructure shared between Europe and Latin America

www.eu-eela.org

– find pattern ‘query_condition’

Returns all entries matching pattern for which query_condition is true

Examples: find /gilda/…/vicinanza/ ‘Runtime > 80’find /gilda/…/vicinanza/ ‘like(MovieTitle, “Mad%”)’

find /gilda/…/vicinanza ‘like(MovieTitle, “Mad%”) AND Runtime > 80’

– selectattr attr... conditionReturns the values of given attributes for all files matching condition

Example:cd /gilda/santiago/vicinanza

selectattr .:MovieTitle .:Runtime 'Runtime > 80'>> >Madagascar<

>> >86<

Metadata Queries

Location, Meeting, dd.mm.yyyy 9IST-2006-026409

E-infrastructure shared between Europe and Latin America

www.eu-eela.org

Exercise:• Log into the Metadata Catalog• Create a directory with your surname into the /grid/gilda/santiago directory

• Add some attributes (Description (varchar(100), Value int, Comment text) to the directory just created

• Add some entries using as entry name the LFNs you uploaded and registered into the File Catalog during the DMS hands-on session

• Fill the attribute fields for the inserted entries• Look for the entries having ‘Value’ > 50

Hands-on Session

Location, Meeting, dd.mm.yyyy 10

IST-2006-026409

E-infrastructure shared between Europe and Latin America

www.eu-eela.org

Query> createdir vicinanza

Query> pwd

>> /gilda/santiago/vicinanza/

Query> addattr /gilda/santiago/vicinanza/ Value int

Query> addattr /gilda/santiago/vicinanza/ Description varchar(100)

Query> addattr /gilda/santiago/vicinanza/ Comment varchar(100)

Query> addentry /gilda/santiago/vicinanza/<file1> Value 1 Description ‘Desc for <file1>' Comment 'Remark for <file1>‘

…

Query> addentry /gilda/santiago/vicinanza/<fileX> Value X Description ‘Desc for <fileX>' Comment 'Remark for <fileX>'

Query> getattr /gilda/santiago/vicinanza/* Value Description Comment

…

Query> find /gilda/santiago/vicinanza/ ‘Value >= k‘ (k=1…X)

Solution

Location, Meeting, dd.mm.yyyy 11

IST-2006-026409

E-infrastructure shared between Europe and Latin America

www.eu-eela.org

– acl_show <directory>Returns the access rights associated to a given pathExample: acl_show /gilda/santiago/vicinanza

– acl_add <directory> <group> <rights> (rights=‘rwx’)

Add/Change the ACL rights associated to the pathExample: acl_add /gilda/santiago/vicinanza gilda:users rx

– acl_remove <directory> <group> <rights> (rights=‘rwx’)

Add/Change the ACL rights associated to the pathExample: acl_add /gilda/santiago/vicinanza gilda:users rx

Metadata ACL

Location, Meeting, dd.mm.yyyy 12

IST-2006-026409

E-infrastructure shared between Europe and Latin America

www.eu-eela.org

Thanks to Riccardo Bruno

who firstly developed these slides