Basics Globus Toolkit™ Developer Tutorial The Globus Project™ Argonne National Laboratory USC Information Sciences Institute Copyright

Basics

Globus Toolkit™ Developer Tutorial

The Globus Project™Argonne National Laboratory

USC Information Sciences Institute

http://www.globus.org/

Copyright (c) 2002 University of Chicago and The University of Southern California. All Rights Reserved. This presentation is licensed for use under the terms of the Globus Toolkit Public License.See http://www.globus.org/toolkit/download/license.html for the full text of this license.

http://www.globus.org/

April 18, 2023 2Globus Toolkit™ Developer Tutorial: Basics

Setting the Foundation

Strategies for the Globus Toolkit– Layered architecture, protocols & APIs

Naming conventions– Functions, variables, structures, etc.

Installation tree structure– Where things get installed

Source tree structure– Where things come from


Globus Approach Software toolkit addressing key technical areas

– Offer a modular “bag of technologies”

– Enable incremental development of grid-enabled tools and applications

– Define and standardize grid protocols and APIs(Our software development supports this goal.)

Focus is on inter-domain issues, not clustering– Supports collaborative resource use spanning multiple

organizations

– Integrates cleanly with intra-domain services

– Creates a “collective” service layer


Globus Approach

Focus on architecture issues– Provide implementations of

grid protocols and APIs as basic infrastructure

– Use to construct high-level, domain-specific solutions

Design principles– Keep participation cost low– Enable local control– Support for adaptation

Diverse global services

Core Globusservices

Local OS

A p p l i c a t i o n s


Layered Architecture

Applications

Core ServicesMetacomputing

Directory Service

GRAMGlobus

Security Interface

Replica Catalog

GASS

GridFTP

Local Services

LSF

Condor MPI

NQEPBS

TCP

AIXLinux

UDP

High-level Services and Tools

Cactus Condor-GMPI Nimrod/Gglobusrun PUNCH

Grid Status

I/O

Solaris

DRM


Why C(Why Not C++)

Portability– Most C++ compilers do not support complete ANSI C++

standard

– This is getting better, but currently C++ is still a headache for highly portable code

Linkage– Linking with Fortran can be difficult

Religion– C++ gives too much rope to hang yourself

– Good C coding practices are usually adequate


Naming Conventions

Globus Toolkit is comprised of a set of modules, each with a unique name

A set of related modules may be part of a larger package

In order to avoid name collisions, all names begin with globus or grid


Function Naming Conventions

All function names and other exported symbols follow the form:

globus_<package>_<module>_<action> The <package> is omitted if the module is

not part of a larger package of related APIs Examples:

– globus_gram_client_job_submit(…)

– globus_nexus_send_rsr(…)


Function Return Values

Most functions return either an int or globus_result_t value

Older APIs return value: int– Like errno

– GLOBUS_SUCCESS (0) indicates that the function was successful

– Any other return value is an error code

– Error codes defined in module headers

– Many modules still use this approach


Function Return Values

Newer APIs return value: globus_result_t– New approach - like exception objects

– GLOBUS_SUCCESS still indicates success

– Otherwise can be expanded to a globus_error_t “object”

> Error type hierarchy with single inheritance

> Can easily match against types (similar to “catch”)

> Can be used to return extended information about the error

– New 1.1 modules use this approach

– Will gradually be pushed through other modules


Library Naming Conventions

Library names follow the same naming convention as the functions in that library:

libglobus_<package>_<module>.a Currently shared libraries are not

supported Examples:

– libglobus_gram_client.a

– libglobus_nexus.a


Program Naming Conventions All program names start with globus or grid Dash (-) is used as a word delimiter Many Globus programs are not directly related to

a single module– May use multiple modules to implement higher

level functionality> e.g. globusrun, globus-job-submit

– May not be directly related to any module> e.g. globus-sh-exec

– But module name used where appropriate> e.g. globus-gass-server


Install Tree Structure

GLOBUS_LOCATION environment variable points to the install tree, containing:– bin: Programs for your path

– sbin: System binaries

– include, lib: Development headers & libraries

– libexec: Programs/scripts used by libraries

– etc: Configuration files

– man: Man pages

– var: Log files, etc.

– share: Other shared data files


Globus Toolkit Modules

common (used by everything)module activation, threads, libc wrappers, callbacks, errors, etc.

io

gss

gss_assist

ldap

Uses

gram_client

duroc_control duroc_runtime

gram_myjob

gass_transfer

nexus

gass_cache

gass

dc

hbm_client

hbm_datacollector

rsl

utp mp


New Globus ToolkitData Grid Modules

Library

Program

Legend

globus-url-copy

ReplicaPrograms

CustomServers

globus_gass_copy

globus_ftp_client

globus_ftp_control

globus_common GSI (security)

globus_io OpenLDAP client

globus_replica_catalog

globus_replica_manager

Custom Clients

globus_gass_transfer

globus_gass

Already exist


Security Modules

gssapi– Grid Security Infrastructure– GSS-API implementation– SSL authentication (public key)

gss_assist– Simplified interface to GSS-API


Resource Management Modules

RSL parser GRAM

– client API: simple client

– myjob API: parallel job bootstrapping

– jobmanager API: server infrastructure

– gatekeeper: root security & dispatch svc DUROC

– control API: client

– runtime API: job startup barriers, etc

– bootstrap API: parallel job bootstrapping


GridFTP Modules

GridFTP control– Low level handling of control and data

channels GridFTP client

– High level client> Get, put, 3rd party transfer, cd, mkdir, rmdir, etc…

– Rich plug-in support for reliability and monitoring


GASS (File Access) Modules GASS url copy

– url-to-url file copy, for file:, http:, ftp: urls– Wrapped by globus-url-copy program

GASS file– open(), close(): works on files and URLs

GASS transfer– Client & server handling of get/put requests

GASS server_ez– Simple, embeddable server

GASS cache– Cache for remote files (prefetch etc.)


Information Services Modules

MDS tools– client programs– server tools– built on OpenLDAP


Miscellaneous Modules

Common– Fundamental system and portability API

IO– Fundamental I/O API

Data conversion UTP timing package Miscellaneous programs and scripts Programs that use multiple APIs


Communication Modules

MP– Simple wrapper around message passing

libraries Duct

– Used by GRAM and DUROC for startup and bootstrapping

Nexus (deprecated)– Multi-method communication library

– Being deprecated


Summary

Hopefully you should be starting to get your bearing within the Globus Toolkit

Next each module will be explored, via:– Summary of functionality

– Walk-through with sample tools

– Walk-through with APIs

– Discussion of interesting issues

Documents

Basics Globus Toolkit™ Developer Tutorial The Globus Project™ Argonne National Laboratory USC Information Sciences Institute Copyright