w w w . h p c - e u r o p a . o r g

Single Point of Access to Resources of HPC-Europa

Krzysztof Kurowski, Jarek Nabrzyski, Ariel Oleksiak, Dawid Szejnfeld

Poznań Supercomputing and Networking Center

Cracow Grid Workshop, 14 December 2004

w w w . h p c - e u r o p a . o r g

Outline• HPC-Europa• JRA2: Single Point of Access• Assumptions & Requirements• Architecture• Components of the SPA• Job Submission Interface• Conclusion

w w w . h p c - e u r o p a . o r g

HPC-Europa• HPC-Europa - Pan-European

Research infrastructure on High Performance Computing for the Science of 21st century

• Goal: to provide advanced computational services in an integrated way to the European research community

• Budget: ~20 mln EURO• 14 Partners across Europe• Project activities

– Transnational Access Programme– Networking Activities– Joint Research Activities (JRA1,

JRA2)

w w w . h p c - e u r o p a . o r g

JRA2: Single Point of Access• Motivation

– To provide a uniform access to all centers resources, transparently and regardless of user physical location

• Main objectives– ease of use,– improve availability,– global optimization of resources utilization

• To achieve these goals– JRA2 builds a HPC-Europa portal providing uniform, flexible and

intuitive user access to Grid resources from anywhere– Develops and/or adapts tools and services needed to establish full

operational Grid environment– Will develop a meta-broker responsible for users’ access to all

consortium’s resources

w w w . h p c - e u r o p a . o r g

Assumptions & Requirements

• User-oriented– Uniform, transparent

access– Easy to use, intuitive

interface– Increased resource

availability and access to all centers

• Provider-oriented– Keep centers autonomy

• To enable it to use its own chosen middleware

• To allow local policies• Not to weaken center’s

security– Optimize global resource

usage

w w w . h p c - e u r o p a . o r g

Accounting & Charging Model• Allocation Unit (AU)

– 1TFlop sustained for 1 hour

• Kept for each resource provider for every single HPC system

• Used for 2 purposes:– charging the EC for the use of HPC

facilities– limiting the maximum amount of

resources a user (or group of users) may utilize

• The common pool of AUs concept– If a center contributes n AUs to the

global infrastructure, then its users get access to n AUs from a common pool

Center BCenter A

Center C

Common pool of AUs

Common pool of AUs

nA AUs nB AUs

nC AUsnC AUs

nA AUs nB AUs

w w w . h p c - e u r o p a . o r g

Typical Grid Architecture

Grid Portal / Grid Resource Broker

HPC centerHPC center HPC center

HPC center

VO

w w w . h p c - e u r o p a . o r g

HPC-Europa SPA

Grid Portal / Grid Resource Broker

HPC centerHPC center HPC center

HPC center

Grid Middleware AGrid Middleware B Grid Middleware C Grid Middleware D

VO

w w w . h p c - e u r o p a . o r g

HPC Centers MiddlewareCenter High-level middleware Underlying Grid

technologyLRMS

CINECA-

UNICORE LoadLevelerOpenPBS with MAUI

HLRS-

UNICORE OpenPBSERS-NQSPBS Pro

CEPBA eNANOS (resource broker)

Globus 3.2 LoadLevelerOpenPBS

NQS

EPCC JOSH (JOb Scheduling Hierarchically)

Globus 3.2 LoadLevelerSGE

NTUA GRIA (Grid Resources for Industrial Applications)

Condor

PSNC GRMS (Grid Resource Management System)

Pre-WS Globus 3.2 OpenPBS LSFSGE

w w w . h p c - e u r o p a . o r g

Architecture of the SPA (1st stage)

GridSphere Framework

Multi-Grid Services

Job submission

Data mngmt

ResourceInformation

Workflow Accounting Sofware

RepositoryUser

ManagerPortlets

Service Plugins

JOSH Plugin

GRMS Plugin

eNanos Plugin

UNICOREPlugin

GRIAPlugin

Globus MDSPlugin

…

SPA

Authorization Service

Software Repository

Data Management System

Accounting System

Information Service

w w w . h p c - e u r o p a . o r g

Architecture of the SPA

GridSphere Framework

Multi-Grid Services

Job submission

Data mngmt

ResourceInformation

Workflow Accounting Sofware

RepositoryUser

ManagerPortlets

Service Plugins

JOSH Plugin

GRMS Plugin

eNanos Plugin

UNICOREPlugin

GRIAPlugin

Globus MDSPlugin

…

SPA

Authorization Service

Software Repository

Data Management System

Accounting System

Information ServiceMeta-broker

w w w . h p c - e u r o p a . o r g

GridSphere• GridSphere:The advanced open-source portlet-based portal framework (

www.gridsphere.org)• Features:

– portlet-based (supporting JSR168 standard)– customization mechanism for a wide range of end users– a flexible easy to use interface for users– a model to create pluggable and dynamic application support for portal

developers– pluggable access to Grid services using the Portlet Services concept– uniform access to various Grid systems using the Grid Portlets model– set of useful core and Grid portlets– open source

• Chosen as the HPC-Europa portal on the basis of portals evaluation made

w w w . h p c - e u r o p a . o r g

Main Services• Meta-broker

– Discovers and choose resources, and submits job to HPC centers

• Replica management system– Goal of the HPC-Europa NA3

• Accounting System– Historical information about resource usage (including AUs)

• Information System– Central information indexing service

• Software Repository– Logical names, meta-data, binaries compiled for various systems, needed

environment (preinstalled libraries, environment variables, etc.)

• Authorization Service – Authorize users’ request, check resource limits

w w w . h p c - e u r o p a . o r g

Multi-Grid Services• Extension of the GridSphere’s Grid Portlets concept by

– Interface based on standards and functionality of several Grid systems– Dynamic loading of multiple service implementations (on the basis of a

given HPC center)– Remote access

• Definition of the common interface– For each functionality, e.g. job submission, resource information etc.– Based on standards where possible (e.g. JSDL)– Taking into account both gathered requirements and the functionality of

Grid middleware deployed in HPC centers• Capability check

– description of implemented capabilities in the form of the XML document

– used to disable not available options (in portal) or to select sites which can accept given job description (meta-broker)

w w w . h p c - e u r o p a . o r g

Job Submission Interface (JSI)

• Job submission operations– submitJob (UNICORE, GRMS,

JOSH, eNANOS, GRIA)– cancelJob (UNICORE, GRMS,

JOSH, eNANOS, GRIA)– getJobInfo (UNICORE, GRMS,

JOSH, eNANOS, GRIA)– findResources (UNICORE,

GRMS, eNANOS)– submitJobToBestResource

(GRMS, JOSH, eNANOS, GRIA)– findBestResource (JOSH)

• Data elements descriptions– Job Description XML schema– Job Information XML schema

• Auxiliary operations– getMiddlewareName – getMiddlewareDesc– getMiddlewareCaps

• Job states– FINISHED, FAILED, PENDING,

QUEUED, RUNNING, CANCELLED, SUSPENDED, WAITING

w w w . h p c - e u r o p a . o r g

Job Description

• Based on the GGF Job Submission Description Language (JSDL) specification

• Subset of JSDL defines the common interface

• The specification is still evolving

w w w . h p c - e u r o p a . o r g

Conclusion• Benefits of the SPA

– Practical:• enables scientists to solve their real problems using HPC

resources distributed across Europe in a uniform and easy way

– Research: • interoperability of various grid technologies and middleware, • hierarchical model of Grid middleware

• Open issues– Interoperability of different security models– Global policies concerning usage of AUs by HPC-Europa users