The Moab Grid Suite CSS´ 06 – Bonn – July 28, 2006

The Moab Grid SuiteCSS´ 06 – Bonn – July 28, 2006

Computation in Life Science Research

High-throughput virtual screening (serial)

Molecular dynamics or Monte Carlo simulations (parallel)

Free energy calculations (parallel)

Cheminformatics (serial)

Structure prediction and design (serial/parallel)

Quantum chemistry (serial/parallel)

©2006 anterio consult & research Moab Grid Suite

Multiple sites, continents

Multiple applications, hardware

Different policies, operating systems, security

Problems a Pharma/Biotech May Face

Moab Grid Suite


Moab Grid Suite is a policy-based grid management solution that integrates scheduling, managing, monitoring and reporting of workloads across

independent clusters.

Moab Grid Suite

Boehringer Ingelheim

Biogen Idec

Fred Hutchinson Cancer Research Center

TeraGrid, WestGrid, CMA, CDAC


Exemples of Customers

Technical Overview

Message PassingMessage Passing

SerialSerialParallelParallel ApplicationApplication

Resource ManagerResource Manager

Operating SystemOperating System

Hardware (Cluster or SMP)Hardware (Cluster or SMP)

AdminAdmin UsersUsers

Se

cu

rityS

ec

urity

ApplicationApplication

PortalPortal

CLICLI

GUIGUI

Cluster Workload Manager: Scheduler, Policy Manager, Integration PlatformCluster Workload Manager: Scheduler, Policy Manager, Integration Platform

Grid Workload Manager: Scheduler, Policy Manager, Integration PlatformGrid Workload Manager: Scheduler, Policy Manager, Integration Platform


Cluster Stack / Framework:

Architecture

Technical Overview


What Moab does

Moab Grid Suite


Optimizes Resource Utilization with Intelligent Scheduling and Advanced Reservations

Unifies Cluster Management across Varied Resources and Services

Dynamically Adjusts Workload to Enforce Policies and Service Level Agreements

Automates Diagnosis and Failure Response

What Moab does not do

Moab Grid Suite


Does not does do resource management (usually) Does not install the system (usually) Not a storage manager Not a license manager Does not do message passing

ClusterB

ClusterB

Grid Types

Administrator

ClusterA

Local Area Grid (LAG)

ClusterB

ClusterC

Shared User SpaceShared Data Space

Moab

ClusterA

Wide Area Grid (WAG)

ClusterB

ClusterC

Multiple User SpacesMultiple Data Spaces

Moab (Master)

Moab Moab Moab

ClusterA

ClusterB

ClusterC

Moab (Grid Head Node)

Moab Moab Moab

Centralized Management

ClusterA

ClusterC

Moab

Centralized & Local Management

All Grid Rules

Moab (Grid Head Node)Shared Grid Rules

Local Grid Rules

Moab

Local Grid Rules

Moab

Local Grid Rules

ClusterA

ClusterC

Moab

Local Management“Peer to Peer”

Local Grid Rules

Moab

Local Grid Rules

Moab

Local Grid Rules

Management Scenarios


Grid Relationship Combination

Administrator

ClusterA

ClusterB

ClusterC


Moab


ClusterD

ClusterE


Local Area Grid Rules

Moab

Local Grid Rules

Moab

Local Grid Rules

1. Join local area grids into wide area grids --- Moab is able to facilitate virtually any grid relationship.

1



Administrator

ClusterA

ClusterB

ClusterC


Moab


ClusterD

ClusterE



Moab

Local Grid Rules

Moab

Local Grid Rules

ClusterF

ClusterG

ClusterH

Moab

Local Grid Rules

Moab

Local Grid Rules

Moab

Local Grid Rules

2. Join wide area grids to other wide area grids (whether they be managed centrally, locally - “peer to peer” or mixed)

12



Administrator

ClusterA

ClusterB

ClusterC


Moab


ClusterD

ClusterE



Moab

Local Grid Rules

Moab

Local Grid Rules

ClusterF

ClusterG

ClusterH

Moab

Local Grid Rules

Moab

Local Grid Rules

MoabLocal Grid Rules

Hosting Site

Moab

Local Grid Rules

3. Resource sharing can be in one direction for use with hosting centers, or to bill out resources to other sites

12

3



Administrator

ClusterA

ClusterB

ClusterC


Moab


ClusterD

ClusterE



Moab

Local Grid Rules

Moab

Local Grid Rules

ClusterF

ClusterG

ClusterH

Moab

Local Grid Rules

Moab

Local Grid Rules

MoabLocal Grid Rules

Hosting Site

Moab

Local Grid Rules

4. Have multiple levels of grid relationships (e.g. conglomerates within conglomerates within conglomerates)

12

3


4

Administrator

Sovereignty: Local vs. Centralized Management Policies


Local Admin

Each Admin can manage their own cluster

Submit to either:• Local cluster• Specified cluster(s) in the grid • Generically to the grid

Local Cluster AResources

Grid AllocatedResources

Portion Allocated

to Grid

Local Admin can apply policies to manage:

1. Local user access to local cluster resources

2. Local user access to grid resources

3. Outside grid user access to local cluster resources (general or specific policies)

Local Users

Outside Grid Users

1

2

3

Grid Administration Body

Grid Administration Body can apply policies to manage:

A. General grid policies (Sharing, Priority, Limits, etc.)

A

Administrator

Data Staging

Manages intra-cluster and inter-cluster job data staging requirements so as to minimize resource inefficiencies and maximize system utilization

Prevent the loss of compute resources due to data blocking and can significantly improve cluster performance.


Administrator


ClusterA

Moab

Local Grid Rules

4

Start job after the file is verified to be in the right location

1

StorageSystem

Data Manager

Job Submission

3

Benefits:Prevents non-staged jobs from blocking usage of nodes

Drawbacks:No job-centric prioritization takes place in the order of which data gets staged first

2

Verified Data Staging

Administrator


ClusterA

Moab

Local Grid Rules

5

Priority order of data staging

1

StorageSystem

Data Manager

Job Submission

4

3

Priority Jobs First

2

Prioritized Data Staging

Administrator


Priority order of data staging and data-staging centric scheduling

Benefits:Prevents non-staged jobs from blocking usage of nodes Provides soft prioritization of data staging requestsIntelligently schedule resources based on data staging information

Drawbacks:Prioritization is only softly provided

ClusterA

Moab

Local Grid Rules

6

StorageSystem

Data Manager

Job Submission

5Priority

Jobs First

32

4

1

Fully Scheduled Data Staging: External Storage

Administrator


Priority order of data staging and data-staging centric scheduling

Benefits:Prevents non-staged jobs from blocking usage of nodes Provides soft prioritization of data staging requestsIntelligently reserves resources based on data staging information

Drawbacks:Prioritization is only softly provided

ClusterA

Moab

Local Grid Rules

Data Manager

Job Submission

52

4S

S S

Storage is on Local Compute Nodes

1

3Priority

Jobs First

6

Fully Scheduled Data Staging: Local Storage

Administrator


Monitoring performance statistics of multiple independent clusters

Detecting and diagnosing failures from geographically distributed clusters

Tracking cluster, storage, network, service, and application resources

Generating load-balancing and resource state information for users and middleware services

Information Services

Administrator


Administer Resources and Workload Policies Through an Easy-to-Use Graphical User Interface

Monitor, Diagnose and Report Resource Allocation and Usage

Moab Grid ManagerTM

Administrator

Moab Grid ManagerTM


Service Monitoring and Management:

Administrator

Moab Grid ManagerTM


RealTime Performance and Accounting Analysis:

End User

Moab Access PortalTM

Submit jobs from a web browser View and modify only your own workload Assist end users to self-manage behaviours

Assists users in better utilizing resources:

General info Job eval. Completed job failure post-mortem. Job start time estimates. Job control. Reservation control.


Contact

Tassilo Eckerle

Sales Manager

Phone: +49 (6 21) 4 00 41-34Fax: +49 (6 21) 4 00 41-40Mobile: +49 (1 63) 4 00 41 34E-Mail: [email protected]

Jianxin Duan, PhD

Research Scientist

Phone: +49 (6 21) 4 00 41-57Fax: +49 (6 21) 4 00 41-40

E-Mail: [email protected]

Thank you for your attention!

For further questions please do not hesitate to contact us.

Documents

The Moab Grid Suite CSS´ 06 – Bonn – July 28, 2006