Academic Technology Services The UCLA Grid Portal - Campus Grids and the UC Grid Joan Slottow and Prakashan Korambath Research Computing Technologies UCLA

Academic Technology Services

The UCLA Grid Portal -Campus Grids and the UC Grid

Joan Slottow and Prakashan KorambathJoan Slottow and Prakashan KorambathResearch Computing TechnologiesResearch Computing TechnologiesUCLA Academic Technology ServicesUCLA Academic Technology Services

http://grid.ucla.edu


https

Uniform browser based interface

Credential mapped to local ID

Grid Appliance

Head node

Cluster II

Head node

Cluster I

Credential mapped to local ID

Grid Portal Grid Appliance

ION Visualization ServerStorage Server

UCLA Grid Architecture

MyProxy Server


Single Campus Architecture

CampusGrid

Portaland CA

A

A

C

C

CampusMyProxyServer

CampusStorageServer

A = Grid ApplianceC = Cluster and Cluster Head Node

• Grid Portal accesses the storage servervia GridFTP.• No other access is provided.


Single CA is shared among all campuses

UC Portal

UCLAPortal UCSB

Portal

UCIPortal

A

A

A A

A

A

C

C

C C C

C

All appliances talk to both theircampus portal and the UC portal

UCLAMyProxy

UCSBMyProxy

UCGrid CA

UC UserDatabase

UCMyProxy

UC Register Service

StorageServer

Storage Server

StorageServer

UCIMyProxy

Multi-Campus Architecture for the UC


Pools• Resource Pool:

– Clusters contribute cycles to the resource pool– Clusters contribute applications to a resource pool:– TeraGrid cycles purchased by the campus can easily be shared.

• Pool Users:– Every cluster user is also a pool user.– Many students and faculty members do not currently have login ids on any

of the clusters. They can easily sign up to be pool-only users.– Each pool-only user is assigned a Storage Area.

• Pool use hides all the details • Pool jobs

– Target cluster selected by the UCLA Grid Portal.– Currently run applications only.

• A similar sharing of resources and applications can be applied across the entire UC.


User Types

• 2 Types of users:– Cluster User:

• Has a login ID on at least 1 cluster

– Pool-Only User:• Does not have a login ID on any cluster

• Both types of users have certificates/usernames to login to the Grid Portal


User Types vs. PortalsPortal

User

Type

Campus Portal UC Portal

Pool-onlyUser

ClusterUser

Can submit jobs to the campus pool

Can submit jobs to theUC pool

Can use those clusters on campus he/she can access

Can submit jobs to the campus pool

Best choice for thosewith access to clusterson different campuses-- can use all clusters,UC-wide, he/she can Access

Can submit jobs to the UC pool


To Login to a Portal a User Needs:

• A certificate• A gridsphere account on that Portal• Additionally:

• A Cluster User must be added to the gridmap file on the appliance node of each of the clusters on which he/she has a login id• A pool-only user needs to be assigned a storage area on the Grid Portal’s Storage Server.


Workflow -- For the UC Grid -- 1

Campus Grid Portal

All authenticationis done at the campuslevel

Sends a messageto campus grid admin

User

AuthenticationForm requesting

Proposedusername

Unique?

Requested username/password pending

Usernames are unique across the entire UC

UCGrid CA

UCMyProxy

UC Register Service

UC UserDatabase

Sends a messageto cluster admin

Pool-only user

Cluster user


Workflow -- For the UC Grid -- 2

Campus Grid Portal

Authenticates

CampusGrid

Admin

CreatesGridSphere

account(assigns storage

area)

Clicks Approve

Sends request +Host Cert

CampusMyProxy

UC RegisterClient

UCGrid CA

UCMyProxy

UC Register Service

UC UserDatabase

Creates and signs user certificate

Creates gridsphere account at UC Portal

Pushes certificate to UC MyProxy

Pushes certificate to campus MyProxy

Requested username/password pending



Pool Job Submission

• Submitting a pool job


How Pools Work

• Currently applications only• A cluster that contributes to the pool must

create:– Guest login id – Certificate for Grid Portal– Put guest in the gridmap file– Mechanism that allocates resources for pool jobs

is up to the cluster administrator


Pool Job Submission

PortalUser

Storage Area

Find the best cluster

Select a guest user

Generate guest user proxy

Stage input files to target cluster

Database

StatusChange

?

Update

PendingActiveDoneError Job submitted via GRAM Job Service and the

guest user proxy

On Storage Server or Originating Custer

Guest UserStorage Area

A

C

Create a uniquely-named subdirectoryAdd Database entry for this job

When notified by GRAM of a status changeUpdate the database

Target Cluster


Pool Job Output


Pool Job Output

• Retrieving Pool Job Output


TeraGrid

• UGP acts as a client to the TeraGrid.• Their Globus Toolkit nodes are open to all of their users.• They allowed us to get the CA Signing Policy and CA

Certificate Public Key for each of their clusters.• It only goes one way.

User Proxy Certificate

myproxy.ncsa.uiuc.edu

UCLAGrid

Portal

Request for user proxy

User proxy certificate returned becauseUCLA Grid Portal can authenticate

DataStar atSDSC

user proxy certificate

UCLA pool jobs currently can run on two TeraGrid clusters.


Submitting Jobs to the TeraGrid

Submitting a job to the TeraGrid


Grid Development Environment and Ajax TechnologiesAjax• Ajax -- Asynchronous JavaScript and XML• Create web applications that are more interactive• Only those parts of a page that change are updated

GDE• Create project, import files, save project• Fully integrated editor interface• GDE-generated Makefile or use pre-existing user Makefile• Can for as many windows as you want. (Can be on different clusters)• Tested in IE, Safari, Firefox• Uses Zimbra Open AJAX Toolkit (BSD license)• Compilations submitted to instantaneous queues (must run on the cluster,

not on the appliance)


GDE

Ajax Grid Development Environment

GDE Project Directory


Visualization


Pools -- Now

• Single Pool– Every campus cluster user is also a pool user.– Pool-only users can only submit to the pool – Any cluster can contribute cycles to the pool

• It can choose which of its applications it will contribute to the pool

• When a pool job is submitted, the Portal decides which cluster will run it.


Futures -- Specialty Pools

• Clusters can form Specialty Pools

– A user must be approved by the pool admin to – A specialty pool includes a list of applications – Grid Portal databases for: pools and pool users


UCLA UC

MatlabAmberQ-ChemMathematicaMathematica

Matlab

Amber

Q-Chem

Gromacs

VaspJaguar

AmberFluent

UCLA

UCSB

TeraGrid

UCI

MathematicaCorsicaOsiris baby b2

Jaguar*

*

* = CNSI

Mathematica

Gromacs

***

Joan is in the default pools

Frank is in the default pools and the UCLA Physics PoolPrakashan is in the default and chemistry pools

Futures -- Specialty Pools


Futures -- Redundancy and Failover

• Also called big IP• It could be Apache or it could be hardware• Farms out work to the portals

.

.

MySQL Master Slave

Disk May be a shared file system RAID5 or just like an appliance

LB

MyProxy MyProxy

RAID5

Load Balancer

Load Balancer

This is a feature of MySQL which automaticallymirrors the database on the slaves.

Portal

Portal

LB


Workflow for Cluster User -- Now -- Single Campus

ClusterHead node

SSH authentication

Sends a messageto cluster admin

Apply Page Approves and adds to the

gridmapfile

Sends a messageTo grid admin Creates

gridsphereaccount

Sends a messageTo the user

Request for password, create and sign certificate, push it to MyProxy

User


Workflow for Pool-only User -- Now -- Single Campus

ISISused at UCLA

link

Apply Page

User authenticated if he/she belongs to the UCLA community

Sends a messageTo grid admin

Sends a messageTo the user User can now log in

User

Purse (on MyProxy

Server)

Creates gridsphere account and assigns storage area on storage server


Workflow -- Add a Cluster on Another Campus -- 1

OtherCampus

Grid Portal

Main PageSends a messageto other campus grid admin

User

Username Authentication

UCGrid CA

UCMyProxy

UC Register Service

UC UserDatabase

Sends a messageto cluster adminGridSphere account

not required


Workflow -- Add a cluster on Another Campus -- 2

Other Campus

Grid Portal

Authenticates

OtherCampus

GridAdmin

CreatesGridSphere

account

Clicks Approve

Sends request +Host Cert

OtherCampusMyProxy

UC RegisterClient

UC CA

UCMyProxy

UC Register Service

UC UserDatabase

Pushes certificate to Other campus MyProxy


Portal Decides Where the Job will Run

Amber Job Submitted

Requirements?Time,

Number of nodes

Current LoadQueueLength

Clusters with Amber

Can resources meet requirements?

Select one

Academic Technology ServicesUCSB Batch Queue Prediction Algorithm

UCSB Prediction Algorithm:Daniel Nurmi

75% of the jobs queued start in317 seconds ~ 5 minutes

95% of the jobs queued start in54,000 seconds ~1.5 hours

Hoffman cluster Parallel Queue


Futures -- Job Status and the UC Grid Portal

A

A

A A

C1

C2

C1 C2JobsDB

JobsDB

JobsDB

Joan submits a job to: - UCLA C1 - UCLA C2 - UCLA poolFrom the UCLA Grid Portal

Joan submits a job to: - UCLA C1 - UCI C1 - UC PoolFrom the UC Grid Portal

UCLAPortal

UCIPortal

UC Portal


Documents

Academic Technology Services The UCLA Grid Portal - Campus Grids and the UC Grid Joan Slottow and Prakashan Korambath Research Computing Technologies UCLA