Alice May02

8/2/2019 Alice May02

1/37


2/37

26 May 2002 Teo Yong Meng, NUS 2

SETI@Home


3/37


Cost of I dle Comput ing Cycles

Desktop Processor Utilisation

$ / processor(desktop) $ / used

$ / usedprocessor

cost of unused cycles

one desktop $1200 $300 $150 $10501000 desktops $1,200,000 $300,000 $150,000 $1,050,000

Source: Adapted from Internet Infrastructure & Services by Bear, Stearns & Co., May2001. Based on IDA tender price, the cost of a Pentium PC desktop is estimated to be$1200.

Assumpt ions:1. Deskt op ut ilizat ion is 25%; 8 hr s/ 24 hr s = 33%; f act or ing in lunch,

r est r oom, et c. a deskt op can be idle up t o 90%

2. When a pr ocessor is in used, assume a peak ut ilizat ion of 50%


4/37


Out line! Gr id comput ing over view! Over view of Globus! ALiCE Middlewar e

! Key f eat ur es!

Pr oducer -consumer model! Templat e-based gr id pr ogr amming! ALiCE applicat ions

! ALiCE vs Globus! Super comput er , Physical and Vir t ual

Clust er


5/37


Gr id Comput ing (1)

Client / Server ModelGrid Model


6/37



Flexible, secur e, coor dinat ed r esour ce shar ing amongdynamic collect ions of individuals, inst it ut ions, and

resourceFrom The Anat omy of t he Gr id: Enabling Scalable Vir t ual Or ganizat ions

Enable communit ies (vir t ual or ganizat ions) t o shar egeogr aphically dist r ibut ed r esour ces as t hey pur suecommon goals -- assuming t he absence of cent r al locat ion,

cent r al cont r ol, omniscience, exist ing t r ust r elat ionships


7/37


Gr id Comput ing (3) Resour ce shar ing

Comput er s, st or age, sensor s, net wor ks,dat abases,

Shar ing always condit ional: issues of t r ust , policy,negot iat ion, payment ,

Coor dinat ed pr oblem solving Beyond client -ser ver : dist r ibut ed dat a analysis,comput at ion, collabor at ion,

Dynamic, mult i- inst it ut ional vir t ual or gs Communit y over lays on classic or g st r uct ur es

Lar ge or small, st at ic or dynamic


8/37



A d v a n t a g e s sharing and a g g r e g a t i o n

of resources l e v e r a g i n g on resourcesyou dont own

co m p u t i n g o n d em an d - focus on business rather than

technology- reduce business costs

r e m o t e a cce ss to expensiveresources (proprietary datasets,..)

capability and scalability fault tolerance ..

Cha l l enges heterogeneity

distributed ownership dynamic behavior internetbased on best effort packetdelivery

security ease of use .


9/37


Design Complexit ies

consist ency GridSyst em

p e r f o r m a n c e

s h a r e d

d a t a

maint ainabilit y

s e c u r i t y

s c a l a b i l i t y

e a s e

of

u s e d e

c e n t

r a l i z

a t i o na v a i l a b i l i t y

a t o m i c i t y

m o b i l i

t y

Too manyobjectives

Not enoughprinciples!


10/37


Thr ee Main Obst acles

in Gr id Comput ing1) New appr oaches t o pr oblem solving

Dat a Gr ids, dist r ibut ed comput ing, peer -t o-peer ,collabor at ion gr ids,

2) St r uct ur ing and wr it ing pr ogr ams

Abst r act ions, t ools

3) Enabling r esour ce shar ing acr oss dist inctinst it ut ions

Resour ce discover y, access, r eser vat ion, allocat ion;aut hent icat ion, aut hor izat ion, policy; communicat ion; f aultdet ect ion and not if icat ion;

Pr o g r a m m i n g Pr o b l e m

Sy s t e m s Pr o b l e m


11/37


Globus Layer ed Gr id Ar chit ect ur e

Co n t r o l l i n g t h i n g s l o ca l l y :Access to, & control of, resources

Ta lk i n g t o t h i n g s :communication (Internet protocols)& security

Sh a r i n g s i n g l e r e so u r ce s :negotiating access, controlling use

Co o r d i n a t i n g m u l t i p l e r e s o u r c es :ubiquitous infrastructure services, app-specific distributed services

Resource

Fabric

Connectivity

Collective

Application

InternetTransport

Application

Link

I n

t er n

e t P r

o t o c ol A r

c h i

t e c t ur

e


12/37


Globus Layer ed Gr id Ar chit ect ur e

Applications

Core ServicesMetacomputing

DirectoryService

GRAMGlobus

SecurityInterface

ReplicaCatalog

GASS

GridFTP

LocalServices

LSF

Condor MPI

NQEPBS

TCP

AIXLinux

UDP

High-level Services and Tools

Cactus Condor-GMPI Nimrod/Gglobusrun PUNCH

Grid Status

I/O

Solaris

DRM


13/37


Globus Toolkit

Sof t war e t oolkit def ines a set of ser vices (gr id pr ot ocols and API s) (par t ially) implement ed as of a collect ion of t ools

Focus is on int er -domain issues, not clust er ing suppor t s collabor at ive resour ce use spanning mult iple

or ganizat ions

int egr at es wit h int r a-domain ser vices


14/37


Key f eat ur es

Pr oducer -consumer model

Templat e-based pr ogr amming

ALiCE Applicat ions

Globus vs ALiCE

Super comput er , physical and vir t ual gr id


15/37


What is ALiCE (Adapt ive and sca LableI nt er net -based Comput ing Engine)?

Client / Server Model

Grid Model

Brokered Grid ModelALiCE

server


16/37


ALiCE (Adapt ive and scaLableint er net - based Comput ing Engine)

" Suppor t f or development and deployment of gr idapplicat ions

" Templat e-based pr ogr amming t o mask complexit y of gr idinf r ast r uct ur e" J ob-par allelism t o maximize t hr oughput

" (J ava) obj ect -par allelism t o maximize per f or mance" Dist r ibut ed load-balancing algor it hm" Task r eplicat ions f or f ault -t oler ant and meet ing

per f or mance deadline" Dif f er ent iat ed levels of secur it y (code, dat a and r esult ) at

var ying cost s

" I mplement ed in Java and J ava J ini/ J avaSpaces


17/37


ALiCE Pr oducer - Consumer Model

Co n s u m e r s ( C)" interface to users" launch point for applications" collection point for results(visualization)

Re so u r ce Br o k e r ( RB )" authentication

" application execution control" resource management

" scheduling" load balancing

"

Pr o d u ce r s ( P)" provide computing power" executes tasks


18/37

26 May 2002 Teo Yong Meng, NUS 18ConsumerConsumer

# J ava SDK 1.3# J ava J ini 1.1/

J avaSpaces# J ava Ref lect ion

API# Swing

ProducerProducer

Resource BrokerResource Broker

task result Task PoolTask Pool

# J ava SDK 1.3# J ava J ini 1.1/

J avaSpaces# Swing

# J ava SDK 1.3# J ava J ini

1.1/ J avaSpaces

ALiCE I MPLEMENTATI ON


19/37


ALiCEALiCE Consumer GUIConsumer GUI

ControlControlPanelPanel

MessagesMessages

ResourceResourceBroker InfoBroker Info

UserUserRequirementsRequirements

Task InputTask Input


20/37


ALiCEALiCE Producer GUIProducer GUI

Control PanelControl Panel

MessagesMessages

PerformancePerformance

TaskTaskInformationInformation

J b E i


21/37


J ob Execut ion

1. JobLaunchersends jar file via TCP


22/37


Types of Applicat ions Suppor t ed

1. Sequent ial J obs (par amet r ic comput at ion) suppor t s single-t asking pr ogr ams wit h well-

def ined met hods like main() or r un()

2. Par allel J obs - Obj ect - level Par allelism suppor t s var ious par allel progr amming modelsvia pr ogr amming t emplat es

allows t ask and result obj ect s t o beexchanged bet ween consumer s and pr oducer st hr ough r esour ce br oker


23/37


Templat e- based Pr ogr amming

I nt er f ace f or pr oducer t o inst ant iat eand r et ur n r esult

Result

Specif y f unct ions t o execut e atpr oducer

Ret ur n a Result obj ect

Task

Visualizer t o be invoke at consumer

Met hod t o r et r ieve r esult s

Result Collect or

I nvoked at r esour ce br oker Met hod t o send t asks t o pr oducerTaskGener at or

Funct ionTemplat e


24/37


J ob Execut ion

process ( )

collectResult()

Result


25/37


Task Generator TemplateTask Generator Template// Task Generator Templateimport alice.application.*; // im por t t he t em pla t es

public class CLASSNAME extends TaskGenerator {

// p lac e your v ar ia bles h ere

// Constructorpublic CLASSNAME(){};

// The no param et e r co ns t ruc tor i s a MUST.

public void init() { // p lac e your in i t i a l i za t ion c ode here

}// init()

/*** generateTasks() - generates tasks***/

public void generateTasks() {

// Th is i s w here th e t ask s a re genera t ed // Usua l ly t ask s a re genera t ed in a loop , // and in t h i s loop eac h ta sk i s sen t fo r // p roc ess ing by c a l l ing t he // // pub l i c vo id p roces s (Task t ) me t hod

} // generateTasks()

/*** main method**/

public static void main(String args[]) {CLASSNAME m = new CLASSNAME();m.init();m.generateTasks();

}

} // end class


26/37


Result Collector TemplateResult Collector Template// Template for ResultCollector//import alice.application.*; // im port t het e m p l a t e s

public class CLASSNAME extendsResultCollector {

// p lac e your v ar iab les her e //

public static void main(String args[]){

CLASSNAME MV = new CLASSNAME ();MV.init();MV.collectAllResults();

}

// th e no a rgument c ons t ruc to r MUST ex is t

public CLASSNAME() {}

public void init() { // p lac e your in i t c odes here //

}

public void collectAllResults() {

// Here i s th e resu l t handl ing c ode . // Usua l ly resu l t handl ing invo lves a loo p // t ha t repea t ed ly c a l l s the c o l lec t Resu l t () // me t hod o f t he Resu l tCo ll ec t o r supe rc l a s s .

// // Th is m eth od re t u rns a Resu l t Objec t . // // The c on t en t s o f th i s Resu l t Objec t c an be // inspec t ed fo r resu l t handl ing /proce ss ing .

}}


27/37


Task TemplateTask Template // Template for Task

import alice.application.*; // im por t t he t em pla t e s

public class CLASSNAME implements Task {

// p lac e you r var i ab les here // public CLASSNAME () {}

public Result execute() {

// Th i s i s w here you do your c a l c u la t ion // The r e su l t s a re s t o red in t he Resu l t c l a s s // w h i c h f u n c t i o n s a s a d a t a s t r u c t u r e // w i t h w h i c h y o u c a n s t o r e r e s u l t s o f a n y O b j e c t t y p e

}

public String toString() { // r e tu r ns a S t r ing t ha t c an be used t o ID your t a sk

}}


28/37


ALiCE Applicat ions

distributed ray tracing

georectification of satellite images (CRISP)

mandelbrot setN-body problem

distributedequation solver

protein alignmentand matching (BII)

Primer Search inChromosome Sequences(Nanyang Polytechnics)

G if i iG if i i f S lli If S lli I


29/37


Geor ect if icat ionGeor ect if icat ion of Sat ellit e I magesof Sat ellit e I mages


30/37


Geor ect if icat ionGeor ect if icat ion ALiCEALiCE Pr ogr amPr ogr am

l


31/37


ALiCEALiCE Applicat ionsApplicat ionsPr ot ein Alignment and Mat ching 50 MBchromosome dat abase, swisspr ot (NCBI BLAST d/ bser ver ), P2, 450MHz, 256MB memor y

- Sequent ial ~ 2 hour s

- ALiCE wit h 8 pr oducer s, 4000 sequences/ t ask

~1400 secondsN- body pr oblem (n=20, 000 bodies) pr edict ing t hemot ion of ast r onomical bodies in space. - Sequent ial 9428 seconds- ALiCE: 2 pr oducer s = 1109 sec, 8 pr oduces = 457sec


32/37


Globus vs ALiCE

Globus is a Gr id Toolkit (pr ovides set of ser vices)

I s an open syst em (user s can develop higher -level ser vices on t op of basic ser vices)

Adv: modular it y & r eusabilit y of ser vicesDis: Gr id I nf r ast r uct ur e set up is complex

app development / deployment is complex

- is lar gely plat f or m dependent (most ly UNI X)

- Resour ce Management (via GRAM) at J ob Level


33/37


Globus vs ALiCE ALiCE

User -Or ient ed Gr id Comput ing Engine I nt egr at es all ser vices (basic+higher level)

t o f acilit at e: Ease of inst allat ion, deployment and administ r at ion Ease of Gr id App Development / Deployment using obj ect

pr ogr amming t emplat e

- Plat f or m I ndependent (cor e ser vices implement ed inJ ava)

- Resour ce Management at Obj ect -level (f ine-gr aincont r ol)


34/37


Super comput er s

J une 29, 2000 I BM ASCI Whit e 8192 RS/ 6000 pr ocessor s, 12.3 TFLOPS 6 TB memor y, 160 TB disk st or age US$ 110m, 106 t ons, 28 t r act or -t r ailer

trucks

Apr il 20, 2002 NY Times J apanese Comput er is Wor ldsFast est

NEC

US$350m, occupies 4 t ennis-court

640 specialized nodes, 5104pr ocessor s

achieved 35.6 TFLOPS ver sus7 TFLOPS in ASCI Whit e


35/37


Cost of a Super comput ing, a Physical Clust er

and a Vir t ual Gr id of 100, 000 PCs

" I BM ASCI Whit e (2000) US$ 110m

" NEC (2002) US$ 350m

" Physical clust er of 100K I nt el P4 ~ US$200m + US$13m

(elect r icit y)

" Vir t ual clust er cost is dist r ibut ed and absor bed by PC owner s


36/37


Acknowledgement s

Collabor at or s:" Cent r e f or Remot e I maging, Sensing and Pr ocessing (CRI SP)" BioI nf or mat ics I nst it ut e" Nanyang Polyt echnic (School of Lif e Sciences)" The Royal I nst it ut e of Technology, Sweden

Acknowledgement : Sun Micr osyst ems


37/37


Thank you.Questions & Answers

Documents

Alice May02