Upload
hallie
View
21
Download
0
Tags:
Embed Size (px)
DESCRIPTION
WW Grid. Convergence Characteristics for Clusters, Grids, and P2P networks. Panel at the P2P conference, Linkoping, Sweden. Gri d Computing and D istributed S ystems (GRIDS) Lab . The University of Melbourne Melbourne, Australia www.gridbus.org. Rajkumar Buyya. Where Did I come From ?. - PowerPoint PPT Presentation
Citation preview
Convergence Characteristics for Clusters, Grids, and P2P
networks
Rajkumar BuyyaGrid Computing and Distributed Systems
(GRIDS) Lab. The University of MelbourneMelbourne, Australiawww.gridbus.org
WW Grid
Panel at the P2P conference, Linkoping, Sweden
2
Where Did I come From ?
*
*
*
*
Melbourne
Stockholm+Linkoping
Singapore
London
3
GRIDS Lab @ the U. of Melbourne, The Gridbus Project:
www.gridbus.org Conduct R&D in Distributed Computational Economy for
Service-Oriented Computing: Architecture, Specification, and Open Source Reference
Implementation in in Collaboration with Any Global Peers. Platform: Cluster, Grid, P2P for diverse applications.
Distributed Application Composition Tools Grid Economy & Scheduling (via Nimrod-G Broker) GridSim Toolkit: Grid Modeling and Simulation (Java based): Libra: Economic Cluster Scheduler Grid Bank: Accounting and Transaction Management World Wide Grid (WWG) – A Global Testbed P2P/Jxta-based Compute Power Market Application Enabler Projects:
Virtual Laboratory Toolset for Drug Design High-Energy Physics and the Grid Network (HEPGrid) Brain Activity Analysis on the Grid Future: GridEmail for Spam Management
—read only if sender pays!
4
Books
5
Technology Evolution: Cluster, Grid, P2P
6
2100
2100 2100 2100 2100
2100 2100 2100 2100
Desktop SMPs or SuperComputers
LocalCluster
GlobalCluster/Grid
PERFORMANCE
Inter PlanetaryGrid!
•Individual•Group•Department•Campus•State•National•Globe•Inter Planet•Galaxy
Administrative Barriers
EnterpriseCluster/Grid
?
Scalable Seamless Computing: Breaking Administrative Barriers
7
Basic Elements
Security
Resource Allocation & Scheduling
Data locality
Network Management
System Management
Resource Discovery
Uniform Access
Computational Economy
Application Development Tools
8
Cluster Architecture
Sequential Applications
Parallel Applications
Parallel Programming Environment
Cluster Middleware
(Single System Image and Availability Infrastructure)
Cluster Interconnection Network/Switch
PC/Workstation
Network Interface Hardware
Communications
Software
PC/Workstation
Network Interface Hardware
Communications
Software
PC/Workstation
Network Interface Hardware
Communications
Software
PC/Workstation
Network Interface Hardware
Communications
Software
Sequential Applications
Sequential Applications
Parallel ApplicationsParallel
Applications
9
Cluster Architecture
Sequential Applications
Parallel Applications
Parallel Programming Environment
Cluster Middleware
(Single System Image and Availability Infrastructure)
Cluster Interconnection Network/Switch
PC/Workstation
Network Interface Hardware
Communications
Software
PC/Workstation
Network Interface Hardware
Communications
Software
PC/Workstation
Network Interface Hardware
Communications
Software
PC/Workstation
Network Interface Hardware
Communications
Software
Sequential Applications
Sequential Applications
Parallel ApplicationsParallel
Applications
10
A Typical Grid Computing Environment
Grid Resource Broker
Resource Broker
Application
Grid Information Service
Grid Resource Broker
databaseR2R3
RN
R1
R4
R5
R6
Grid Information Service
2
11
A Typical Grid Computing Environment
Grid Resource Broker
Resource Broker
Application
Grid Information Service
Grid Resource Broker
databaseR2R3
RN
R1
R4
R5
R6
Grid Information Service
2
12
A Typical P2P Computing Environment
Peer Agent
Peer Agent
Application
Peer Discovery Service
Peer Agent
P1
P3
pN
P2
R7
Who
can
hel
p ?
Peer
P 2, P
7 can
hel
p!
pM
Request
Response
p4 p5
RequestSorry, I am busy.
13
A Typical P2P Computing Environment
Peer Agent
Peer Agent
Application
Peer Discovery Service
Peer Agent
P1
P3
pN
P2
R7
Who
can
hel
p ?
Peer
P 2, P
7 can
hel
p!
pM
Request
Response
p4 p5
RequestSorry, I am busy.
14
Cluster, Grid, P2P: Characteristics
Characteristic Cluster Grid P2PPopulation Commodity
ComputersHigh-end computers Edge of network
(desktop PC)
Ownership Single Multiple Multiple
Discovery Membership Services
Centralised Index & Decentralised Info
Decentralized
User Management Centralised Decentralised Decentralised
Resource management
Centralized Distributed Distributed
Allocation/Scheduling
Centralised Decentralised Decentralised
Inter-Operability VIA based? No standards yet No standards
Single System Image Yes No No
Scalability 100s 1000? Millions? [@Home]
Capacity Guaranteed Varies, but high Varies
Throughput Medium High Very High
Speed(Lat. Bandwidth)
Low, high High, Low High, Low
15
Nimrod-G: Modeling Distributed Computing in Economic Terms
Grid Middleware
Nimrod/G Client Nimrod/G ClientNimrod/G Client
Grid Information Server(s)
Schedule Advisor
Trading Manager
Nimrod-G Engine
GridStore
Grid Explorer
GE GISTM TS
RM & TS
Grid Dispatcher
RM: Local Resource Manager, TS: Trade Server
Globus, Legion, Condor, etc.
G
G
CL
Globus enabled node.Legion enabled node.
GL
Condor enabled node.
RM & TSRM & TS
C L
16
On-Demand Computing
On-Demand Computing
CostCostDeadlineDeadline
Legion hosts
Globus Hosts
Bezek is in both Globus and Legion Domains
Arlington
Alexandria
Richmond
HamptonNorfolk
Virginia BeachChesapeakePortsmouth
Newport News
Roanoke
Ap p om a toxRive r
Ja m esRive r
Shena nd oa hRive r
Ra p p a ha nnoc kRive r
Potom a cRive r
VIRGINIA77
81
64
64
66
85
17
Active Sheet Nimrod-G:Transparent Spreadsheet Processing from PCs to
Clusters to World-Wide Grids
NimrodNimrodProxyProxy
Nimrod-GNimrod-G
World-Wide Grid
18
MEG(MagnetoEncephaloGraphy) Data Analysis on the Grid: Brain Activity
Analysis
Life-electronics laboratory,AIST
Data Analysis
•Provision of expertise in the analysis of brain function•Provision of MEG analysis
Data Generation
Nimrod-G
64 sensors MEG
Results
Analysis All pairs (64x64) of MEG data by shifting the temporal region of MEG data over time: 0 to 29750: 64x64x29750 jobs
World-Wide Grid•[deadline, budget, optimization preference]
1
5
4
3
2
[Collaboration with Osaka University, Japan]
Backup Slides
20
Evolution Time Line
* Sputnik
1960 1970 1975 1980 1985 1990 1995 2000
* ARPANET
* Email* Ethernet
* TCP/IP* IETF
* Internet Era * WWW Era
* Mosaic
* XML
* PC Clusters* Crays * MPPs
* Mainframes
* HTML
* W3C
* P2P
* Grids
* XEROX PARC wormCO
MP
UTIN
GN
ETW
OR
KIN
G
* Web Services
* Minicomputers
* PCs
* WS Clusters
* PDAs* Workstations
* HTC
21
What is Cluster/Grid ?
A type of parallel and distributed system that enables the sharing, selection, & aggregationof resources distributed in _____/_______ administrative domains depending on their availability, capability, performance, cost, and users quality of service requirements.
A Single
Multiple
Cluster Grid
22
CPM: DC Economy-based P2P Computing
(Jxta based Implementation)
Accounting
Bill
Resources
(Provider)
User (Consumer)
CPM Agent
Trader
Market Server
Market Repository
Job Management
- Discovery
- Membership
23
Drug Design: Data Intensive Computing on Grid
It involves screening millions of chemical compounds (molecules) in the Chemical DataBase (CDB) to identify those having potential to serve as drug candidates.
Protein
Molecules
Chemical Databases(legacy, in .MOL2 format)
[Collaboration with WEHI for Medical Science, Melbourne]
24
Virtual Drug DesignA Virtual Lab for “Molecular Modeling for Drug Design” on P2P Grid
“Screen 2K molecules in 30min. for $10”
Grid Market Directory
ResourceBroker
Grid Info. Service
GTS
GTS
GTS
GTS
“Give me list PDBs sourcesOf type aldrich_300?”
“serv
ice co
st?”
(GTS - Grid Trade Server)
PDB2
“get mol.10 from pdb1 & screen it.”
Data Replica Catalogue
“service providers?”
GTS
PDB1
“mol.10 please?”
“mol.5 please?”
(RB maps suitable Grid nodes and Protein DataBank)