The Anatomy of the GridThe Anatomy of the GridIan Foster, Carl Kesselman, Steven TueckeIan Foster, Carl Kesselman, Steven Tuecke

Professor: Dr. M. SadjadiProfessor: Dr. M. Sadjadi

Presenter: Sri MedamPresenter: Sri MedamAntagonist: Tariq KingAntagonist: Tariq King

OutlineOutline

Purpose Purpose IntroductionIntroduction Concept of Virtual OrganizationsConcept of Virtual Organizations Need for Grid TechnologyNeed for Grid Technology Grid ArchitectureGrid Architecture Relationships with other TechnologiesRelationships with other Technologies ConclusionConclusion

PurposePurpose

“ “ To argue that the Grid concept is indeed To argue that the Grid concept is indeed motivated by a real and specific problem motivated by a real and specific problem (Grid problem) and that there is an (Grid problem) and that there is an emerging, well-defined technology base emerging, well-defined technology base that addresses significant aspects of this that addresses significant aspects of this problem”problem”

IntroductionIntroduction

Grid Computing Grid Computing

Large-scale, coordinated resource sharing.Large-scale, coordinated resource sharing.

Grid ProblemGrid Problem

Flexible, secure, coordinated resource Flexible, secure, coordinated resource

sharing among dynamic, multi-sharing among dynamic, multi-institutional Virtual Organizations (VOs).institutional Virtual Organizations (VOs).

Virtual OrganizationsVirtual Organizations

Virtual OrganizationsVirtual Organizations

Group of individuals or institutions defined by Group of individuals or institutions defined by sharing rules to share the resources of “Grid” sharing rules to share the resources of “Grid” for a common goal.for a common goal.

Eg: Application service providers, storage Eg: Application service providers, storage service providers, databases, crisis service providers, databases, crisis management team, consultants.management team, consultants.

Need for Grid technologyNeed for Grid technology

Common concerns and requirements forCommon concerns and requirements for Flexible sharing relationshipsFlexible sharing relationships Sophisticated and precise control over how shared Sophisticated and precise control over how shared

resources are used and conditions under which resources are used and conditions under which sharing occurs.sharing occurs.

Sharing of varied resourcesSharing of varied resources Diverse usage modes.Diverse usage modes.

Grid technology provides a general resource-sharing Grid technology provides a general resource-sharing framework that addresses these VO requirementsframework that addresses these VO requirements

Nature of Grid ArchitectureNature of Grid Architecture

Protocol ArchitectureProtocol Architecture

Why interoperability main concern?Why interoperability main concern?

To ensure that sharing relationships can be established To ensure that sharing relationships can be established among varied participants across different platforms.among varied participants across different platforms.

Why protocols critical to interoperability?Why protocols critical to interoperability?

“ “ Local control is preserved ”Local control is preserved ”

Grid Architecture DescriptionGrid Architecture Description

Idea: Standardize the interfaces between interacting components of Idea: Standardize the interfaces between interacting components of the grid.the grid.

Grid Architecture Maps to IP Grid Architecture Maps to IP ArchitectureArchitecture

Basis: Map Grid architecture to any standard protocol architecture Basis: Map Grid architecture to any standard protocol architecture (here Internet protocol architecture)(here Internet protocol architecture)

Fabric LayerFabric Layer

Fabric : Interfaces to local controlFabric : Interfaces to local control

Provides resources.Provides resources. Fabric components implement resource specific Fabric components implement resource specific

operations operations Resources should implement Enquiry, resource Resources should implement Enquiry, resource

management mechanisms…management mechanisms…

Fabric layer Fabric layer

Resources should implement Enquiry, resource Resources should implement Enquiry, resource management mechanisms.management mechanisms.

Computational resources: for starting programs and , for Computational resources: for starting programs and , for monitoring and controlling the execution of the process. monitoring and controlling the execution of the process.

Storage resources: Reading and writing files.Storage resources: Reading and writing files. Network resources: require mechanisms for controlling Network resources: require mechanisms for controlling

resources allocated to network transfer, enquire functions to resources allocated to network transfer, enquire functions to determine network characteristics and load.determine network characteristics and load.

Code repositories: for managing versioned code.( CVS)Code repositories: for managing versioned code.( CVS) Catalogs: require mechanisms for implementing query and Catalogs: require mechanisms for implementing query and

update operations ( relational database)update operations ( relational database)

Connectivity LayerConnectivity Layer

Connectivity: Communicating Easily and Connectivity: Communicating Easily and Securely.Securely.

Defines core communication and authentication protocols.Defines core communication and authentication protocols. Communication requirements: transport, routing, namingCommunication requirements: transport, routing, naming Authentication solutions : Authentication solutions :

Single sign on, delegation, integration with various local Single sign on, delegation, integration with various local security solutions, user-based trust relationships.security solutions, user-based trust relationships.

Resource LayerResource Layer

Resource: Sharing Single ResourcesResource: Sharing Single Resources

Defines protocols for secure negotiation, initiation, monitoring, Defines protocols for secure negotiation, initiation, monitoring, control, accounting, payment of sharing operations on individual control, accounting, payment of sharing operations on individual resources.resources.

Two primary classes of Resource layer protocols areTwo primary classes of Resource layer protocols are

Information protocols, Management protocols.Information protocols, Management protocols.

Management protocol – “ policy application point”.Management protocol – “ policy application point”.

Collective LayerCollective Layer Collective:Collective: Coordinating Multiple ResourcesCoordinating Multiple Resources

Defines protocols that capture interactions across collections of Defines protocols that capture interactions across collections of resources.resources.

Directory services – GRRP, GRIPDirectory services – GRRP, GRIP Co-allocation, scheduling, brokering services – Condor-G, Nimrod-G, DRM Co-allocation, scheduling, brokering services – Condor-G, Nimrod-G, DRM

broker.broker. Monitoring and diagnostic Monitoring and diagnostic Data Replication Data Replication Grid enabled programming Grid enabled programming Workload management and collaboration frameworks Workload management and collaboration frameworks Software discovery - NetSolve, NinfSoftware discovery - NetSolve, Ninf Community authorization Community authorization Community accounting and payment Community accounting and payment Collaboratory services – Access Grid. Collaboratory services – Access Grid.

Application LayerApplication Layer

Programmer’s View of Grid ArchitectureProgrammer’s View of Grid ArchitectureGrid ArchitectureGrid Architecture

Application: implement business logic over fabric, connectivity, resources and collective layers

Example of Grid - GlobusExample of Grid - Globus Define protocol architecture (layers)Define protocol architecture (layers)

Fabric: GT primarily uses existing fabric components. Otherwise, it Fabric: GT primarily uses existing fabric components. Otherwise, it includes the missing functionalityincludes the missing functionality

Connectivity: GT uses TCP/IP for communication, and PKI & TLS based Connectivity: GT uses TCP/IP for communication, and PKI & TLS based Grid Security Infrastructure (GSI) for authenticationGrid Security Infrastructure (GSI) for authentication

Resource: GT uses Resource: GT uses Grid Resource Information Protocol (GRIP) which is based on LDAP,Grid Resource Information Protocol (GRIP) which is based on LDAP, Grid Resource Registration Protocol (GRRP) to register resource information Grid Resource Registration Protocol (GRRP) to register resource information

at servers called Grid Index Information Servers (GIIS),at servers called Grid Index Information Servers (GIIS), Grid Resource Access & Management (GRAM) protocol to allocate and Grid Resource Access & Management (GRAM) protocol to allocate and

monitor computational resources, monitor computational resources, FTP based GridFTP protocol for data access, and,FTP based GridFTP protocol for data access, and, LDAP for catalog accessLDAP for catalog access

Collective: GT uses Collective: GT uses GIISs to support resource views,GIISs to support resource views, LDAP to access GRISLDAP to access GRIS

Application: GT allows development of many sophisticated frameworks Application: GT allows development of many sophisticated frameworks and libraries to implement application layer logic for end use.and libraries to implement application layer logic for end use.

Relationships with other Relationships with other TechnologiesTechnologies

Other technologies:Other technologies: DCE (Open Group) - inflexibleDCE (Open Group) - inflexible Internet (IETF, W3C) – it is Client-ServerInternet (IETF, W3C) – it is Client-Server ASP and SSP (singleton resource type)ASP and SSP (singleton resource type) Enterprise CS (CORBA, DCOM, EJB) – Enterprise CS (CORBA, DCOM, EJB) –

Sharing arrangements are static and Sharing arrangements are static and restricted to single organization.restricted to single organization.

P2P – weak security, strong anonymityP2P – weak security, strong anonymity

ConclusionConclusion

Current technologies either does not accommodate Current technologies either does not accommodate the range of resource types or does not provide the the range of resource types or does not provide the flexibility and control on sharing relationships needed flexibility and control on sharing relationships needed to establish VOs.to establish VOs.

Current technologies integrated with Grid Current technologies integrated with Grid technologies can enhance capabilities to great extent.technologies can enhance capabilities to great extent.

Grid technologies support the sharing and Grid technologies support the sharing and coordinated use of resources in dynamic and scalable coordinated use of resources in dynamic and scalable VOs.VOs.

ReferencesReferences

The Anatomy of the Grid, Ian Foster et al., The Anatomy of the Grid, Ian Foster et al., Supercomputing Journal, 2001Supercomputing Journal, 2001

Evolution of Grid Computing Architecture, J. Evolution of Grid Computing Architecture, J. Joseph, Joseph, IBM Systems Journal, 2004IBM Systems Journal, 2004

Agnostic QuestionsAgnostic Questions

For the connectivity layer, which the authors describe For the connectivity layer, which the authors describe "communicating easily and securely", most of the attention is "communicating easily and securely", most of the attention is given to authentication solutions. Traditional Grid given to authentication solutions. Traditional Grid infrastructure, such as the GSI from Globus, has also infrastructure, such as the GSI from Globus, has also concentrated on authentication and not provided a sufficient concentrated on authentication and not provided a sufficient infrastructure for the rest of the trust hierarchy (i.e. infrastructure for the rest of the trust hierarchy (i.e. authorisation, policy implementation, etc). authorisation, policy implementation, etc).

Question 1:Question 1: To date, what measures have been made towards providing a To date, what measures have been made towards providing a

more complete security infrastructure on the Grid? more complete security infrastructure on the Grid?

Agnostic QuestionsAgnostic QuestionsAnswer:Answer:

As far as the trust hierarchy concerned verification, validation As far as the trust hierarchy concerned verification, validation of Sender and receiver and protection of communication of Sender and receiver and protection of communication channel are important. channel are important.

Verification and Validation of sender, receiver will be taken Verification and Validation of sender, receiver will be taken care by CA verification.care by CA verification.

Channel is protected by many ways - TCP/IP for Channel is protected by many ways - TCP/IP for communication, and PKI & TLS based Grid Security communication, and PKI & TLS based Grid Security Infrastructure (GSI) for authenticationInfrastructure (GSI) for authentication

All of which are very strong and standard protocols for All of which are very strong and standard protocols for encrypting the channel.encrypting the channel.

Agnostic QuestionsAgnostic QuestionsQuestion 2:Question 2: Are any of these solutions also “easy” to install and Are any of these solutions also “easy” to install and

configure? configure?

Answer:Answer:

Grid Accounts Management Architecture (GAMA)Grid Accounts Management Architecture (GAMA) With GAMA, end users never have to know anything about grid security, With GAMA, end users never have to know anything about grid security,

credentials, proxies, or other technical matters. They simply request an credentials, proxies, or other technical matters. They simply request an account using a typical Web form interface, and after the account is account using a typical Web form interface, and after the account is created, they log in to the portal using a familiar username/password created, they log in to the portal using a familiar username/password combination. All the grid activity happens in the background -- from combination. All the grid activity happens in the background -- from creation of grid credentials to retrieval of these credentials for use by creation of grid credentials to retrieval of these credentials for use by portlets. portlets.

With respect to storage resources, the authors mention With respect to storage resources, the authors mention that "third party and high performance transfers are that "third party and high performance transfers are useful". Grid applications today generate very large useful". Grid applications today generate very large datasets, and I/O is already the slowest computational datasets, and I/O is already the slowest computational component by several orders of magnitude when component by several orders of magnitude when compared to memory or processor speed.compared to memory or processor speed.

Question 3:Question 3: As number and size of Virtual Organizations grow, will As number and size of Virtual Organizations grow, will

not data access and retrieval become a major not data access and retrieval become a major bottleneck, and hence remove the attractiveness of the bottleneck, and hence remove the attractiveness of the grid for meeting the needs of some members? What grid for meeting the needs of some members? What approaches are being used to address problem now approaches are being used to address problem now rather than waiting for network speeds to improve?rather than waiting for network speeds to improve?

Agnostic QuestionsAgnostic Questions

Answer:Answer:

Keeping redundant copies in various portions Keeping redundant copies in various portions of grid for faster parallel access using of grid for faster parallel access using distributed system concepts.distributed system concepts.

Data replication services: Support the Data replication services: Support the management of VO storage resources to management of VO storage resources to maximize the data access performance with maximize the data access performance with respect to time, reliability and cost.respect to time, reliability and cost.

Question 4:Question 4: What work is being done to address some of the more What work is being done to address some of the more difficult I/O topics for Grid computing such as difficult I/O topics for Grid computing such as

noncontiguous I/O, caching, and fault noncontiguous I/O, caching, and fault tolerance?tolerance?

Answer:Answer: In my opinion Noncontiguous I/O, caching, fault In my opinion Noncontiguous I/O, caching, fault tolerance are implementation details of the resources. tolerance are implementation details of the resources.

Grid architecture is about the interaction of Grid architecture is about the interaction of resources resources rather than their implementation.rather than their implementation.

Quesiton 5: This paper discusses solutions to many of the technical Quesiton 5: This paper discusses solutions to many of the technical challenges of the Grid. Do you think that challenges of the Grid. Do you think that

management management of collaboration efforts is just as of collaboration efforts is just as important as solving important as solving the technical challenges? If the technical challenges? If so, why? Could including so, why? Could including such information in such information in the paper have improved the the paper have improved the discussion of any discussion of any particular section?particular section?

Answer:Answer: This paper mainly about the anatomy means structural This paper mainly about the anatomy means structural description of the grid. Collaboration to improve description of the grid. Collaboration to improve

coordination and resolve conflicts among coordination and resolve conflicts among process is process is highly important.highly important.

Management of collaboration will be further research Management of collaboration will be further research into into this field which will be more interesting and this field which will be more interesting and challenging.challenging.

For this paper the idea is to present the anatomy how For this paper the idea is to present the anatomy how it it looks, structure wise and conceptslooks, structure wise and concepts..

Question6:Question6: By the definition of Grid computing, the authors stress By the definition of Grid computing, the authors stress the need for interoperability across organizations. In the need for interoperability across organizations. In your opinion, is the ultimate vision of Grid computing your opinion, is the ultimate vision of Grid computing obscured by the lack of interoperability standards obscured by the lack of interoperability standards among Grid technologies? among Grid technologies?

Answer: Answer: Interoperability is necessary to ensure that sharing Interoperability is necessary to ensure that sharing relationships can be initiated among dynamic relationships can be initiated among dynamic participants across different platforms, languages, and participants across different platforms, languages, and programming environments for resource sharing.programming environments for resource sharing.

without interoperability it becomes a basic without interoperability it becomes a basic client/server client/server technology.technology.

Applications and services on Grid will work together Applications and services on Grid will work together based on their requirements and set guidelines by based on their requirements and set guidelines by

the the system architects.system architects.

Question7:Question7: In previous presentations Global Grid Forum (GGF) In previous presentations Global Grid Forum (GGF) and OASIS have been mentioned as leaders in moving and OASIS have been mentioned as leaders in moving towards a primary set of Grid standards. What other towards a primary set of Grid standards. What other organizational bodies are currently pushing towards organizational bodies are currently pushing towards guaranteeing interoperability among Grids? Is this guaranteeing interoperability among Grids? Is this effort sufficient considering the grand challenges facing effort sufficient considering the grand challenges facing the Grid?the Grid?

Answer:Answer: Other leaders contributing to grid standards are W3C, Other leaders contributing to grid standards are W3C,

Distributed Management Task Force, Web Distributed Management Task Force, Web ServicesServices

Interoperability Organization, Internet2, Liberty Interoperability Organization, Internet2, Liberty AllianceAlliance