NextGRID Architecture Stephen Davey, NeSC, UK. Copyright 2006 NextGRID Project Partners 2 Contributors & Acknowledgments Particular thanks to Malcolm

NextGRID Architecture

Stephen Davey, NeSC, UK

Copyright 2006 NextGRID Project Partners 2

Contributors & Acknowledgments

Particular thanks to Malcolm Atkinson (National e-Science Centre)

David Snelling (Fujitsu Labs Europe)

Bryce Mitchell (British Telecommunications plc) NextGRID Architecture WP1 & many

others

www.nextgrid.org


Introduction - Aim & Scope

These slides intend to cover the following:NextGRID Project & GoalsNextGRID ArchitectureService Level AgreementsChallengesExperiments & Applications


NextGRID Project 11M€ EU FP6 project

3 years starting sept 2004 22 partners, some industrial, some academic. Developing Architecture for Next Generation

Grids. Business focus. SLAs & QoS, not just best effort (academic). Basis in Standards (for interoperability &

stability). Research and exploration project: A 5-10 year

lookout


NextGRID Partners

Academic partners


NextGRID Partners

Commercial partners


NextGRID’s Goals Develop architectural components that will lead to the

emergence of the Next Generation Grid. Prepare the way for the broader use of Grid technologies

and their widespread adoption by organisations and individuals from the business and public domains.

Improve the technology in key areas such as security, development of applications, interoperability, quality of service and robustness.

Contribute to the standards which are essential if NGGs are to succeed.


Project Vision

Grids Applicable to Industry “... a passive information medium into an active tool for

creating and exploring new knowledge and fuelling business and industry.”

Inter Enterprise Grid Integrate the interests and business models of competing

stakeholders

Define the architecture that will lead to the emergence of the Next Generation Grid. for organisations and individuals from across business and

public domains

Middleware Components


Grid Computing Vision

Grid offering services

In the Grid computing model, businesses need to own fewer of their own resources. Third parties provide facilities (Grid service providers); users get access to services. Businesses can also offer services over the Grid, an extension to the electricity grid analogy.

Incrementally delivering virtual regularity and homogeneity –

progressively making it easier to build and operate distributed systems – choosing to forgo some autonomy for economy

and interoperability


Electricity GridHomes, offices and industry get their power from a central grid, which looks the same to everyone

Electricity Grid


Impediments in “Now Grid” Performance and Scalability Focus

Security and operational integrity are critical. Service Level is Typically only Best-effort

Partners need to agree on service levels of the relationship. Resource Allocation Done via “Tokens”

Transition to an open marketplace needs currencies. Diversity of Commercial Applications

The benefits of a Grid approach to typical business applications have not been clearly demonstrated.

Unreliable Costs Estimates Lack of Stability in Platforms/Standards Management Complexity

IT operational costs in industry are dominated by administration (i.e. people).


Why does Architecture Matter? Essential Framework

Partitioning e-Infrastructure Enables collaboration Enables alternatives Enables composition Identifies gaps & opportunities

Essential Discipline Without club rules there isn’t a club

Sense of Direction Synthesis of Current understanding Synthesis of Requirements and Opportunities Proposition for a new future


NextGRID Approach Schemas

Message, Security, SLA, Service Description Support Systems

Naming and Addressing Trust and Security SLA Management Registry

Functional Systems Data Focused

Provenance, Transfer, Storage, Processing, Access

Orchestrators Workflow and Federation


NextGRID Interactions

Registry

Functional

SLAManagement

Trust andSecurity

Naming andAddressing

Orchestration

Register /Update

Query

Resolve

Generate /Verify

Administer policy

Monitor/Control

Get tokens

Negotiate SLA

Invoke

Get tokenassertions

Register /Update /Query

Get tokenassertions

Get tokenassertions

Get tokenassertions

Schemas


Big themes (focus areas)

Workflows

Service Level Agreements

Security

Data


Workflows across domains

CompositionBrokering, Planning, ReservationQuality of Service metricsWorkflow description languages

SecurityWhat player can do what / interact with whom

at what point in time


Service Level Agreements

Establishing SLAsDescription, preparation, negotiation

Evaluation and monitoringQuality of Service

AftermathAccounting, billing, dissolution


What are SLAs for?

The SLA is a commercial tool that manages customer expectations and clearly sets out supplier obligations with respect to the offered service.

It also sets out any penalties that might occur for SLA violation


Goals of SLAs

Predictable, reliable performance Performance guaranteesSecurity

Industrial acceptabilityUse methods and tools industry is familiar and

comfortable with using SLAs are tool to achieve these goals


service provider

serviceTechnical Perspective

CustomerSLA

Business Relationship

Configuration Policy

Monitoring Policy

EventEvent

Event

Business Perspective


service provider

service

User

SLA

Configuration Policy

Monitoring Policy

EventEvent

Business layer

Technology layer

NextGRID ServiceNextGRID Service

SLA

SLA

SLA

SLA

SLA

SLA?

SLA SLA


SLAs in NextGRID

Key Architectural Principle All interactions composed of bi-partite

agreements governed by SLAs All tasks need to understand the role of SLAs in

NextGRID Tasks role in the SLA lifecycle Business tool

Build trust – signed by real people Differentiate between offerings using same base

language


Why not use WSLA/WS-Agreement?

Not using WS-Agreement or WSLA “Inspiration” for our SLA

WSLA disappeared inside IBMsuperseded by WS-Agreement

WS-Agreementgood structure for foundationno business context (static terms)


Examples of SLAs

ConsumerA

ServiceB

SLA(A,B)

ServiceC

SLA(A,C)

SLA(B,C)?

Orchestration

ConsumerA

ServiceB

SLA(A,B)

ConsumerD

SLA(D,B)?

Brokering

ConsumerA

ServiceB

SLA(A,B)

ServiceC

SLA(B,C)

Outsourcing


Challenges

Some challenges identified by analysis of NextGRID's key application driven experiments (conducted by WP7).

Security Related Challenges: Virtual Organisations

It is envisaged many types of services will be on offer to clients. E.g. Implied volatility calculation, data feeds and data storage services.

Security Inter-organisational character of the Grid requires

very strong security and confidentiality measures when it comes to the operation of enterprise business applications that deal with sensitive data.


Security

VO lifecycle support Dynamic trust establishment Multiple security mechanisms Process-based authorization Operational considerations


Management Related Challenges Availability

Availability and the end-to-end quality of service provided by the underlying infrastructure are essential.

NextGRID provides architectural constructs, such as dynamic SLAs, targeted at addressing this challenge.

Service Discovery In a dynamic Grid, discovery of services cannot be performed

"out of band" from the architecture. Integration of Legacy Code

Many Grid applications interconnected with traditional local applications which have had highly expensive investment.

NextGRID is designed to be conservative enough in its approach that a wholesale replacement of infrastructure is at no time required.


Management Related Challenges (cont.) Usage Control over Resources

Grid system has to offer a system to monitor, control and restrict the actions of Grid users in order to restrict financial and operational risk.

In NextGRID manifested as a policy framework used in SLAs and based on mechanisms similar to those of the Semantic Web.

Dynamic Configuration Reconfigured to speedup service invocations, or to increase the

security. Should be able to switch between multiple protocols.

Interoperability and non-exclusive resources Need for interoperability on the middleware layer. Grid may need to share resources with traditional infrastructures.

Data Management Technologies Traditional approaches mostly file based, but enterprise business

applications typically using relational database management systems.


Performance Related Challenges Response Times

Should be as fast as possible in financial applications. Data Rates

High peak data rates required for fast access to market data sources.

Network bandwidth must be maximised when data transferred, and latencies must be eradicated.

Note that performance requirements of the applications themselves are internal to the service provider, easily managed with SLAs, and have little effect of the architecture of the Grid.


Data related challenges Multiple data sources Distributed Heterogeneous Autonomously managed Extract new information from multiple data sources Many cases where the assembly of all the data into a

single data warehouse is inappropriate.

No unnecessary data movement Build on existing standards Extensible architecture – cannot anticipate all desired

functionality


Data Transfer Data Movement Transport Protocols

SOAP, SOAP with Attachments, MIME, DIME, MTOM, GridFTP, Parallel HTTP.

0

10000

20000

30000

40000

50000

0 10 20 30 40

Number of MBytes transferred

Du

rati

on

(m

s)

Axis base 64

.net base 64

Socket

Axis Sw A


Data Movement

Data Movement(Standard) InterfacesTransport Protocol Agreement

NextGRID SLAs?

GGF - Data Movement Interface Standardisation Working Group, DMIS-WG.


Application examples & Experiments NextGRID reference applications from WP7 focus

on 3 primary developments from the areas: Financial modelling

Implied Volatility Derivatives Pricing

Digital media production On-demand video rendering

Supply chain management

Challenges already presented are manifested in these applications and resolved by the NextGRID Architecture from the experimental evaluation process of NextGRID.


Development of key components

Data Discovery components Data Integration components

Query planning and processingQuery matchingDistributed query execution

Unified Data and Compute Resource Model

Registry components

40

Notions of Grid

• Collaboration Grids− Multiple institutions, secure, widely distributed, VOs− Service level agreements & commercial partnerships− Business model: Increase overall revenue

• Enterprise Grids− Virtualization of enterprise resources and applications − Aggregation and centralization of management− Business model: Reduce total cost of ownership

• Clusters− Networks of Workstations, Blades, etc.− Cycle scavenging, Homogeneous workload− Business model: Lower marginal costs

• Parallel Processing Systems− Parallel processing for single applications Incr

ea

sing

Co

mp

lexi

ty a

nd

Rev

enu

e


NextGRID Architecture White Paper Aims

Clarify and present the NextGRID Vision Create a shared view of the conceptual architecture Inform the future work of the project Act as living document to track progress in the project

Overview Project Vision and Technical Challenges Architectural Principles

Dynamics, Composition, Infrastructure Experiment Validation

http://www.nextgrid.org/download/publications/NextGRID_Architecture_White_Paper.pdf


Questions?

http://www.nextgrid.org

Documents

NextGRID Architecture Stephen Davey, NeSC, UK. Copyright 2006 NextGRID Project Partners 2 Contributors & Acknowledgments Particular thanks to Malcolm