1

Architecture Task Force2 Third Meeting

CeSCCambridge

Centre for Mathematical Sciences18th December 2003

2

Administration

PresentMalcolm Atkinson (chair & note taker)Howard ChiversJohn DarlingtonAndrew HerbertDave De RoureTom RoddenJoe Sventek

Notes of Last MeetingAgreed as useful

ActionsSome people still haven’t supplied info for the Web Page!!!!

3

Agenda

Notes of Last MeetingActionsPresentations

ChiversSventek

PresentationsVisions & Showstoppers

Integration & ClassificationThe Next StepsThe Next Meeting

4

Note

The slides beyond this one were typed by me (Malcolm Atkinson) as we went through the meeting – they were slightly modified afterwards by John Darlington and myself reading them through quicklyThey are an impression of the meeting in chronological order, with some repetitionThey do not necessarily represent the combined views of the meeting or the eventual position of the ATF2

5

Presentation 1: Howard Chivers

Architecture in IndustryOr what is an architecture?

What kind of things do we want to constrain?Large scale data warehousing systemsData cleaning _. AnalysisLarge systems + high Tx rates + lots of usersEach component large

User requirement sets dictate divisionDifferent developers

Enterprise viewWant to share infrastructureLife cycle costs, etc.Continuous business processes

6

HC: cont

Total system-wide properties (e.g. security)

Security, legal, etc.

Architecture Good enough set of block diagrams and infrastructure constraints

Can give guarantees at total business requirements

To constrain individual developments just enough

Leave the component development teams enough freedom

What is the enterpriseBuilding for the future don’t know use cases

7

HC: cont

Design approachesTop down – scales well

High initial cost, inflexible & bespoke subsystems results

Cyclic replacement Flexible, maximises immediate,, COTS infrastructure

– Unspecific contracts, assumes infrastructure – ignores whole life cost, difficult to scale

Alternatively, set of infrastructures around Define a virtualisation layer Does it exist already? E.g. Java was a VM for HC’s example

Deployment issues What is an application What is the deployment life cycle

8

HC: cont

Process objectivesEarly choice of the infrastructure

What set of services Virtualisation layer? What physical services buy & deploy First set Cycled development of infrastructure

Void early specification Collect generic requirements initially Not specific requirements reqs change Avoid collecting detail!

9

HC: cont

Three viewsApplication view

Business process view – data flows and workflows Broad but shallow use cases

Functional model at a high-level early Major interfaces aservices Deployment into application subsystems

Quantitative model Data requirements get missed unless you do this at a high-

level Pairwise agreements ignore

10

HC: cont

Infrastructure ModelFoundation layers: Network & ComputingApplication Environments (EJB, web, DB, …)Infrastructure services (security, directories/registries, event logging, data storage, messaging, …)Quantitative modelApplication deploymentSystem management features

Event monitoring, system diagnosis & control)

11

HC:cont

InterfacesLayered from infrastructure to application

Comms style, e.g. SOAP, opaque msg delivery Generic standards, XSDs, protocols,

Generic interoperability << valuable place to look at this

Format and content or message Environments, e.g. namespaces Services required to interpret content

Application integration Behaviour of application (high-level protocol) May have deployment information

12

HC: cont

Frameworks, e.g.Data access

Defines ‘n-tier’ access architecture between user & data Ensure common approach between several systems Prevent each application inventing their own

Security << care about this in larger system

Policy framework for whole system– E.g. localise data requiring a particular sensitivity applies– People’s names, etc. in only one place– Then protect with a firewall

Required services Required constraints on parts of the system

– Infrastructure & application model

13

HC: discussion

Never time to just do the circle/triangleDo we have a clear idea of what the apps will be

Major changes to business practicesHigh rate of perturbation now

What was Grid specific?Cross organisational aspectsDiscussion about relationships between organisationsNo one without core business survivesProjects & IPRBusiness strategyMicrosoft is the “Big shark” needs fish in the ecosystem

14

Presentation 2: JS

Systems & Infrastructure VisionPrevious experience at DOE lab.

Most successful infrastructure mechanisms

IP IP6

What should we doIP at the Grid levelBuild on distributed O/S

15

JS: cont

O-O architectureStatically defined and assembled into apps

Inflexible and brittle

Didn’t scale at network levelANSA/CORBA transparencies

Aspect-Oriented ComputingStatically defined separate viewsTools weave views together to produce applications

Again static result

Policy-Oriented ComputingWhere we are going

16

JS: cont

Policy-orientedConstraints/preferences that must be metDynamically use the policies to integrate componentsContinue to adjust compositionBeware large monolithic applicationsTwo forms of dynamism

Every person / group uses it differently– Business processes evolve

Changing technology

Control-plane is where we most need commonality“Lesson” from telecoms partners of ANSA

But business model adapted to infrastructure – we carry flaky phones

17

JS:cont

Why did IP workSit at centre of hour glass must adapt to changes above & belowAgreed that wouldn’t always be optimal

What does the hour glass mean for us at this level

Big change – we have many kinds of resourcesJob dispatching

What are the crucial activities above us?What do we have beneath?

Dynamically changing sets of machines, …

18

JS: cont

Can view this as a distributed virtual operating system

Deal with scheduling and sharing of all kindsof dynamically varying resources

Back to policySome work on policy based managementDoes this this lead to (semi) autonomic computingMuch work needs doing here

Basis for understanding the hour glassWe then understand what we are trying to defineKey features: Virtual OrganisationsResources

Plug & Play only recentThink about what they do as an input

19

JS:Cont

Where does policy take us?How do we represent policyWhat can we say / not sayWhat do policies apply toWhat resolution systems can we buildHow do we engage the humanHow do policies compose

Internet routing policies are a microcosm

20

JS:cont

History of networksWe just add the number of features we can assume

Seamless access to all resources available in the virtual organisation, at any scale.This implies

Distributed O/S, with pre-emptive scheduling

Aggressive split between policy and mechanism split

21st Century security

21

JS:cont

Big questionWhat are we trying to architect

The policy space at the centre of the hour glassAbstract patterns needed?

Expect that these will evolve

What tests our progressWhat information do we want to get into policiesFollow the abstract patterns

Avoid the concrete framework, etc. trapsWhat are the dominant patterns we should captureWhat vocab do we need?

22

JS: Discussion

PolicyChance to link between formal models & systemsOpportunity for good tools

Can map to what you have today With tool input for what we want tomorrow

Declarative things should play well hereSeparate meaning from behaviour

Separation of concerns

How do we define our boundariesHow far above / below the waist of the hour glassNeed a temporal model

23

Presentation 3: Howard Chivers

Architecture – some requirementsCentre projects by typeData centric (information grids) and users/visualisation applications39% of the projects developing infrastructure

Because it wasn’t there (MPA’s theory)Indefinite behaviour of composing thingsMay always have the wrong emphasis in common infrastructure

“The Grid” is too simple a deployment model

24

HC: cont

The Grid has missing structureSupport for groups of resourcesSupport for groups of people

Separate the VO of usersFrom the VO of resource suppliers

Get linked by policy and brokering

VirtualisationApplies to provide & publish resourcesDoes the separation of supply and consumption win us anything?

25

JS: cont

Resource provider states what policy operates for use of the resources

26

Presentation 4: John Darlington

Deploying applications over Grids Information Capture & Use

Keywords from e-Science Easy, transparent, … Use, development & execution

Keywords from Grid Complex applications on complex machinery

Capture knowledge about build, composition, use

ICENIComponent architecture

Place to hang knowledge

Resources that will be used Place knowledge about these with their proxy

Discover, interact, broker Dynamically revise binding between Component & Resource

27

JD: cont

ComponentUnit of composition in which all context dependencies are explicit

ServiceComponent performing a task

Abstraction levels for componentsRoles

Sources of information (in CXML)

Developer Produces implementations

Scientist Designs useful components

User Composes and executes application

28

JD: cont

Separation of concernsMeaning, behaviour & implementation

Composition must satisfy constraintsTool help possible

Slides on steps from composition to executionComponent definition to Execution

On slides

29

JD: continued

What descriptionNot yet related to standardsDescribes both resources and servicesStructured language

High-level language Set of information that needs to be collected

Tools to extract workload and performance

Magpie [HOTOS 2002]

Feedback loop into development and maintenance

Failure analysis

30

Oral Presentation: Andrew Herbert

Conversation last weekUsed to be difficult to persuade company interest

Hard to find believable business case

Even harder on standardsTrying to get focus

Steve Tuecke + Indigo project teamDebate went surprisingly wellDistinguish between services and resourcesRecognition of the possibilities

31

AH: Cont

Where it ended up WS-Context story carrying things aroundMany major problems evaporated

Now seems to be dialoguePerson assigned person

Marvyn Theimer

Encouraging conversationFrustrations between Globus

Harmony about where want to go with WS

32

AH: cont

We should look at composing WS & resources

Where do we see opportunities

CollaborationLiving rooms, consumers & wireless

33

Presentation 5: Dave De Roure

Semantic Grid workshop @ GGF8Grid problem: interoperabilityScale driving the requirement

E-ResearchComputation, dataPeople & VOsKnowledge – digital artifactsExtended enterprise

Need large scale heterogeneous, interoperability

Components, … bring them all together

34

DDR: cont

Architecture that describes frameworks for components and their operation

PrinciplesMeans of describing collections of infrastructure cmpntsMeta-information architecture

Quote from Tom (coherence crucial)Not just doing itMake it easy to doShift of focus

From developer Through experience builder Towards participant

35

DDR: cont

Focus on userEase of assemblyEase of design-build-testSelf configurationEase of ownership

Semantic Web – brings technologyNetwork effectURI server/relationshipsRDF bus – rules

Examples: comb-e-chem goes to lab and publication – early provenance, etc. – smart tea

36

DDR: cont

Bootstrap problemIncentives to create metadata not apparent to early usersStandards for standards versus standardsCommunity-based standards

Gene-ontology vs Globus ontology Can we every reach such a complete / extensive / coherent

knowledge integration

GGF provides some of the framework for thisBoth sharing and interoperating have potential problems

Lots of small things that interoperate

37

DDR: cont

Semantic web might not workDistributed semantic web unprovenStill being researchedLive & real work underway

38

Digesting Meeting

Consistent pointsDescriptions centralNot a common infrastructureMinimal knowledge and tools to make architecture work

Agreed Document target for completion by MayIdentification of scope, requirements and structure of architecture

ConstraintsCan’t assume static systemAvoid detailBut danger of nothing to get teeth into

Understand variants from practical systemsPilots and IRCs at NeSC

39

Digestion 2

February’s meetingHow do we extract the high-level patternsAre these classes of business practicesOr common patterns within applications

What is our idea about compositionExamples

Information used in build and deployLists of terms / wordsOMII harvest – look at this – find persistent principlesDescribing architectures and testsAgent-oriented architecture relevant?

40

Digestion 3

Very high cost of looking after components and deployed code

Several lines of discussion hereGot involved and stopped taking notes

Timed out

41

DONMs

February UCLIdentification of Common Patterns to Support

General business Outline of Report (Scope, Limits, Goals & Structure) Commitment to write Parts for Review at next meeting

Invite 2 or 3 e-Science visionary scientists for 2 hours Peter Coveney was agreed to be a good choice

Late March & early AprilImperial

May/JuneGoal: A report / published matrix / draft road map

By end of June 04