View
218
Download
1
Tags:
Embed Size (px)
Citation preview
A PPARC funded project
AstroGrid’s Common Execution Architecture
Guy Rixon, reporting on behalf of Paul Harrison and the other AstroGrid developersSC4DEVO-1, Pasadena, July 2004
AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 2
AstroGridUK national virtual-observatory projectSee cover slide for participants and fundingTasked to produce a VO toolkit to be reused by service providersCollective author of a SOA using web servicesCommon Execution Architecture: A major part of AstroGrid architecture Part of the service-oriented architecture A way of making and using web services Some important applications to data mining
AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 3
SOA for data mining
How can we carve up the distributed application into services?
And where do we put the application logic?
AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 4
Distributed app: conservativeDesktop
Application
Data-centre 1
Data source
Data-centre 2
Data transform
Application logic in here
AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 5
Distributed app: browser-based
Desktop
Browser
Data-centre 1
Data source
Data-centre 2
Data transform
Application logic in here
AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 6
Distributed app: intragrid
Desktop
Browser/app
Data-centre 1
Data source
Data-centre 2
Data transform
Compute grid
NodeNodeNodeNode
Some application logic in here;
Rest still in client or in user
AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 7
Distributed app: scripted
Desktop
Browser/app
Data source
Data-centre 2
Data transform
Data centre 3
Data transform
Data-centre 1
Portal
Workflowengine
Script
App logic in here
AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 8
Distributed app: embeddedDesktop
Browser
Data source
Data-centre 2
Data transform
Data mine
Application
Data-centre 1
Portal
Workflowengine
Script
Most app logic here
Little app logic here
AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 9
Distributed app: data gridDesktop
Browser/app
Data source
Data-centre 2
Data transform
Data centre 3
Data transform
Data-centre 1
Portal
Workflowengine
Script
AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 10
Issues to be addressedBoring, technical:
How to write apps as w/s?
How to write w/s clients?
How to connect apps to storage?
How to register functions?
Interesting, scientific:
Who has useful algorithms?
How to match algorithms to data?
Best place to run computations?
How to share code? Provenance? How to share kudos? Is more science
possible?
AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 11
AstroGrid architecture and products
AstroGrid bits and pieces let techies solve the tech problems so that scientists can concentrate on the science issues.AstroGrid supports: Browser-based app Scripted app Data grid Embedded app (pre-installed exes)
Common Execution Architecture lets all parts but exe for embedded app be reusable code.
AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 12
Common Execution Architecture (1)
Desktop
Browser/app
Data source
Data-centre 2
Data transform
Data centre 3
Data transform
Data-centre 1
Portal
Workflowengine
Script
Web service(CEC)
Application(local exe)
MySpace
Data mine
“CEC” = CommonExecutionConnector
CECconfiguration
AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 13
Common Execution Architecture (2)
Desktop
Browser/app
Data source
Data-centre 2
Data transform
Data centre 3
Data transform
Data-centre 1
Portal
Workflowengine
Script
Workflow executor (JES)
MySpace
Pagepresentation
Workflowdefinition
Portal
Scripts stored here,as files
AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 14
Common execution architecture (3)
Common language throughout system allows s/w reuse.
AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 15
How to write an app using CEA?Don’t write a web service!Write a command-line exeConfigure a CEC to run your app: I.e. describe parameters in the CEA
language
Or: get your app configured on someone else’s CEC
AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 16
How to publish an app using CEA?
Take the <ApplicationDefinition> that informs the CEC.Wrap it in a <VOResource> of sub-type CeaApplicationType.Publish the <VOResource> in IVO registry.
AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 17
How to call an app-service using CEA?
Look it up in the resource registry.Get the <ApplicationDefinition> from its <VOResource>.Give the <ApplicationDefinition> to a CEA delegate => define SOAP call.Add parameters from UI according to <ApplicationDefinition>.Bake in pre-heated SOAP engine @ HTTP 1.1.
AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 18
CEA language example<ApplicationDefinition>
<Parameters> <ParameterDefinition name="DetectionImage"
type="binary"> <UI_Name>Detection Image</UI_Name> <UI_Description>The image that is used to detect sources.
Basic position, shape and size information is derived from this image</UI_Description>
</ParameterDefinition> <ParameterDefinition name="PhotoImage" type="binary"> <UI_Name>Measurement Image<UI_Name> <UI_Description>The Image that is used to measure
photometric parameters</UI_Description>
</ParameterDefinition> <ParameterDefinition name="config_file" type="text"> …
AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 21
CEA facilities so far
PALCEC
HyperZCEC
Otherservice
MySpace
JESCEA Delegate Other Delegate
SIAPCEC
AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 22
Future evolutionAdd facilities to CEC: Asynchronicity Security More parameter types
New facility: field-programmable CEC (PCEC)
AstroGrid’s CEA; Guy Rixon; SC4DEVO-1; Pasadena, July 2004. Slide 23
PCEC (1): local code libraries
Client
PCEC
Data mine
Code library
Load exe
Select exe