Upload
lesley-gaines
View
225
Download
1
Embed Size (px)
Citation preview
The Common Component Architecture and XCAT
Indiana UniversityExtreme! Lab
What is a Component Architecture?
• A systematic way of encapsulating special functionality and behaviors of a piece of software into reusable units.
• More than an object model.• It defines the rules for the way objects can be
instantiated and composed.• It defines the environment of services that a
component can use.• It defines the way in which we discover how to interact
with them.
Isn’t this what a subroutine library does?
– Subroutine libraries or class libraries have• well defined interfaces• encapsulate functionality
– But are often hard to reuse because they• often have complex resource requirements• make conflicting assumptions about their operating
environments.
– Subroutine libraries that follow very strict sets of design and use rules get reused.
• Above all else, a component architecture is framework of standard rules of behavior for objects.
Standard Component Archtectures
• Examples: – Microsoft COM/DCOM, COM++
• the foundation of all Microsoft office products
– Java Beans• Java standard for building user interfaces
– Enterprise Java Beans• a standard for client-server applications in Java
– CORBA 3.0• The new component spec for CORBA.
Look at One Design in Detail
• The U.S. Dept of Energy DOE2000 project– The Common Component Architecture
• Lawrence Livermore National Lab• Sandia Labs• Argonne National Labs• Los Alamos National Labs• Universities: Utah, NCSA, Indiana
– A specification for component design for parallel and distributed applications
Two parts to the CCA Architecture
• Components– An encapsulated “object” defined by its public
interfaces.– Can be Java, C, Python, C++ or C++ and Fortran.
• Frameworks– The software systems that provides basic services
to components– Used to compose components to make apps.– Many implementations: Parallel and Distributed.
Some Concepts in Detail
• Ports: the public interfaces of a component– defines the different ways we can interact with a
component and the ways the component uses services and other components.
Image ProcessingComponent
setImage(Image I)
Image getImage()
adjustColor()
setFilter(Filter)calls doFFT(…)
Provides Ports - interfacesfunctions provided by component
Uses Ports - interface of aservice used by component
How do you “compose” two components?
• There are three basic approaches:– Events: A component can generate an “event” of a particular
type. Other components are added as “listeners” for such events from that component.
– Services: A source component “uses” services “provided” by a target components.
• “use” = calls a function provided by the target.
– Dataflow: A typed data “output” stream is connected to a similarly typed “input” port of another.
• In XCAT all three are supported.– The services model is the basic mechanism. – Dataflow is emulated in CCA by “pushing” data from a user to
a provider, or by having a user “pull” data from a provider.– Events are managed by a separate mechanism.
AcmeFFT
component
Building Applications by Composition
• Connect uses Ports to Provides Ports.
Image ProcessingComponent
getImage()
adjustColor()
Image tool graphical interface component
Imagedatabase
component
setImage(…)
doFFT(…)
Ports and Interfaces (CCA)
• Component composition:– Connect (possibly at runtime) a “provides” port of one
component to a “uses” port of another.– A provides port is simply an implementation in the
component of the interface defined by the port type (interface defn.).
– A Uses port is a “proxy” that a component “uses” to make a request of another component.
SourceComponent
TargetComponent
A Provides portA uses port
How do you describe the interfaces a port has?
• Standard Methods:– Use an Interface Definition Language - a formal
description of the interface objects. • CORBA IDL is one.• The LLNL Scientific IDL by Scott Kohn, Andrew Cleary,
Steven Smith, and Brent Smolinski is better for CCA.
– A simple IDL compiler can be used to automatically generate the Uses and Provides Port code for a specific IDL type.
– For the XCAT java version the interface of a component port are defined as Java interfaces.
Component Communication
• Use a simple Remote Procedure Call Mechanism– XSOAP
• Implementation of Simple Object Access Protocol (SOAP)
• Performance Issues
– in-process calls– Events/Messages
• Objects encoded as XML documents
– Proteus• Multiprotocol Messaging and RMI
X
Creation Service
• Creates a running instance of another component
• Encapsulates authentication issues
CreationService
Component
Launch an instance ofcomponent X on resource Y
Returns: remote referenceto new component instance
Globus resource Y
Connection Service
• A component that can be used to connect a “uses” port of one component to the “provides” port of another
• Can export ports of another component
Y
ConnectionService
Component
X
Connect port A of component Xto port B of component Y
A
B
Builder Service
• combination of Creation and Connection service
• standardized as part of recently updated CCA specification
• we are now in process of implementing it in XCAT
Simple Example in Java
public class SimpleEchoImpl implements SimpleEcho
{
public String echoHello(String s) throws RemoteException { return "SimpleEchoImpl says: Hello " + s; }}
Simple Example in Java
public class EchoPrinterComponentimplements Component
{public void setServices(Services cc){
PortInfo portType = new PortInfoImpl( "simpleEchoProvidesPort", "http://example.com/echo.wsdl");
simpleEchoImpl = new SimpleEchoImpl(); cc.addProvidesPort(simpleEchoImpl,portType); }}
Simple Example in Java
public class EchoGeneratorComponent implements Component
{public void setServices(Services cc){
PortInfo portType = new PortInfoImpl( "simpleEchoUsesPort", "http://example.com/echo.wsdl"); cc.registerUsesPort(portType); }}
Simple Example in Java
SimpleEcho usesSimpleEcho = (SimpleEcho) cc.getPort("simpleEchoUsesPort");usesSimpleEcho.echoHello("Extreme Lab");cc.releasePort(“simpleEchoUsesPort");
Echo
Generator
Component
EchoPrinter
Component
A Provides portsimpleEchoProvidesPort
A uses portsimpleEchoUsesPort
Scripting XCAT Applications
import xcatgenerator = xcat.createComponent(‘GeneratorComponet’)printer = xcat.createComponent(‘Printer’)
xcat.setCreationMechanism(generator, ‘gram’)xcat.setCreationMechanism(printer, ‘ssh’)
xcat.createInstance(generator)xcat.createInstance(printer)
xcat.connectPorts(generator, ‘simpleEchoUsesPort’, printer, ‘simpleEchoProvidesPort’)
Scripting XCAT Applications
generatorComponent = EnvObj()generatorComponent.put("exec-name",
"simpleGeneratorComponent")generatorComponent.put("exec-fqn",
"samples.simpleEchoGenerator.SimpleEchoGeneratorComponent")
printerComponent = EnvObj()printerComponent.put("exec-name",
"simplePrinterComponent")printerComponent.put("exec-fqn",
"samples.simpleEchoPrinter.SimpleEchoPrinterComponent")
# create component wrappersgenerator =
cca.createComponent(generatorComponent)
printer = cca.createComponent(printerComponent)
# assign a machine nameprintermc = "exodus.extreme.indiana.edu"generatormc = "exodus.extreme.indiana.edu"cca.setMachineName(generator,generatormc)cca.setMachineName(printer, printermc)
# create live instancescca.createInstanceWithTimeOut(printer, 120000)cca.createInstanceWithTimeOut(generator,
120000)
# connect their portscca.connectPorts(generator,
"simpleEchoUsesPort", printer, "simpleEchoProvidesPort")
# start the components – invoke go() on GoPort
usesGoPortClassName = "samples.idl.goPort.UsesGoPort"
usesPortType = "http://example.com/go.wsdl"providesPortName = "providesGoPort"methodName = "go"methodParams = zeros(0, Object)cca.invokeMethodOnComponent(generator,
usesGoPortClassName, usesPortType, providesPortName, methodName, methodParams)
print "Done"
Scientific Components and Applications
• Recognized as one of the “Top Ten Science Achievements in 2002” by the DOE Office of Science (see http://www.sc.doe.gov/sub/accomplishments/top_10.htm)
• Many demonstrated at SC2001 and SC2002• Combine application-specific components with more
general-purpose components that can be reused across a range of applications– More than 40 components, many reused in apps such as
• PDEs on unstructured and adaptive structured meshes• Unconstrained minimization problems
• Leverage and extend parallel software developed at different institutions– Including CUMULVS, CVODES, Global Arrays, GrACE, MPICH,
PETSc, PVM, and TAO
Current Interface Development Activities
CCA Scientific Data Components Working Group
• Basic Scientific Data Objects– Lead: D. Bernholdt (ORNL)
• Unstructured Meshes– Lead: L.F. Diachin (SNL, formerly
ANL)– Collaboration with TSTT ISIC– TSTT = Terascale Simulation Tools
and Technologies, PIs: J. Glimm, D. Brown, L.F. Diachin, http://www.tstt-scidac.org
• Structured Adaptive Mesh Refinement– Lead: P. Colella (LBNL)– Collaboration with APDEC ISIC– APDEC = Algorithmic and Software
Framework for Applied PDEs, PI: P. Colella, http://davis.lbl.gov/APDEC
Other Groups
• Linear and nonlinear solvers, eigensolvers, and optimizers– Coordinator: L.C. McInnes (ANL)– Collaboration with TOPS ISIC– TOPS = Terascale Optimal PDE
Simulations, PI: D. Keyes, http://tops-scidac.org
• MxN Parallel Data Redistribution– Lead: J. Kohl (ORNL)– Part of CCTTSS MxN Thrust
• Quantum Chemistry– Leads: C. Janssen (SNL) and T.
Windus (PNNL)– Part of CCTTSS Applications
Integration Thrust
Motivation for Common Interfaces• Many-to-Many couplings
require Many 2 interfaces– Often a heroic effort to
understand details of both codes– Not a scalable solution
• Common Interfaces: Reduce the Many-to-Many problem to a Many-to-One problem– Allow plug-and-play
interchangeability & interoperability
– Require domain specific experts– Typically difficult & time-
consuming– A success story: MPI– Challenges
• Interface agreement• Functionality limitations• Maintaining performance
NWGrid
Overture
MDB/CUBIT
AOMDHypre
PETSc
SuperLU
mesh libraries
linear solverlibraries
NWGrid
Overture
MDB/CUBIT
AOMD
SuperLU
PETSc
Hypre
Data
Others …
Others …
Solvers
TSTTData
Interfaces
TOPSSolver
Interfaces
Framework Stays “Out of the Way”
of Component Parallelism•Single component multiple
data (SCMD) model is component analog of widely used SPMD model
•Each process loaded with the same set of components wired the same way
P0 P1 P2 P3
Components: Blue, Green, Red
Framework: Gray
MCMD/MPMD also supported
•Different components in same process “talk to each” other via ports and the framework
•Same component in different processes talk to each other through their favorite communications layer (i.e., MPI, PVM, GA)
There are 3 CCA Frameworks
• There’s 3 parallelism models in scientific computing
Ccaffeine
SPMDDistributed
Threaded
Uin
tah
Ccaffeine
• SPMD• GUI and Scripted
Interface• Interactive or Batch• Serial or Parallel• Components written in
C++
• Separates CCA Pattern from Implementation– 3 Bindings
• Classic Components• SIDL Components• Chasm Components
• Demonstrated at SC2001 & SC2002
• Used in CCA Tutorials
Characteristics Achievements
SCIRun/BioPSE/Uintah
• multithreaded & distributed
• C++ only • Used in “real science”
(gov’t, academic, commercial) 1K procs
• Testbed for CCA Concepts• Novel work in IDL-based MxN • Parallelism and concurrency
research for CCA done on Uintah
• SCIRun2 will integrate SCIRun, BioPSE and Uintah
Characteristics Achievements
XCAT
• Distributed/Grid model• Web Services
• Developed Proteus multi-protocol communication package
• Novel MxN work at MPI-I/O level
• Java and C++ components
Characteristics Achievements
Material Archive
GoData
GSCntl
GoGoGS
GSGo
Data Provider CompSimulationComponent
exported
exported
exportedApplicationCoordinator
sendParameters, start, killApplication
Specific componentApplication
FactoryService
Directory/Discovery Service 1
23
AuthorizationService
ResourceBrokerService
45
6 7 wsdl
Ensemble Application
8 Application control
A Science Portal View of “programming” the Grid
• A Science Portal is a Web server that– Uses a “prepackaged” set of scripts to
• Get the users proxy cert from a cert repository• Configure a specific application to users needs• Contact the appropriate application factories
– Look for event histories of application execution
– Allow the user to contact and control the app.
User Portals/ Science Portals
Launch, configureAnd control
Appfactory
AppInstance
Application Factory Service• A service that understands a
how to instantiate a running instance of an app component.– You provide it with appropriate
requirements initial conditions, etc. via an XML file
– The service• checks you credentials and
authorization• May consult resource broker• launches the app or runs the
appropriate grid script.
Appfactory
AppInstance
Provide me with an instance of application X
Application Component Execution Environment Specification
<componentStaticInformation> <componentInformation> <uniqueID>WebsterComponent</uniqueID> <name>Webster Component</name> <author>Indiana University Extreme! computing</author> <componentAuthor>Dennis Gannon</componentAuthor> </componentInformation> <executionEnv> <hostName>olympus.cs.indiana.edu</hostName> <hostName>linbox2.extreme.indiana.edu</hostName> <hostName>rainier.extreme.indiana.edu</hostName> <creationProto>gram</creationProto> <creationProto>ssh</creationProto> <nameValuePair> <name>exec-dir</name> <value>/u/gannon/xcat_tutorial/scripts</value> </nameValuePair> ….
The information neededto generate input to the factory service
Steps in creating a app instance from the portal to the factory
• User Configures Application from Web Browser
• Sets application parameters• Launches job
User Portals/ Science Portals
WebServerWeb
Server
• Web Server contacts appropriate application factory service (FS)• Supplies FS with task parameters • FS contacts Resource Broker and secures job
launches.• Returns App WSDL to server to browser
•Job begins execution •Publishes its contact point (WSDL) to discovery service•Begins publishing status events to event channel.
•Web server discovers application •Allows user to interrogate it or retrieve event traces
Appfactory
ResourceBroker
AppInstance
Eventchannel
Discserv
An Application Instance• Typical Instance
– Publishes event stream to “well known” channel.
– Has a Control Interface to allow portal level interrogation or remote steering
– May be linked to other components/services
LinksToOtherApps/services
Control Interface• check status• control messages
ApplicationInstance
Event stream
Encapsulating Legacy Apps• Common Case
– Legacy App that reads files and writes files.
– Use a “scripted component” called an app manager.
• Component runs a python script loaded at startup or through a control command
– The App Script• Stages files• Launches and monitors
application• Writes Output files• publishes event streams
application
App Mgr
inputfiles
output files
App Script
EventStream
Control
XCAT And OGSI
• A component based model to the Grid services– Component Assembly: Composition in space– Workflow: Composition in time
• Ability to use widely available Web services tooling to interact with components
• A richer messaging and notification system that extends the model proposed by OGSI
• An application factory service that extends the standard OGSI factory service
XCAT-OGSI Big Picture
XCAT Component
GridService Port(for lifecycle and metadata)
Other XCAT Provides Ports
ComponentID (GSH)(uniquely identifiesComponent)
ServiceData(includes list ofPort references)
Incorporating OGSI into XCAT
• Converting an XCAT component to a Grid service– Addition of an OGSI GridService Port– Addition of Service Data Elements (SDEs) containing
references to each of the other ports– Uniquely identifying the component using a
ComponentID (GSH)– Using a WSDL representation for the GSR
• Adding a set of helper services that are OGSI compliant– A HandleResolver service for mapping a GSH to a GSR– Modifications to the XCAT Creation service to make
appropriate calls to the Factory and GridService ports for lifetime management
XCAT Vs OGSI Messaging
• OGSI messaging is simple, point-to-point, and non-reliable, whereas XCAT uses XMessages that provides a reliable, persistent network of message channels suited for application level messaging and events
• OGSI messaging is push based, and only current moment state can be pulled by accessing SDEs whereas XMessages can be both push and pull based
• In XCAT, clients can have more mobility and interoperate more easily with firewalls
• there is no assumption that client has IP reachable address
Factories in XCAT and OGSI
• OGSI provides a standard Factory Port type for instantiating services
• XCAT generalizes this Factory Port type to instantiate a set of XCAT components that constitute an application– Accepts a description of a connected network of
components– Launches an Application Coordinator for each application
that creates, and links together instances of the described network of components
– Exports a subset of the functionality of the components, which are useful for the application, to the end-user
– Enables creating and managing complex distributed applications