CoursGrille3-Globus.pptJean-Marc Pierson December 2007
Ian Foster: “Resource sharing & coordinated problem solving in dynamic, multi-institutional virtual organizations”. http://www- fp.mcs.anl.gov/~foster/
“HW and SW infrastructure that provides dependable, consistent, pervasive and inexpensive access to high-end computational capabilities”
3
Grid Definition Refined Use Open Protocols Is decentralized Deliver non-trivial QoS
4
Service Oriented Architecture
Open Grid Service Architecture WSRF : Web Service Resource Framework Everything is resources, WS-Resources Access through services
2
5
VO Management Service: resources allocation to each Virtual Organization.
Resource Discovery and Management Service Job Management Service And much more: security (authentication,
authorisation, data management)…
All all services interact: example Job Management Service needs Resource Discovery
Need Standardization for interfaces to services Example: JobSubmissionService has a submitJob() method
6
architecture OGSA? The method invocation should also be
standardized. Corba? RMI? RPC? No : Web Services!!
But need Stateful Web Services!
WSRF: Web Services Resource Framework
7
Web Services in Grid? Pros:
XML, language and platform independence HTTP usage for transmitting messages, no
security problems with firewall
Cons: XML, overhead by the verbosity Lack of versatility, only basic invocation (no
other services provided, unless using higher level Web Services, role of WSRF)
3
9
specification Developed by the Globus Alliance,
www.globus.org
11
More inside Web services invocations
You don’t have to program the stubs/nor the SOAP requests/responses Just like Corba and RMI
15
Normal services, written in Java for instance
SOAP engine: to marshall/serialize requests/response
Ex: in GT4, it is Axis
Application Server: for instance Jakarta/Tomcat
Apache for instance
5
17
WS-Resources
Web Service + Resource = WS-Resources
To address these, we need a endpoint reference to specify the resource
18
WS-ResourceLifetime: manage lifecycle of the WS- Resources
WS-ServiceGroup: group services or WS-Resources together allow to find in the group services meeting a particular
property allow also to address all services of the group by one
entry point
19 20
Writing a WSRF Web/Grid Service
Five Steps, only ! 1. Define the service’s interface. This is done with
WSDL 2. Implement the service. This is done with Java. 3. Define the deployment parameters. This is done
with WSDD and JNDI 4. Compile everything and generate a GAR file.
This is done with Ant 5. Deploy service. This is also done with a GT4 tool
6
21
public interface Math {
}
22
targetNamespace="http://www.globus.org/namespaces/examples/core/MathService_in stance"
<operation name="add"> <input message="tns:AddInputMessage"/> <output message="tns:AddOutputMessage"/> </operation>
<operation name="subtract"> <input message="tns:SubtractInputMessage"/> <output message="tns:SubtractOutputMessage"/> </operation>
<operation name="getValueRP"> <input message="tns:GetValueRPInputMessage"/> <output message="tns:GetValueRPOutputMessage"/> </operation> </portType> </definitions>
24
<!--====== M E S S A G E S ======--> <message name="AddInputMessage">
<part name="parameters" element="tns:add"/> </message> <message name="AddOutputMessage">
<part name="parameters" element="tns:addResponse"/> </message>
<message name="SubtractInputMessage"> <part name="parameters" element="tns:subtract"/> </message>
<message name="SubtractOutputMessage"> <part name="parameters" element="tns:subtractResponse"/> </message>
<message name="GetValueRPInputMessage"> <part name="parameters" element="tns:getValueRP"/>
</message> <message name="GetValueRPOutputMessage"> <part name="parameters" element="tns:getValueRPResponse"/>
</message>
7
25
targetNamespace="http://www.globus.org/namespaces/examples/core/Ma thService_instance" xmlns:tns="http://www.globus.org/namespaces/examples/core/MathServ ice_instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<xsd:element name="subtract" type="xsd:int"/> <xsd:element name="subtractResponse"> <xsd:complexType/> </xsd:element>
<xsd:element name="getValueRP"> <xsd:complexType/> </xsd:element> <xsd:element name="getValueRPResponse" type="xsd:int"/> 26
<!-- RESOURCE PROPERTIES -->
<xsd:element name="MathResourceProperties"> <xsd:complexType> <xsd:sequence>
<xsd:element ref="tns:Value" minOccurs="1" maxOccurs="1"/> <xsd:element ref="tns:LastOp" minOccurs="1" maxOccurs="1"/> </xsd:sequence>
</xsd:complexType> </xsd:element>
</xsd:schema> </types>
http://www.globus.org/namespaces/examples/core/MathS ervice_instance=org.globus.examples.stubs.MathServ ice_instance
http://www.globus.org/namespaces/examples/core/MathS ervice_instance/bindings=org.globus.examples.stubs .MathService_instance.bindings
http://www.globus.org/namespaces/examples/core/MathS ervice_instance/service=org.globus.examples.stubs. MathService_instance.service
Notion of QName (Qualified Names)
8
29
package org.globus.examples.services.core.first.impl; import java.rmi.RemoteException;
30
/* Resource properties */ private int value; private String lastOp; /* Get/Setters for the RPs */ public int getValue() { return value; } public void setValue(int value) { this.value = value; } public String getLastOp() { return lastOp; } public void setLastOp(String lastOp) { this.lastOp =
lastOp; }
/* Required by interface ResourceProperties */ public ResourcePropertySet getResourcePropertySet() { return this.propSet; }
Caution: Properties have same names than in WSDL, Beware of uppercase/lowercase
31
try { ResourceProperty valueRP = new ReflectionResourceProperty( MathQNames.RP_VALUE, "Value", this); this.propSet.add(valueRP); setValue(0);
ResourceProperty lastOpRP = new ReflectionResourceProperty( MathQNames.RP_LASTOP, "LastOp", this); this.propSet.add(lastOpRP); setLastOp("NONE");
} catch (Exception e) { throw new RuntimeException(e.getMessage());}
} 32
value += a; lastOp = "ADDITION"; return new AddResponse(); }
public int getValueRP(GetValueRP params) throws RemoteException {
return value; }
9
33
To: public MultiplyResponse multiply(Multiply params)
throws RemoteException { int a1 = params.getA1() int a2 = params.getA2() // Do something return new MultiplyResponse(); }
Boxing return type if void or complex type
Boxing parameters if number is not one, or not simple type
34
Configuring the deployment With WSDD and JNDI
Deploying into a container
This will translate to http://localhost:8080/wsrf/services/examples/core/first/MathService <service name="examples/core/first/MathService" provider="Handler" use="literal"
value="org.globus.examples.services.core.first.impl.MathService"/> Automatically generated from Math.wsdl written before. <wsdlFile>share/schema/examples/MathService_instance/Math_service.wsdl</wsdlFile> <parameter name="allowedMethods" value="*"/> <parameter name="handlerClass" value="org.globus.axis.providers.RPCProvider"/> <parameter name="scope" value="Application"/> <parameter name="providers" value="GetRPProvider"/> Resource Property provider <parameter name="loadOnStartup" value="true"/> </service>
</deployment> 36
<?xml version="1.0" encoding="UTF-8"?> <jndiConfig xmlns="http://wsrf.globus.org/jndi/config">
<service name="examples/core/first/MathService"> <resource name="home"
type="org.globus.wsrf.impl.ServiceResourceHome"> <resourceParams> <parameter> <name>factory</name> <value>org.globus.wsrf.jndi.BeanFactory</value> </parameter> </resourceParams> </resource> </service> </jndiConfig>
10
37
Generating a GAR file: Grid Archive
What to be done: 1. (Pre-) Processing the WSDL file to add
missing pieces (such as bindings) 2. Creating the stub classes from the WSDL 3. Compiling the stubs classes 4. Compiling the service implementation 5. Organize all the files into a very specific
directory structure 38
script for this!
./globus-build-service.sh -d <service base directory> -s <service’s WSDL file>
39
Using the globus-deploy-gar command $ globus-deploy-gar name_of_gar.gar
undeployment: globus-undeploy-gar
40
// Create endpoint reference to service+resource EndpointReferenceType endpoint = new EndpointReferenceType(); endpoint.setAddress(new Address(serviceURI)); MathPortType math = locator.getMathPortTypePort(endpoint); math.add(10); math.subtract(5); System.out.println(“Value: "+ math.getValueRP(new GetValueRP()));
} catch (Exception e) {e.printStackTrace(); }}}
$ source $GLOBUS_LOCATION/etc/globus-devel-env.sh $ javac -classpath
$ java -classpath ./build/stubs/classes/:$CLASSPATH org.globus.examples.clients.MathService_instance. Client http://127.0.0.1:8080/wsrf/services/examples/core /first/MathService
Value: 5 // if we call again, we will have: Value:10, then
Value: 15, … The grid service is stateful! 42
From a service+resource embedded
to … service/resource home/resource
Remember : in JNDI file, we had <resource name="home" type="org.globus.wsrf.impl.ServiceResourceHome">
44
implementation will change !! Utility class Qname: no change, except the
package name ;) Resource class: same as previous service
one, except removing the operations (add/substract/…), and the Resource Property initialization is changed from Constructor to a method initialize()
12
45
no longer there public AddResponse add(int a) throws RemoteException { MathResource mathResource = getResource(); mathResource.setValue(mathResource.getValue() + a); mathResource.setLastOp("ADDITION"); return new AddResponse(); } private MathResource getResource() throws RemoteException { Object resource = null; try { resource =
ResourceContext.getResourceContext().getResource(); } catch (Exception e) { throw new RemoteException("", e); } MathResource mathResource = (MathResource) resource; return mathResource; } 46
Resource Home
47
Deployment and test
No change in WSDD, except the names of service and class ;)
In JNDI, change to indicate the new service name and its associated Resource Home:
<service name="examples/core/singleton/MathService">
<resource name="home" type="org.globus.examples.services.core.singleto n.impl.MathResourceHome">
Creating GAR file, deploying GAR file, testing with the same client !
48
13
49
New needs
Factory is a service that must be developed and deployed, just as the normal one: write WSDL, Java implementation
Service: no change (quick look back) Resource: must implement a ResourceIdentifier
interface (getId() method), meaning giving an identifier (any type here) to the resource
Resource Home: only change to extend ResourceHomeImpl able to handle multiple resources
WSDD and JNDI file must be changed to integrate new Factory service, and the link between Factory and Service: in JNDI, we have now:
<resource name="home" type="org.globus.examples.services.core.factory.impl.MathResourceHome">
EndpointReferenceType factoryEPR, instanceEPR; FactoryPortType mathFactory; MathPortType math;
factoryEPR = new EndpointReferenceType(); factoryEPR.setAddress(new Address(factoryURI)); mathFactory = factoryLocator.getFactoryPortTypePort(factoryEPR);
CreateResourceResponse createResponse = mathFactory.createResource(new CreateResource());
math = instanceLocator.getMathPortTypePort(instanceEPR); math.add(10); math.subtract(5);… 52
More about Resource Properties: WS-ResourceProperties interface
GetResourceProperty: to retrieve a property value (same role as the get accessor)
GetMultipleResourceProperties SetResourceProperties: possibility to update,
insert or delete a RP QueryResourceProperties: with a Xpath
syntax, since RP are in XML document
14
53
xmlns:tns="http://www.globus.org/namespaces/examples/core/MathService_instance”> <tns:Value>10</tns:Value> <tns:Value>30</tns:Value> <tns:Value>50</tns:Value> <tns:Value>40</tns:Value> <tns:LastOp>ADDITION</tns:LastOp> <tns:LastOp>ADDITION</tns:LastOp> <tns:LastOp>ADDITION</tns:LastOp> <tns:LastOp>SUBTRACTION</tns:LastOp> </MathResourceProperties>
54
Change in the WSDL file, adding: <portType name="MathPortType" wsdlpp:extends="wsrpw:GetResourceProperty wsrpw:GetMultipleResourceProperties wsrpw:SetResourceProperties wsrpw:QueryResourceProperties" wsrp:ResourceProperties="tns:MathResourceProperties">
Change in the WSDD file, changing the providers line to <parameter name="providers" value="GetRPProvider GetMRPProvider
SetRPProvider QueryRPProvider"/>
valueRP = math.getResourceProperty(MathQNames.RP_VALUE); lastOpRP = math.getResourceProperty(MathQNames.RP_LASTOP);
value = valueRP.get_any()[0].getValue(); lastOp = lastOpRP.get_any()[0].getValue(); System.out.println("Value RP: " + value); System.out.println("LastOp RP: " + lastOp);
Since what is returned is a XML document, with potentially different values, we have to access the right one
56
Some already existing services, deployed in a “container” : globus-start-container
Some libraries/tools to falicitate the development of new services and their deployment : globus-deploy-gar
Some command line tools for managing the jobs on the grid : grid-proxy-init, globusrun-ws, …
15
57
Using gridFTP in GT4
globus-url-copy : copying from one site to another, but gridFTP needs to run on both machine.
$ globus-url-copy -vb gsiftp://raisin.irit.fr/etc/issue file:/tmp/foo
58
remote storage element(s)
Command example: % globusrun-ws -submit -c /bin/touch touched_it Submitting job...Done. Job ID: uuid:002a6ab8-6036-11d9-bae6- 0002a5ad41e5 Termination time: 01/07/2005 22:55 GMT Current job state: Active Current job state: CleanUp Current job state: Done Destroying job...Done.
60
Complete Factory Contact
•Override default EPR Select a different host/service : here the service is located on machine raisin.irit.fr, on port 8443 Relies on proprietary knowledge of EPR format!
•Command example: % globusrun-ws -submit –F https://raisin.irit.fr:8443/wsrf/services/Manag edJobFactoryService -c /bin/touch touched_it
Contents of touch.xml file: <job> <executable>/bin/touch</executable> <argument>touched_it</argument> </job>
62
% globusrun-ws -monitor -j job_epr job state: Active Current job state: CleanUp Current job state: Done Requesting original job description...Done. Destroying job...Done.
63
</job>
<rftOptions> <subjectName>
</subjectName>
</job>
66
UUID of the job resource, for job reliability/recoverability
Subscribe Request Client can include a request to subscribe for job state notifications with
the job submission to avoid an extra operation call
Job Description / RSL Either a single or multi-job description
Output: newTerminationTime - new termination time of the job
resource managedJobEndpoint - EPR of the newly created job resource subscriptionEndpoint - EPR of the notification subscription
67
release operation Release a holdState set in the job description
Only one hold state can be set/released
Input: None Output: None
State change notifications State - job state (Active, Pending, Done, Cleanup…) Fault - fault causing a Failed state (if applicable) Exit Code - exit code of the job process Holding - boolean indicating if the job is in a hold state
68
ManagedExecutableJobService Executes the requested job process(es)
specified in the RSL Resource Properties
(ManagedExecutableJobPortType) serviceLevelAgreement - the RSL / Job Description state - the current job state faults - the fault causing a Failed state localUserId - the username of the resource owner userSubject - the GSI subject of the resource owner holding - boolean indiciating the job is holding stdoutURL - the GridFTP URL to the stdout file stderrURL - the GridFTP URL to the stderr file credentialPath - the local path to the user proxy file exitCode - the exit code of the job proces (if applicable)
18
69
submits the sub-jobs to the specified ManagedJobFactoryService.
Sub-jobs cannot be multi-jobs themselves.
Resource Properties (ManagedMultiJobPortType) serviceLevelAgreement - the multi-job RSL / Job Description state - the current overall state faults - the fault causing a Failed state localUserId - the username of the resource owner userSubject - the GSI subject of the resource owner holding - boolean indiciating all jobs are holding subJobEndpoint - list of endpoints to the sub-jobs
70
Architecture. Ian Foster, Carl Kesselman
The Globus Toolkit 4 Programmer’s Tutorial, Borja Sotomayor
www.globus.org