Spring Web Service, Spring Integration and Spring Batch

Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited.

Spring Web Service, Spring Integration and Spring Batch

Eberhard Wolff Principal & Regional Director

SpringSource Germany

Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 2

About the presentation...

•  Take a concrete project... •  How can you solve your problems (using the Spring

Portfolio)?

•  You will see new Spring technologies in action... •  You will get a different impression about Spring... •  You will see how to do Web Services, EAI... •  ...and even batches


The Case


Build an Application

•  Should process orders •  Orders may come in as a file •  Or with a web service •  Express orders are processed immediately •  Other orders in a batch at night for the

next day


Architecture

•  I don't do Model Driven Development – Sorry, Markus

•  I don’t do PowerPoint architectures

•  I have something far better...


Architecture


Web Services


How you could have done it...

•  Generate a service POJO using Java

•  Export it via XFire •  Clients use WSDL generated from the interface

•  Drawbacks: –  XFire is deprecated and you depend on how it generates the

WSDL –  Exposes an interface that might be used internally –  ...and makes it unchangeable because external clients depend

on it –  Types like java.util.Map cannot be expressed in WSDL, so

work arounds must be used


Contract First

•  Contract First: Define the interface before the implementation

•  Contract First is the key to interoperability

•  Web Services are used for interoperability •  Not using Contract First with Web Services is therefore

unwise (almost a contradiction)

•  Also good to organize projects –  Decide about the interface

–  Start coding the implementation –  ...and the client

•  That used to work well for CORBA in the last century...


So let’s use WSDL

•  Send an order with some order items

•  80 lines of WSDL in Eclipse formatting

•  Show in 5 point font here

•  Just too much code

<?xml version="1.0" encoding="UTF-8"?><wsdl:definitions xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/"xmlns:sch="http://www.springsource.com/order"xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"xmlns:tns="http://www.springsource.com/order"targetNamespace="http://www.springsource.com/order"><wsdl:types><schema xmlns="http://www.w3.org/2001/XMLSchema"elementFormDefault="qualified"targetNamespace="http://www.springsource.com/order"><complexType name="Order"><sequence><element maxOccurs="1" minOccurs="0" name="express"type="boolean" /><element maxOccurs="unbounded" minOccurs="1"name="order-item" type="tns:OrderElement" /></sequence></complexType><complexType name="OrderElement"><all><element maxOccurs="1" minOccurs="1" name="count"nillable="false" type="positiveInteger" /><element maxOccurs="1" minOccurs="1" name="item"nillable="false" type="string" /></all></complexType><element name="orderRequest"><complexType><sequence><element name="order" type="tns:Order" /></sequence></complexType></element><element name="orderResponse"><complexType><sequence><element name="result" type="string" /></sequence></complexType></element></schema></wsdl:types><wsdl:message name="orderResponse"><wsdl:part element="tns:orderResponse" name="orderResponse"></wsdl:part></wsdl:message><wsdl:message name="orderRequest"><wsdl:part element="tns:orderRequest" name="orderRequest"></wsdl:part></wsdl:message><wsdl:portType name="Order"><wsdl:operation name="order"><wsdl:input message="tns:orderRequest"name="orderRequest"></wsdl:input><wsdl:output message="tns:orderResponse"name="orderResponse"></wsdl:output></wsdl:operation></wsdl:portType><wsdl:binding name="OrderSoap11" type="tns:Order"><soap:binding style="document"transport="http://schemas.xmlsoap.org/soap/http" /><wsdl:operation name="order"><soap:operation soapAction="" /><wsdl:input name="orderRequest"><soap:body use="literal" /></wsdl:input><wsdl:output name="orderResponse"><soap:body use="literal" /></wsdl:output></wsdl:operation></wsdl:binding><wsdl:service name="OrderService"><wsdl:port binding="tns:OrderSoap11" name="OrderSoap11"><soap:addresslocation="http://localhost:8080/order-handling/services" /></wsdl:port></wsdl:service></wsdl:definitions>


So?

•  We want Contract First...

•  ...but not with WSDL •  We mostly care about the data format •  ...which is defined with XML Schema

•  Spring Web Services lets you focus on the XML Schema


WSDL vs. XSD

•  34 lines are XSD

•  These actually define the data on the wire

•  Easy to deduce from XML sample messages –  Tool support: XML Spy, Trang, Microsoft XML

to Schema …

•  ...and can be used for POX (Plain Old XML) without SOAP as well

•  The rest is SOAP binding/ports/operations

•  Can the WSDL be generated?

<?xml version="1.0" encoding="UTF-8"?><wsdl:definitions xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/"xmlns:sch="http://www.springsource.com/order"xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"xmlns:tns="http://www.springsource.com/order"targetNamespace="http://www.springsource.com/order"><wsdl:types><schema xmlns="http://www.w3.org/2001/XMLSchema"elementFormDefault="qualified"targetNamespace="http://www.springsource.com/order"><complexType name="Order"><sequence><element maxOccurs="1" minOccurs="0" name="express"type="boolean" /><element maxOccurs="unbounded" minOccurs="1"name="order-item" type="tns:OrderElement" /></sequence></complexType><complexType name="OrderElement"><all><element maxOccurs="1" minOccurs="1" name="count"nillable="false" type="positiveInteger" /><element maxOccurs="1" minOccurs="1" name="item"nillable="false" type="string" /></all></complexType><element name="orderRequest"><complexType><sequence><element name="order" type="tns:Order" /></sequence></complexType></element><element name="orderResponse"><complexType><sequence><element name="result" type="string" /></sequence></complexType></element></schema></wsdl:types><wsdl:message name="orderResponse"><wsdl:part element="tns:orderResponse" name="orderResponse"></wsdl:part></wsdl:message><wsdl:message name="orderRequest"><wsdl:part element="tns:orderRequest" name="orderRequest"></wsdl:part></wsdl:message><wsdl:portType name="Order"><wsdl:operation name="order"><wsdl:input message="tns:orderRequest"name="orderRequest"></wsdl:input><wsdl:output message="tns:orderResponse"name="orderResponse"></wsdl:output></wsdl:operation></wsdl:portType><wsdl:binding name="OrderSoap11" type="tns:Order"><soap:binding style="document"transport="http://schemas.xmlsoap.org/soap/http" /><wsdl:operation name="order"><soap:operation soapAction="" /><wsdl:input name="orderRequest"><soap:body use="literal" /></wsdl:input><wsdl:output name="orderResponse"><soap:body use="literal" /></wsdl:output></wsdl:operation></wsdl:binding><wsdl:service name="OrderService"><wsdl:port binding="tns:OrderSoap11" name="OrderSoap11"><soap:addresslocation="http://localhost:8080/order-handling/services" /></wsdl:port></wsdl:service></wsdl:definitions>


XSD: Domain Types

<schema ...> <complexType name="Order"> <sequence> <element name="customer-number" type="integer" /> <element name="express" type="boolean" minOccurs="0" /> <element name="order-item" type="tns:OrderElement" maxOccurs="unbounded" /> </sequence> </complexType> <complexType name="OrderElement"> <all> <element name="count" type="positiveInteger" /> <element name="item" type="string" /> </all> </complexType>

Note: Optional elements and positive integers cannot be expressed in Java i.e. this is more expressive


XSD: Request and Response

<element name="orderRequest"> <complexType> <sequence> <element name="order" type="tns:Order" /> </sequence> </complexType> </element> <element name="orderResponse"> <complexType> <sequence> <element name="result" type="string" /> </sequence> </complexType> </element></schema> WSDL can now be easily generated:

The operation is essentially defined


Benefits

•  Contract First becomes an option

•  You can define the data format on the wire –  Interoperability –  Full power of XML Schema

•  You are not tied to SOAP but can also use POX •  ...potentially with different transport like JMS


How do you handle the message? •  Use a Object / XML mapper (OXM)

–  I.e. JAXB, XStream, ...

•  ...and make the request / response a Java Object •  Then handle it (much like a controller in MVC)

•  Easy •  Adapter between external representation and internal

•  Benefit: Decoupling business logic from changes of interface


How do you handle the message? •  Class for Endpoint needs only some annotations •  Will be instantiated as a Spring Bean •  (Almost) no Spring configuration needed (Spring!=XML) •  ...but can be done in XML as well •  Spring==Freedom of Choice @Endpointpublic class OrderEndpoint { @PayloadRoot(localPart = "orderRequest", namespace = "http://www.springsource.com/order") public OrderResponse handleOrder(OrderRequest req) { ... }}


Problems...

•  Robustness Principle (aka Postel’s Law): –  Be conservative in what you do; be liberal in what

you accept from others. –  I.e. try to accept every message sent to you

–  I.e. only require the data filled that you really need –  ...but only send complete and totally valid responses

•  Some Object / XML support this –  but for some the XML must be deserializable into

objects


XPath to the Rescue

•  Only needed XML parts are read

@Endpointpublic class OrderEndpoint { @PayloadRoot(localPart = "orderRequest", namespace = "http://www.springsource.com/order") public Source handleOrder( @XPathParam("/tns:orderRequest/tns:order/tns:order-item") NodeList orderItemNodeList, @XPathParam("/tns:orderRequest/tns:order/tns:express/text()") Boolean expressAsBoolean, @XPathParam("/tns:orderRequest/tns:order/tns:customer/text()") double customer) { ... }}


Web Services

•  Using Spring Web Services you can... –  ...use Contract First without the WSDL overhead –  ...decouple the business logic from interface and clients –  ...implement robust Web Services using XPath easily –  ...or with an Object/XML mapper (less robust but easier)

•  Currently in 1.5.6 –  1.5 introduced jms-namespace, email transport, JMS

transport...


Architecture


The core


The core

•  Essentially an integration issues

•  A Batch or JMS online output •  A Web Services or File input •  Internal routing and handling

•  Best practice: Pipes and Filter –  Pipes transfer messages –  ...and store / buffer them –  Filter handle them (routing, etc.)

•  Spring Integration supports this


Pipes and Filter: Frontend

incomingfile

fulfillment Web Service

File FileParser

Was just covered


Spring Integration Configuration for Frontend

Directory is polled, file name read Then content is put as a string in the incomingfile channel

incomingfile


File FileParser

<file:inbound-channel-adapter id="incomingfilename" directory="file:/tmp/files"> <poller> <interval-trigger interval="1000" /> </poller></file:inbound-channel-adapter>

<file:file-to-string-transformer delete-files="true" input-channel="incomingfilename" output-channel="incomingfile" />


Parsing the file Class automatically instantiated as a Spring Bean

MessageChannel with matching name is injected

Method called each time a message is available on the channel incomingfile

incomingfile


File FileParser

@MessageEndpointpublic class FileParser { @Resource private MessageChannel fulfillment;

@ServiceActivator(inputChannel = "incomingfile") public void handleFile(String content){ Order order = ...; GenericMessage<Order> orderMessage = new GenericMessage<Order>(order); fulfillment.send(orderMessage); }}


Pipes and Filter: Backend

express fulfillment

fulfillment FulFillment Router

JMS

normal fulfillment

Normal FulFillment

This channel has a queue Further processing is async


FulFillmentRouter: express or normal?

@MessageEndpointpublic class FulFillmentRouter {

@Router(inputChannel="fulfillment") public String routeOrder(Order order) { if (order.isExpress()) { return "expressfulfillment"; } else { return "normalfulfillment"; } }}

Class automatically instantiated as a Spring Bean


JMS Adapter for express orders <beans ...> <bean id="connectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory"> ... </bean>

<jms:outbound-gateway id="jmsout" request-channel="expressfulfillment" request-destination="fulfillment-queue" /></beans>

JMS adapter handles JMS replies transparently i.e. they are sent to correct Spring Integration response channel Adapters allow the integration of external systems with FTP, RMI, HttpInvoker, File, Web Services etc.


Database for normal orders

@MessageEndpointpublic class NormalFulFillment {

private OrderDao orderDao; @Autowired public void setOrderDao(OrderDao orderDao) { this.orderDao = orderDao; }

@ServiceActivator(inputChannel = "normalfulfillment") public Order execute(Order order) { ... }

}RendezvousChannel is used to feed back the success Passed in as reply to header in the message


Other remarks

•  Sending a message can be asynchronous using a queue •  I.e. a different thread will take it and actually handle it •  Spring Integration can also be used with:

–  Plain XML Spring configuration –  By setting up the environment with Java

•  Spring Integration is currently in 1.0.2


Architecture


The Order Processing Batch


Batches

•  Typically consists of steps •  Steps read and write data •  In this case only one step: Read the orders,

process them, write them back

•  Typical issues: – Restarts – Optimizations (i.e. commits) – Large volumes of data cannot be loaded at once


How to read the data...

•  Load data using a cursor

•  Alternative: Only read the primary keys •  Alternative: Load chunks of data •  No option: Load all data (just too much

data)


Spring Batch Configuration: Processing and Invoices

<job id="fulfillmentjob"> <step id="process" next="invoice"> <tasklet> <chunk reader="processOrderReader" writer="processOrderWriter" commit-interval="10" /> </tasklet> </step> <step id="invoice"> ... </step></job>

<job-repository id="jobRepository" data-source="dataSource"transaction-manager="transactionManager" />

Store for restarts etc.

•  Commit every 10 items •  Commit optimization is transparent •  Thanks to Spring transaction support


Order Reader

<bean id="processOrderReader" class="....JdbcCursorItemReader"> <property name="dataSource" ref="dataSource"/> <property name="fetchSize" value="10" /> <property name="sql" value="SELECT * FROM T_ORDER WHERE C_PROCESSED=0" /> <property name="rowMapper"> <bean class="....OrderParameterizedRowMapper"/></property></bean>

Read the data using a database cursor


Order Writer

public class OrderBatchProcess extends AbstractItemStreamItemWriter<Order> {

private OrderDao orderDao;

@Autowired public void setOrderDao(OrderDao orderDao) { this.orderDao = orderDao; }

public void write(List<? extends Order> items) throws Exception { for (Order item : items) { // do the processing item.setProcessed(true); orderDao.update(item); } }

}

•  Just a POJO service •  Works on a chunk


Other features of Spring Batch •  More complex batches with dependent steps, validation, etc.

•  Other datasources (i.e. XML files, other files) •  ...and other targets for the data •  Can be controlled using JMX

•  Status of job / step instance is automatically persisted •  Therefore: Easy restart

•  Present jobs is restartable anyway •  Scalability by Remote Chunking and Partitioning •  Spring Batch is in 2.0.1


Sum up Σ


Sum up

•  Spring is far more than the Spring Framework

•  Spring Web Services: –  Contract first using XML Schema –  Robust implementations using XPath –  Easy annotation based programming model

•  Spring Integration: –  Infrastructure for asynchronous Pipes and Filters including

Routing etc. –  Integration with JMS, Files, XML, databases... available –  Annotations, plain Spring XML or plain Java code

Σ


Sum up

•  Spring Batch – Easy infrastructure for Batches – Restart etc. included – Monitoring possible

Σ


Run

Manage

Build

Technology

Spring Web Service, Spring Integration and Spring Batch