Flow Composition for the Web

Francisco Curbera, Matt Duftler, Rania Khalaf, Douglas Lovell

Presentation of the paper: Bite: Workflow composition for the web. F Curbera, M Duftler, R Khalaf, D Lovell, Service oriented computing: fifth international conference, Springer, 2007.

Outline

• Services and resources

• Composition

• Web flows and flow scenarios

• What is next

About SOA and the Web

Services and Resources

• SOA is about composition: process or structural (so far)• SOA assumes:

– A two level development and execution environment (atomic services, composite services)

– Two standard assembly models, BPEL, SCA

• The Web was NOT built on a composition requirement– Assumes HTTP exists, gives clients access to resources– And a few shared data types

• REST resources are not compositional– Resources are information oriented: Data composition?– But many resources have “behavior” - drive processes

• Process composition for the Web?

End-to-end deployment view in SCA

Component/Composite

Component

B

Component

A

Composite

Composite

Entry

Point

implementation

implementation

WireWireWire

Composite

External

Service

End-to-end business logic in WS-BPEL

Partner link

WSDL porttype

Process

The value of the end-to-end view

• Beyond component oriented use/reuse, first class SOA compositions provide an end-to-end view of composite applications

• Supports end-to-end configuration, validation, management, etc.

– Policy driven configuration• Reduce errors in QoS configuration of distributed components,

ensure correctness of complete configuration

– End-to-end static analysis, at many levels• Tools check “compatibility” of a set of composed components:

functional and non-functional

– Runtime monitoring and validation• Uses structural/process topology to interpret monitoring information

Some interesting trends

• Web mashups– Quickly assemble a new end-user applications by

reusing existing ones

• Process centric programming

– Adding the flow perspective to Web apps: Spring Web Flows, continuations

• The Web (REST) interaction model making inroads as an alternative to enterprise models– The SOAP vs POX debate, and other uninteresting

debates

Mashups

• Mash-up is essentially a form of Web application composition– Google maps are consumed and

aggregated with additional information

– Display final result on browser– Aggregate in the client or the

server sides– There are security issues but the

idea is simple

• Data centric• Composition w/o a component

model!

SOA capabilities for the Web?

• How lightweight can we make it? Who is going to use it?

• Common data model?– But we already have one – XML + mime types– XML is already pervasive from the DB to the UI, but it is just one of the relevant data

types

• Component model?– Reuse is nice, when it works. – Do we need machine readable component definitions? Or just good documentation?– Agree on component interaction primitives based on the resource model (ATOM)

• Resource composition– Process oriented – seems unavoidable since processes already run on the Web– Data composition – it is data model dependent, so far– Structural composition (???) – need good use cases probably because so far Web

apps usually don’t expand many Web components

Bite – Process Composition for the Web

Feed Aggregation

receive

fetchYahooFeed fetchBBCFeed

aggregate

sort

reply

Resource aggregationRelies on a well defined data model –ATOM/RSS

Feed Aggregation

receive

fetchYahooFeed fetchBBCFeed

aggregate

sort

reply

HTTP GET

HTTP GET

HTTP GET

Call local code (no overhead):<action name=“aggregate” target=“java:com.ibm.Aggregator”

operation=“aggregate”><input value=“fetchYahooFeed”/><input value=“fetchBBCFeed”/>

</action>

Use an extension activity:<sort name=“sort” sortOrder=“…’”>

<input value=“fetchYahooFeed”/><input value=“fetchBBCFeed”/>

</sort>

REST interaction<GET name=“sort” sortOrder=“…’”/>

Feed Aggregation

receive

fetchYahooFeed fetchBBCFeed

aggregate

sort

reply

<process name="sortSample">

<GET name="getYahooFeed“target="'http://newsrss.bbc.co.uk/etc…'">

</GET>

<GET name="getBBCFeed"target="'http://rss.news.yahoo.com/rss/etc..'">

</GET>

<aggregate name="aggregate"><input value="getBBCFeed"/><input value="getYahooFeed"/>

</aggregate>

<sort name="sort"><input value="aggregate"/>

</sort>

</process>

Operational semantics

• Data links:– Carry single data item by value– Implies control and data

dependency

• BPEL like execution semantics– Graph style– Dead path elimination and

associated restrictions in place

• Control links– When a dependency is not

associated with data passing– E.g. manager approval

requirement– Data variables

• Data variables– Can be combined with data flows

<receivePOST name="orderRcv" url=“initiateCase" />

<sendMail name="sendToManager" >(…)

</sendMail>

<receive-replyGET name=“MgrApproval”><control value=“sendToManager/>

</receive-replyGET/>

<sendMail name="sendToSupplier" address="orderRcv.mfrEmail[0]" …><control value="MgrApproval"/><input value="orderRcv"/>

</sendMail>

A unified flow model

• Bite supports two flow scenarios for the Web:

– Data flows, where aggregation of feeds is the main model.

– Interactive flows, where a flow drives a set of Web centric interactions

– Any combination of the two – we believe it does make sense to combine data and interaction flows.

Another feed aggregation

Fetch customized

catalog

Customized catalog

and prices

Fetch price and

availability updates

GET FEEDGET FEED

A document pipeline model

Adding interaction capabilities

Plus primitives to encode business logic

Bite: language constructs

Control primitives

Non interactive steps

Utility activities<wait> <empty> <terminate>

Data dependency that serves as a control link as well if the value is an activity

<input>

Control link. <control>

Iteration<while> <foreach>

Call local code<action>

Sending HTTP requests<GET> <PUT>

<POST> <DELETE>

Also *POST. Receiving and replying to messages coming over HTTP. Contain a relative URL attribute used to match an incoming message.

<receiveGET>, <replyGET>,

<receiveReplyGET>

NotesInteraction activities

ATOM Publication Protocol

Atom collection Resources

GET collection�List of resources inATOM envelope

GET resource POST to collection URI �(new) resource, returns resource URIDELETE to resource URI eliminates

Bite processes as resources

Deployed Bite process Process instances

[GET collection:Management only]

[GET resourceManagement]

POST to collection ����(new)

process instanceDELETE instance: terminationRegular termination is implicit

Note: many resources may be hierarchically associated to a process instance

Collection is an implicit process instance factory

Resource oriented model lifecycle

Deployed model collection Process models

Deployed Bite process

...

...

GET collection:Runtime management

POST to collection:Deploys new model

Data models in Bite

• Feed composition only requires support for one (two) models– ATOM (RSS)

• The Web is designed to support an extensible set of data models– MIME types– HTML, XML, JSON, forms, etc.

• Web flows demand more flexibility: pluggable and dynamically adaptable datamodel for supporting different:– content types

• XML, Form content, URL-encoded parameters, JSON, JSON-RPC, text, …

– expression languages for data selection and query• Javascript, XPath, …

Extensible activity set

• Extensible tag libraries for high-level, highly-reusable primitives:– Community-based– User-defined: binds XML syntax to code.

• Enable new activity types to be defined directly in the syntax– Similar to ‘ant’, JSPs, etc …

• Steps:– Register handler that read/writes/invokes– Optionaly provide human readable description

for users.

• Does NOT require:– Write XML Schemas, fancy tools, etc ..

• Implementation not yet released

<aggregateFeeds> <input name=“feed1”><input name=“feed2”>…

</aggregate>

<email subject=“”><to>…</to> <cc>…</cc><input…/><control …/>

</email>

The WS-* overhead in Bite

• Not ready to pay the price? Keep it simple

– Script-like approach to data typing (versus strongly typed interfaces)• Usage implies definition• Errors happen, focus is on short development cycle, typing is optional

– Standard HTTP interfaces (application defined interfaces)• External interactions are resource centric + eMail ☺

– Single protocol – HTTP (extensible protocol set)• All you need – ok, maybe email also.

– HTTP defines all your interactions QoS (extensible, declarative policies)• That gets you a long way

– One tool required: a text editor (many complex tools required to manage all required artifacts)• Fancier tools are available for the typing challenged

What is available now

1. Language specification� Extensive documentation – User Guide, Programming

Guide

2. Full runtime implementation� Currently on ProjectZero.org: tightly architected

according to Zero principles� Positions flow model as an extension of the basic Zero

programming model

3. Tools:� Deployment and management interface� Browser based Delivered with Zero Launch

http://www.ProjectZero.org/

What is next

• Bite profiles through extensibility– Define new activity sets to capture typical steps in

focused use cases:

• Feed manipulation

• Complex user interaction – full browser support

• Core flow QoS: persistence, recovery

– What is the right way to expose these capabilities

• Composite applications beyond flows

Summary

Conclusions

• There is significant value in the PM in the large approach

– Which need not be limited to the enterprise application space

• Web application development is slowly enabling a “service” approach to development

– Reusing large granularity services

– How far will it go?

• A challenge and an opportunity for the SOC community

Thanks!

Questions?

Thanks!

Backup slides

Composition takes place all over – in different models

Browser

Business

Logic Container

Database

Web Container

HTTP/REST Connectors

Local or remote Calls (IIOP,

SOAP?)

Java/.Net c

omposition

Portal composition

Relational data

composition

SOA Composition taking over the middle tiers

Browser

Business

Logic Container

Database

Web Container

HTTP/REST Connectors

Local or remote Calls (IIOP,

SOAP?)

Java/.Net c

omposition

Portal composition

Relational data

composition

Composition coming to the browser

Browser(Javascript/XML)

Business

Logic Container

Database

Web Container

HTTP/REST Connectors

Local or remote Calls (IIOP/SOAP?)

Java/PHP/SOA

composition

Portal composition

Relational data

compositionBrowser mash-ups

Process-centric programming is already here

• Method and page oriented programming has dominated the Web– And most enterprise programming models

• Result is that end-to-end character of a process is lost– Factored out into a set of separate PHP pages, servlets, session beans. Or have to

go through an MVC framework– Business and compositional logic get fragmented– It becomes hard to capture the end-to-end logic of an application, hard to track and

manage

• The focus of successful frameworks and languages has been in easing the DB to HTML access– PHP, Ruby on Rails– Integration is improved across tiers but business logic integration is neglected

• Process centricity is well accepted in the enterprise in the form of WfMSs, modeling tools etc.– But is has barely made an impact on Web programming

Successful Web Frameworks focus on data and presentation

Browser

Database

PHP, Rails

HTTP/REST

Language specific Connectors

Composition is

page aggregatio

Relational data

composition

Continuations in Cocoon -JavaScript

function sellItem() {(...)

var url = "page/getRateAmt";cocoon.sendPageAndWait(url);rate =

cocoon.request.getParameter("rate"));qty =

cocoon.request.getParameter("qty"));(...)

url="page/getZone";cocoon.sendPageAndWait(url,...);zone=cocoon.request.getParameter("zo

ne");(...)

<form method="post" action="continue.#{$cocoon/continuation/id}">

<para>Enter Rate: <input type="text“name="rate"/></para>

<para>Enter Quantity: <input type="text" name="qty"/></para>

<input type="submit" name="submit" value="Next"/>

</form>Source: http://www-128.ibm.com/developerworks/library/j-contin.html

Continuations

• BPEL’s pick is the real thing:<pick>

<onMessage partnerLink="buyer“... >

<!-- activity to add line item to order --></onMessage><onMessage partnerLink="buyer“

... ><!-- activity for order

completion --></onMessage><onAlarm>

<for>'P3DT10H'</for><!-- set an alarm after 3d and 10h

to handle timeout for completion --></onAlarm>

• Also available in:– Cocoon for JavasendPageAndWait(url);

– Rubycallcc {|cont| return cont}cont.call

– Jetty 6

– RIFE

Process style applications

• Continuations enable low overhead flow-like programming.– While not assuming thread

programming or consuming resources unnecessarily

– No need to deal with resource contention

• With obvious limitations:– No concurrency– No persistence – Unclear how to guarantee

consistent outcomes – no transactional model

• How much of this is really needed– And how much can we deliver

• Persistent continuations:– Continuations as persistent Web

resources

• Concurrency– Solutions (Java threads for

example) are usually too complex, require managing resource contention

– A native process model – flow or structured- is likely to be much more usable

• Transactional flows– Sure, but who really needs that!

Groovy DSL Examplepackage samples.feedsgroovy;

import com.ibm.splice.scripting.*;

def flow() {

def builder = new FlowBuilder(writer);

builder.process( name:"feedGroovyExample") {

receiveGET( name:"rssRcv", url:"getFeeds")

GET( name:"getYahooFeed", target:"'http://newsrss.bbc.co.uk/rss/newsonline_uk_edition/world/rss.xml'") {

control( source:"rssRcv")

}

GET( name:"getBBCFeed", target:"'http://rss.news.yahoo.com/rss/topstories'") {

control( source:"rssRcv")

}

aggregate3( name:"aggregateFeeds" ) {

input( value:"getBBCFeed")

input( value:"getYahooFeed")

}

sort3( name:"sortFeeds") {

input( value:"aggregateFeeds")

input( value:"rssRcv_Output.sortOrder")

}

replyGET( name:"rssRply", url:"getFeeds") {

input( value:"sortFeeds")

}

}

}

The SCA implicit runtime model

Enterprise Service Bus: Transform, Route, Notify, Augment, Side Effect

Portal Service

WorkflowBusiness Activity

Business-to-Business Interactions

Enterprise Information System

Adapter

Script, POJO, Stateless Session Bean

DistinguishedServices

Information MgmtXML Database