Advancing Integration Competency and Excellence with the WSO2

Integration Platform

Kasun Indasiri Software Architect, WSO2

Miyuru WanninayakaTechnical Lead, WSO2

Agenda

• Fundamentals of WSO2 ESB• Cloud Connectors• RESTful Integration • Cloud Connector – Scenarios • RESTful Integration – PizzaShop Scenario • Discussion

Software in Enterprise

• Large enterprises have many software systems in their “line of business”• Enterprise Resource Planning systems (ERP)• Management Information Systems (MIS)• Decision Support Systems (DSS)• Data stores• Legacy systems

• These systems are often independent and controls one or more related business activities

• But they are all parts of a single business process

Need for Enterprise Integration

• Individual software applications cannot achieve much

• Systems should be integrated to work together• Enables free data flow across the organization• Improves production and management efficiency• Application users can make better decisions

Easier Said Than Done!

• Integrating a multitude of complex software applications is no child's play

• Systems are diverse• Different platforms• Different programming languages• Different protocols and messaging standards• Different QoS requirements• Not everyone adhere to standards

Point-to-Point Connections?

Point-to-Point Connections?

Point-to-Point Connections?

Point-to-Point Connections?

Bus

Bus

What is an ESB?• A piece of software that

resembles a data bus used in computers

• Provides a uniform approach for connecting systems

• Monitor and control routing of message exchange between services

• Software applications communicate via the bus

• The service bus acts as a carrier or a message broker

• Replaces direct contact between applications – Reduces coupling

http://en.wikipedia.org/wiki/Enterprise_service_bus

WSO2 Enterprise Service Bus

• A lightweight, high performance ESB• Configuration driven over coding• REST, SOAP and WS-* support• Domain specific protocol support (eg: SAP, FIX)• Extensible to support custom protocols /

message formats• Cloud enabled• Supports all EIP patterns

http://www.eaipatterns.com

KEY FEATURES

Routing• Simple proxy • Header-based Routing • Content-based Routing• XPath/Property/Regular

Expression• Rules• If/Then/Else• Router mediator for complex

routing

Traffic Filtering• Xpath and regex based filtering• Script based filtering• Schema based filtering and

validation

<id>foo</id>

<id>bar</id>

<id>bar</id>

<id>foo</id>

Service Orchestration• Service chaining• Split/Aggregate/• Clone/Merge• Parallel/Serial execution• If/Then/Else

Transformation• XSLT• Payload Factory• Smooks• XQuery

Protocol & Message Format Switching

• Message Formats• SOAP, REST, JSON, Binary

• Protocols• HTTP(S), JMS, VFS(File), TCP, SAP, FIX, HL7, SMTP

Business/Cloud Connectors

Connectors

Load Balancing

QoS : Security, Throttling, Caching• WS-Security / REST Security • Throttling

• Concurrency• Rate

• Caching• Local• Replicated• Suitable for idempotent

operations

REST Capabilities

• Expose any service as RESTful APIs

• URL Mapping • URL Templates

REST

http://www.infoq.com/articles/webber-rest-workflow http://wso2.org/library/articles/2012/09/get-cup-coffee-wso2-way/

Store and Forward

• Store incoming request in message store• In memory, JMS queue, Database

• Forward to backend using message processor• Guaranteed delivery

Message Store

WSO2 ESB NUTS AND BOLTS

WSO2 ESB – Messaging Architecture

Runtime

Mediator

Configuration Data

Input Message Output Message

Sequences

• A sequential arrangement of mediators – A mediator chain

• When a message is handed to a sequence it is given to the first mediator of the chain - The outcome of that will be handed to the second mediator of the chain and so on…

• The messages are sent through the chain while mediators perform various actions on it along the way

• Pipelining

More on Sequences…

Consider the example sequence shown above Messages are first logged by the log mediator The property mediator sets a property on the messages Then they are sent to a specified endpoint by the send mediator It is a simple log-and-forward message flow

Endpoints Defines an endpoint reference (EPR) to which messages can

be sent/forwarded from the ESB The send mediator takes an endpoint as an argument

Various operational constraints can be enforced on an endpoint Timeout duration Message format (POX, SOAP 1.1, SOAP 1.2)

QoS expectations can be set on an endpoint WS-Security

Proxy Services / API Acts like a virtual service that can receive requests from

clients Received client requests are processed and routed to a

specified endpoint (usually to an actual service – backend service)

Responses coming back from the backend service are further processed and forwarded to the clients

Resembles traditional HTTP proxy servers

Proxy Service – Abstract View

• With complex business requirements, ESB config can grow bigger..

• Need a way to reuse the configuration

• WSO2 ESB 4.0 introduces – Templates

• An analogy… classes vs instances

Templates

• Sequence Templates

Templates

<template xmlns="http://ws.apache.org/ns/synapse" name="xslt_func"> <parameter name="xslt_key"/> <sequence> <log level="full"> <property name="BEFORE_TRANSFORM" value="true"/> </log> <xslt key="{$func:xslt_key}"/> <log level="full"> <property name="AFTER_TRANSFORM" value="true"/> </log> </sequence></template>

Templates

• Call Template Mediator – Invoking a Template

<inSequence><call-template target="xslt_func">

<with-param name="xslt_key" value="xslt-key-req"/></call-template><send>

<endpoint><address uri="http://localhost:9000/Foo"/>

</endpoint></send>

</inSequence>

WSO2 ESB Cloud Connectors

WSO2 ESB Cloud Connectors

• Cloud to Cloud

WSO2 ESB Cloud Connectors

• Cloud to Cloud

WSO2 ESB Cloud Connectors

• Cloud to Enterprise

Cloud Connector Architecture

• Objectives Connect with any cloud API

Supporting diverse APIs (REST, SOAP, SDKs)

Dynamic configuration (No predefined configuration language)

Development methodology (config, java code, javascript, pojo)

Reduce development time(100+ connectors)

Dynamic Tooling with DevStudio

Performance

Dynamic Synapse Configuration based on Templates

• WSO2 ESB config language is predefined– Mediator Factories and Serializers

• Eg: Property Mediator Serializer/Factory

– So, dynamic synapse config language is not possible?• Custom Mediators/Mediator per each connector?– Won’t scale and it will be tedious task to write a

connector• A new approach… – Inspired from Templates and Mediation Library

Structure of a Cloud Connector

• Connector Structure

• A connector can have

• Synapse config as templates

• Custom Mediators

• External Libararies (eg: twitter4j)

• Java Script (or any other scripting lang.)

– Connector Deployer

• Loading required libraries

Structure of a Cloud Connectorgooglespreadsheet-connector.|-- connector.xml|-- googlespreadsheet| |-- component.xml| |-- config.xml| |-- create_spreadsheet.xml| |-- update_worksheet_metadata.xml| `-- username_login.xml|-- googlespreadsheet.properties|-- icon| |-- icon-large.jpeg| `-- icon-small.jpeg|-- lib| |-- gdata-client-1.0.0.wso2v1.jar| |-- gdata-core-1.0.0.wso2v1.jar| |-- gdata-docs-3.0.0.wso2v1.jar| |-- gdata-media-1.0.0.wso2v1.jar| `-- gdata-spreadsheet-3.0.0.wso2v1.jar|-- META-INF`-- org `-- wso2 `-- carbon `-- connector `-- googlespreadsheet |-- GoogleSpreadsheetAuthentication.class |-- GoogleSpreadsheetBatchUpdater.class |-- GoogleSpreadsheetCellAddress.class |-- GoogleSpreadsheetCellData.class |-- GoogleSpreadsheet.class

Templates

External Libs

Connector custom code

New Secure Vault

• No hardcoded secrets or passwords in the ESB Config

• wso2:vault-lookup

<twitter.config> <consumerSecret>xx</consumerSecret>

<accessTokenSecret>{wso2:vault-lookup('my_secret')}</accessTokenSecret> <accessToken>{wso2:vault-lookup('my_token')}</accessToken> <consumerKey>{wso2:vault-lookup('my_key')}</consumerKey></twitter.config>

Cloud Connectors – Use Case (I)

• Salesforce + Twilio – Lead Generation

Cloud Connectors – Use Case (II)

• Salesforce + Google Spread Sheet – Opportunity Management

Cloud Connectors – Use Case (III)

• Salesforce + JIRA + Twilio – Account Management

Cloud Connectors – Use Case (IV)

• Twitter + Salesforce – Lead Monitor

Advanced RESTful Integration • HTTP Endpoint

<endpoint xmlns="http://ws.apache.org/ns/synapse" name=“PizzaShopJaxRS"> <http uri-template= "http://localhost:8080/{uri.var.servicepath}/restapi/{uri.var.servicename}/menu?category={uri.var.category}&amp;type={query.param.type}" method="GET"> </http></endpoint>

Advanced RESTful Integration • JSON Payload Factory

<payloadFactory media-type="json"> <format>{"purchaseInformation": {"amount":

"$1","cc": "$2"}} </format>

<args> <arg evaluator="json"

expression="$.payment.amount_lkr"></arg>

<arg evaluator="json" expression="$.payment.card_no">

</arg> </args></payloadFactory>

Advanced RESTful Integration • Pizzashop Scenario

Discussion

THANK YOU