Devoxx 2011 integration-camel-cxf-servicemix-activemq

Integration with Apache ServiceMix, Camel, CXF and ActiveMQ Charles Moulliard – Gert Vanthienen Apache Committer FuseSource

Speaker : Charles Moulliard

• Engineer in Agronomy & Master in Zoology – 18 years of experience in IT world development (J2EE, .NET), specialized in new technologies web/Internet & integraNon, Project manager in bank, financial, telco, insurance and transport world

• SoluNon Architect at FuseSource • CommiSer on projects : Apache ServiceMix, Apache Karaf (PMC) & Apache Camel

TwiSer : hSp://twiSer.com/cmoulliard

LinkedIn : hSp://www.linkedin.com/in/charlesmoulliard

My blog : hSp://cmoulliard.blogspot.com

Speaker : Gert Vanthienen

• @Apache SoYware FoundaNon – Apache Camel, Karaf, ServiceMix – Committer / PMC member – Apache Member

• @FuseSource – Member of the engineering team – Mainly working on FUSE ESB

Agenda

• IntegraNon – introducNon

• PresentaNon of projects -‐ Camel, CXF, AcNveMQ and ServiceMix

• Architectures presentaNon

• High availability, scalability and clustering

• PresentaNon of Fuse IDE

• Fabric new Strategy for provisioning and cloud

Par t I

Par t II

Agenda

• Integra(on – introduc(on






Par t I

Par t II

IntroducNon (1)

• IntegraNon projects are really hard and painùl

• We have to integrate different technologies/systems/infrastructures

• Lot of standards/products/tools exist (SOA,EAI,…) and formats (EDI, Fixedlength, XML, CSV, FIX,

• Their implementaNons are resource and Nme consuming

• SOA, BPEL and EAI have not been successfull

IntroducNon (2)

Apache foundaNon proposes alternaNve projects to simplify « integraNon project development »

• Apache Camel – integraNon « Swiss knife » toolbox

• Apache CXF – webservice/REST stack

• Apache AcNveMQ – middleware layer

• Apache ServiceMix – Container for IntegraNon project

Apache ODE (BPEL),

Apache Tuscany (SCA)

Apache Synapse (ESB – XML)

Not covered

Agenda


• Presenta(on of projects -‐ Camel, CXF, Ac(veMQ and ServiceMix





Par t I

Par t II

Camel

• Birthdate -‐ March 2007

• OpenSource IntegraNon Framework implemenNng

EIP paSerns (book published in 2005 by Gregor Hohpe)

• Why EIP To provide word definiNon, grammar and language when designing integraNon projects

• Camel uses a Domain Specific Language transposing the EIP paSerns into Java, XML

Camel

• Some of its features/characterisNcs – In memory bus AlternaNve to JBI using NMR

– Route objects can be XML, File, Stream, Bytes

– TransacNonal architecture

– Sync/Async exchanges

– Threads Management

– Error and excepNon handling

Camel (1)

• More than 50 paSerns are implemented

hSp://camel.apache.org/enterprise-‐integraNon-‐paSerns.html

Camel (2)

• More than 100 components

hSp://camel.apache.org/components.html

Camel (3)

• 18 data formats to simplify data exchange between layers, applicaNons connected

• Example : JAXB, Bindy, SOAP

• Uses marshaler (xml object) and unmarshaler (object xml)

hSp://camel.apache.org/data-‐format.html

Camel (4)

The model Content Based

Router

And its DSL language

Camel (5)

Camel (6)

Camel (7)

Camel (8)

URI corresponds to component name +

properNes

Camel (9)

• Transpose the DSL using Java

• First the configure() method must created

Camel (10)

• Next we create a java class extending RouteBuilder class

Camel (11)

• Language is flexible and proposes

different ways to define the Endpoints and predicates

Camel (12)

• Integrate easily with IntelliJ, Eclipse,

NetBeans

Camel (13)

• Instead of Java, you can use

Spring / XML DSL

Camel (14)

• IDE assistance

DEMO 1

Apache Camel

Agenda







Par t I

Par t II

CXF

• Merge of 2 projects (CelNc and Apache Xfire)

• Simplify the creaNon and deployment of webservices

• 2 approaches “java to wsdl” or “wsdl to java”.

• Support the following standards : – JAX-‐WS : Web Services (XML/SOAP)

– JAX-‐RS : RESTfull service (JSON)

– SOAP 1.1, 1.2, WSDL 1.1

– WS-‐Security, WS-‐Addressing, WS-‐RM

CXF (1)

<plugin> <groupId>org.apache.cxf</groupId> <artifactId>cxf-codegen-plugin</artifactId> <version>${cxf-version}</version> <executions> <execution> <id>generate-sources</id> <phase>generate-sources</phase> <configuration> <sourceRoot> ${basedir}/target/generated/src/main/java </sourceRoot> <wsdlOptions> <wsdlOption> <wsdl> ${basedir}/src/main/resources/report.wsdl </wsdl> </wsdlOption> </wsdlOptions> </configuration> <goals> <goal>wsdl2java</goal> </goals> </execution> </executions> </plugin>

To use it, simply add cxf maven plugin to the pom of the project

CXF (2)

• … and configure a spring config file <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:cxf="http://camel.apache.org/schema/cxf" xsi:schemaLocation=" http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://camel.apache.org/schema/cxf http://camel.apache.org/schema/cxf/camel-cxf.xsd">

<import resource="classpath:META-INF/cxf/cxf.xml"/>

….

<cxf:cxfEndpoint id="reportIncident" address="http://localhost:9080/incident" wsdlURL="etc/report_incident.wsdl" serviceClass="org.apache.camel.example.ReportIncidentEndpoint"> </cxf:cxfEndpoint>

… </beans>

CXF (3)

• Plugin will generate Java classes – JAXB2

– Interface + @Webservice

@WebService(targetNamespace = "http://reportincident.devoxx.fusesource.org", name = "ReportIncidentEndpoint")!@XmlSeeAlso({ObjectFactory.class})!@SOAPBinding(parameterStyle = SOAPBinding.ParameterStyle.BARE)!public interface ReportIncidentEndpoint {!

@WebResult(name = "outputReportIncident",! targetNamespace = "http://reportincident.devoxx.fusesource.org",! partName = "out")! @WebMethod(operationName = "ReportIncident",! action = "http://reportincident.devoxx.fusesource.org/ReportIncident")! public OutputReportIncident reportIncident(! @WebParam(partName = "in”,! name = "inputReportIncident”,! targetNamespace = "http://reportincident.devoxx.fusesource.org")! InputReportIncident in! );!}!

DEMO 2

Apache CXF

Agenda







Par t I

Par t II

AcNveMQ (1)

Clients connect to the broker using connectors

with simple URIs

File & JDBC-‐based message stores supported.

… all based on a flexible Spring-‐based

core.

Topics and queues created dynamically.

Network connectors control how the

broker interacts with other brokers for

discovery, replicaNon, failover, clustering and store

& forward.

• A high performance, reliable messaging fabric, supporNng JMS, C, C++,.Net, Stomp clients. Offers persistence for messages and bridge between ServiceMix instances

• Point to Point – Sender/producer sends messages to a queue.

– Receiver/consumer listens for messages from the queue

– Messages are stored unNl read (or expired)

– Messages can be persisted on disk, are read only once.

c1:Consumer p:Producer

<<jvm>>��frodo:ActiveMQ

Foo:File m

1: Producer sends a message to the broker

m 2: Consumer reads

the message

AcNveMQ (2)

• Publish to subscribe – A JMS client sends message to the topic.

– The JMS broker sends message to all subscribers that are currently alive. • AlternaNvely, durable subscrip@ons can be used so that all subscribers will receive message even if not alive at Nme of sending

– Messages are consumed x Nmes (relaNon 1 to many)

s1:Subscriber

p:Producer

<<jvm>>��frodo:ActiveMQ

Foo:Topic m

m s2:Subscriber

m

AcNveMQ (3)

DEMO 3

Apache Ac(veMQ

Agenda







Par t I

Par t II

ESB – ServiceMix (1)

DemysNfy the concept Enterprise Service Bus

• → This is not

• → Or a

But an exchanging plaòrm of messages, objects moving within transport layers

ESB -‐ ServiceMix (2)

• Allow to separate the business layer from those that will transport the informaNon and transform (marshal) it

• Provide connectors(= endpoints) to collect informaNon and send it on the bus as messages

ServiceMix (3)

• ServiceMix – Project of the Apache FoundaNon

• Started in 2005 to implement the Java Business IntegraNon specificaNon

• Decoupling of the components (business, technical) by using XML messages

• Define a model to package/deploy applicaNons (SU/SA)

• Layer transport is called Normalised Messaging Router

ServiceMix (4)

Message is ACK or ERR

Routing is defined into

the SU

Normalized = XML

exchanges

ServiceMix (5)

• JBI approach is interesNng but restricNve

– Everything must be XML format

– EncapsulaNon of the rouNng – Components dvlpt (25 vs more than 100 for camel)

– Few support from IT actors (IBM, Oracle, …) to improve exisNng spec 1.0 2.0

– Parallel with EJB story ….

ServiceMix (6)

• Decision has been taken 3 years ago by members of project to

– Become independent of JBI specificaNon

– Use « modularity » for jars management, versioning based on OSGI, OSGI EE specificaNons

– Adopt Camel as the new rouNng engine

ServiceMix (7)

• ServiceMix 4

• Based on an OSGI kernel – Apache Felix – Eclipse Equinox

• Becomes an IntegraNon Plaòrm for open-‐source

– Camel

– CXF – AcNveMQ,

– Aries (JPA, TransacNon, JNDI, Web)

Karaf – Heart of ServiceMix (8)

• Birthdate - 16th of June 2010 • Runtime using OSGI • Provide a lightweight container where

Java applications Camel routes WebServices

can be deployed


<<jvm>>��:ServiceMix4

Java Boot (Register as service - Wrapper)

Lightweight container (Apache Karaf)

OSGI Kernel (Apache Felix / Eclipse Equinox)

Container Container Container Container Container Container


• The plaòrm is « modular » because we can select the librairies, containers that we would like to use

Integration

WebService

Web

IoC

Middleware Java

Camel, JBI Jetty, Pax Web

CXF

ActiveMQ

Spring, Blueprint

Karaf – heart of ServiceMix (11)

• CharacterisNcs :

AdministraNon console (local, remote, ssh, web, jmx)

Provisioning system (features)

Hot deployment and configuraNon management

Instances management

Security integraNon (JAAS → ldap, jdbc, file)

Logging management (log4j, logger, commons logging, ….)

DEMO 4

Apache Karaf/ServiceMix

ServiceMix

ActiveMQ

Camel

CXF

<<supports jbi endpoints>> <<deploy into>>

<<supports cxf endpoints>>

<<supports activemq endpoints>> <<deploy into>>

<<deploy into>>

<<supports transport>> <<supports transport>>

<<implements flow with>>

The big picture

Agenda



• Architectures presenta(on




Par t I

Par t II

<<jvm>> ServiceMix4

Architecture -‐ Messaging

FTP Web Service

Bean<<MyBean>>

from(«Yp://server/dir») .beanRef(«myBean»);

from(«cxf:bean:myWS») .beanRef(«myBean») ;

• The bus operates the exchanges between the endpoints using Camel, NMR, JBI

BUS

<<jvm>> ServiceMix4

Architecture -‐ Java

• The OSGI plaòrm proposes a Service Registry (comparable to JNDI) where "interfaces" are registered

• Clients access to the service using lookup mechanism

« Interface » DAO

A

OSGI registry B

<bean id="incidentServiceTarget" class="org.apache.camel.service.impl.IncidentServiceImpl"> <property name="incidentDAO"> <osgi:reference interface="org.apache.camel.dao.IncidentDAO"/> </property> </bean>

Service PoJo

C

Hibernate/JPA

<<jvm>> ServiceMix4

Architecture -‐ Messaging + Java

• Decoupling of the "services" from integraNon layer

from(« cxf:bean:myWS ») .beanRef(« myBean », « validate ») .beanRef(«org.devoxx.jpaService») .beanRef(« myBean », « generateResponse »);

C

Web Service

B

Bean A

« Interface » DAO

OSGI registry

Service PoJo Hibernate/JPA

<<jvm>> ServiceMix4

« Interface » DAO

OSGI registry

Service PoJo

Hibernate/JPA

Architecture -‐ Messaging + Java + Web

• Can be web plaòrm, it uses JeSy Web ApplicaNon Server, Pax Web – a tooling for war and OSGI HTTP Service to register Servlet, …

Web ApplicaNon

OSGI RunNme

Bundle

Web Service Bean

A

Architecture – OSGI limitation • CamelContext are isolated due bundles

classloader • They can’t communicate directly using direct://,

vm:// component

56

<<jvm>> :ServiceMix4

BUS

FTP

from(Yp) .to(direct://order)

from(direct://order) .beanRef(“order”)

WebService

Bundle A Bundle B

<<jvm>> :ServiceMix4

Architecture -‐ LimitaNon

• NMR is required to bridge routes

NMR

Camel Context

FTP

from(Yp) .to(NMR://order)

from(NMR://order) .beanRef(“order”)

Bean Order

Bundle A Bundle B

Architecture -‐ Extend

• When we need asymetric deployment because CPU, memory is not sufficient then we use AcNveMQ to interconnect the instances and persist messages

Agenda







Par t I

Par t II

High-‐availability -‐ AcNveMQ

• 2 mechanisms pure Master/Slave & Shared message store

<<jvm>> samwise:ActiveMQ

Master state replicated with the slave

<<jvm>> frodo:ActiveMQ

<<jvm>> samwise:ActiveMQ

The brokers compete to acquire lock on the shared system (file or DB)

<<jvm>> frodo:ActiveMQ

Server 2 Server 1

Master

• A clustered, highly available approach with 2 machines

Slave

<<jvm>> Frodo:ActiveMq




Network Of

Brokers

AcNveMQ -‐ Network of brokers

High Availability -‐ ServiceMix

• Only one mechanism is available – Lock based (File, DB)

• Bundles can be started or not

Master

<<jvm>>��frodo:SMX4

<<jvm>> samwise:SMX4

Slave

Master

<<jvm>>��frodo:SMX4

<<jvm>> samwise:SMX4

Slave

AcNve/Passive

AcNve/AcNve

High-‐Availability – All

• Combine AcNveMQ & ServiceMix + Network of brokers

Slave

Headquarter

queue:order BUS

Camel -‐ FTP

from(“amq:queue:order”) .beanRef(“order”)

BUS

Bean

queue:order

Master

queue:order

BUS

SMX4 -‐ Master

queue:order

from(“Yp”) .to(“amq:queue:order”)

Regional offices

Camel -‐ WS

BUS queue:order

from(“cxf”) .to(“amq:queue:order”)

from(“amq:queue:order”) .beanRef(“order”)

Bean

SMX4 – Slave

NetWork of Broker

Fuse Fabric Strategy





END OF PART I

A break is welcome !

Par t I

Fuse IDE & Fabric New Strategy to build integration projects






• Presenta(on of Fuse IDE


Par t I

Par t II

DEMO

Fuse IDE








Par t I

Par t II


• Why Fuse Fabric ? – “Integration” projects are really hard to install, configure, run and specially on OSGI platforms

– Distribution of workload/process is not an easy task

– Packaging and deployment must be simplified

– We want to make it easy!

What is Fuse Fabric? • http://fabric.fusesource.org/ • Open source software for configuring, provisioning

& running Fuse and Apache software on any machines – physical, virtual, private, public, private+public cloud or not

etc

• Keeps you DRY from those rainy clouds :) • Weave your containers into an easy to manage

fabric

Fuse Fabric: What does it solve? • Fact :

– Pay twice “dependencies” creation (maven and OSGI)

– maven-felix-pluglin simplifies bundle creation (*)

BUT this is still a complex process • Solution :

• Fab (Fuse Archive Bundle) • Use pom.xml dependencies instead of

MANIFEST to deploy bundles

P A C K A G I N G

Fuse Fabric: What does it solve?

• Fact – OSGI : Karaf – ServiceMix allows to create new instances and administrate them

BUT

DEPLOYMENT

• Instances are not “cloned”

• Configurations must be updated (manually)

• No Central Deployment platform

• Fabric will help us to create instances and manage them – Locally – Remotely (ssh) – In the cloud

DEPLOYMENT

Fuse Fabric: What does it solve?

Fuse Fabric: What does it do? • Scalability of HTTP, WebServices, RESTfull

services depend on external infrastructure (web farm, loadbalancer, hardware, …)

• Configuration changes are required (IP address, port number, …) to deploy new servers/services

• Solution : – Virtualization of services, endpoints, brokers – Discovery of services, endpoints and

message brokers

SCALAB I L I TY

What do we need ?

• Registry = repository of information(s) organised in a tree model

• Agent = process waiting instructions about things to do on servers (karaf, servicemix, activemq)

• Profile = list of tasks to do like – Insert component(s) – Setup a repository – Configure endpoint

Registry • Based on Apache ZooKeeper

– Is a Centralized Coordination Service, distributed and highly reliable

– Uses a directory & file based abstraction for looking at entries

Agent

• Feature deployed per Karaf/ServiceMix container • Agent connects to the registry to discover “tasks,

instructions” • Agent registers important informaNon into the registry • its locaNon, how to manage it via JMX etc

• Agent can control child Karaf containers or child processes

Profiles • Allow for easy group configuration & control

– like using Access Control Lists for security or LDAP trees

• Contains artefacts definition • An agent is started with one or more profiles

associated with it • You can then easily control a group of processes • Profiles support multiple inheritance to simplify

configuration work • Profiles are versioned for easy rolling update /

rollback

Profiles • Allow for easy group configuration & control

– like using Access Control Lists for security or LDAP trees

• Contains artefacts definition – Bundle – Features = collection of bundles – Repository = features xml file – Configuration = parameters of the endpoints, …

• An agent is started with one or more profiles associated with it

Profiles • Registry entries for a given agent:

– /fabric/configs/agents/{agent} – /fabric/configs/versions/{version}/agents/{agent} – /fabric/configs/versions/{version}/profiles/{profile}

• Profiles support multiple inheritance to simplify configuration work

• Profiles are versioned for easy rolling update / rollback

Camel Fabric • Expose / Virtualise a endpoint into the fabric space • Allow to invoke a remote endpoint • Loadbalance requests

DEMO

Fabric, Zookeeper. Provisioning and loadbalancing (ElasNc services)

D-OSGi Fabric • Implementation of distributed OSGI defined in

Enterprise OSGI Specification v4.2 • Allow to register “Interfaces” and look-up them

from remote instances • Use a TCP channel to send bytes over the wire

between jvm of fabric karaf instances • Services can be distributed everywhere and

easily faciltating the scalability

D-OSGi Fabric (next)

DEMO

Distributed OSGI services

ActiveMQ Fabric • Message brokers automatically discovery each

other • Clients use connection URI “fabric:default” to

connect to a message broker • If you have multiple logical ActiveMQ clusters just

name them in the URI e.g. – fabric:us-west, fabric:us-east, fabric:europe

• Workload can be shared between brokers (Apache Apollo)

ActiveMQ Fabric (next)

Other Fabrics • CXF Fabric

– adds a feature to the CXF bus for discovery & load balancing

• ServiceMix Fabric – allow local NMR to use remote endpoints when no local

NMR endpoint available

• Process Fabric – start & monitor processes – keep processes running across machines

References

• IntegraNon -‐ Camel : hSp://camel.apache.org

• ESB -‐ ServiceMix : hSp://servicemix.apache.org

• RunNme -‐ Karaf : hSp://karaf.apache.org

• WebService -‐ CXF : hSp://cxf.apache.org

• Broker -‐ AcNveMQ : hSp://acNvemq.apache.org

• OSGI EE -‐ Aries : hSp://aries.apache.org • FuseSource : hSp://fusesource.com

• EIP : hSp://www.enterpriseintegraNonpaSerns.com

References • Fabric web site :

– hSp://fabric.fusesource.org/ – hSps://github.com/fusesource/fabric

• Apache Zookeeper project – hSp://zookeeper.apache.org/

• Demo

– hSps://github.com/fusesource/fabric/tree/master/fabric-‐examples/fabric-‐camel-‐cluster-‐loadbalancing

– hSps://github.com/fusesource/fabric/tree/master/fabric-‐examples/fabric-‐camel-‐dosgi

A Progress SoYware Company

Any Questions?

• twitter: @cmoulliard@gertv #fusenews • http://fusesource.com

Technology

Devoxx 2011 integration-camel-cxf-servicemix-activemq