SOA & EDAThis is a response and also my recap of the discussions that followed Jean-Jacques

Dubray’s response http://www.slideshare.net/jdubray/soa-vs-eda

to my original SOA & EDA presentationhttp://www.slideshare.net/jeppec/soa-and-event-driven-architecture-soa-20

all of which originated from a “discussion” on InfoQ.comhttp://www.infoq.com/news/2014/08/microservices_ballmud

Some of the discussion is still evolving in this document trying to make examples of the different approaches to SOAhttps

://docs.google.com/a/tigerteam.dk/document/d/1Vt96dy3HUzSPD42k6lJFGNNequPIQPuRkFNU99CmBlgand a lot of interaction occurred on Twitter here the last entry as of this presentation

https://twitter.com/clemensv/status/507188489163923456

Hopefully this response can correct what I believe are misinterpretations of my points of view and perhaps any

clear up some presumptions about my lack of knowledge ;)

I respect Jean Jacques deep knowledge of and contribution to SOA. Some of his ideas I fully agree with. The form of communication can be bit harsh at times. I hope this is because he feels deeply about the subject

and not a sign of disrespect*?

* Maybe I’m wrong? http://savas.me/2008/04/soa-cohesion-granularity-coupling-rudeness-eh/ http://www.ebpml.org/blog/76.htm

I’ve added my comments to Jean-Jacques comments to my original slide set to try

and keep it simple.

Where possible I’ve linked to online discussions, references, presentation, blog posts, videos of different people that give a

more deeper or better explanation/information/argumentation

than I can put into this slideset.

This slide set is not intended to do bashing or to muddy up the waters further.

Hopefully it can clear up some confusion and hopefully people will benefit from it and use it to widen the discussion of and

knowledge about different SOA approaches

There are no consensus or conclusions as of now and I’m not sure there will be

anytime soon…

A little while into the discussion it became clear to me that Jean-Jacque, on the basis of my SOA & EDA slide set, had deduced that

I implied a versus between SOA and EDA.

To be clear: I believe SOA and EDA are twins.

I believe there is a need for both Orchestration as well as for an Event Driven approach to Service integration/coordination.

For me it is not an either/or For me it’s that both are needed and they have different

strengths and weaknesses.

The requirements IMO determine which is most applicable.

To make the different SOA approaches more clear we created a document

In https://docs.google.com/document/d/1Vt96dy3HUzSPD42k6lJFGNNequPIQPuRkFNU99CmBlg Jean-Jacques wrote:The orchestration is not just a “nice to have” it is key to delivering a superior user experience.

To which my response was:With this design and the requirements, we need immediate confirmation and therefore events are useless ….Just to be clear: I think Orchestration is useful and needed. A SOA without Orchestration doesn’t make sense in the same sense the SOA without events makes little sense. Events are useful across temporal boundaries where there’s no business expectation for immediate consistency or confirmation.

Which spawned the following Twitter discussion

So we agree that orchestration/coordination is important in SOA.

The point of my SOA & EDA slide set was to introduce people to how business events can be used to trigger business processes

There are problems with SOA

Systems are more fragile

Development and Maintenance costs are higher

Your services are not being reused

You thought SOA would solve your integration problems, but they’ve gotten worse

No one wants to build or maintain services

System performance is worse

Source: Jeppe Cramon

All these symptoms can be experienced if you do not understand what you are doing

Systems are more fragile

This is because there are more moving parts and services are shared assets and when changes are needed it impacts/breaks more components

Development and Maintenance costs are higherTo Quote Peter H. : “the second you introduce distributed, you need to leverage infrastructure that addresses network latency, fault tolerance, message serialization, unreliable networks, asynchronicity, versioning, varying loads within the application tiers etc […] Takes top level talent at the moment, not something all enterprises have access to.”

Source: Jean-Jacques Dubray

Systems are more fragile• I think we agree on more moving parts and a distributed programming model are

large contributors• What I’ve experienced is that the ad-hoc way many services/systems are coupled

to each other result in more fragility than necessary– E.g. when I call a service that calls 3 services, which then again several other services.

Suddenly the stability of the service I call is dependent upon the stability of the combined list of services.

– Sometimes upfront (speculative) thinking in reuse can cause a landscape with too many small services (could perhaps be called the microservice/nanoservice trap) that provide very narrow functionality. To perform their job they need many other services to provide functionality.

• Many call this ad-hoc integration for SOA – but it’s probably closer to EAI• Other times fragility is inflicted because temporal boundaries, across which there

is no business expectation of immediate consistency or confirmation, are ignored– So in stead of decoupling what could have been decoupled we end still try to couple

them because we only see one solution/style of integration.

Development costs are higher• We agree that integration contributes a lot of this• The main reason I hear from developers is that exposing their services over

the company’s central ESB is very time consuming– I think we both agree that the ESB doesn’t need to be centralized, but most

companies seem to end up in a broker/hub-spoke ESB model (probably because of licensing costs or organizational structure)

• This of course varies with ESB product, organization, team skills, etc.• In the last project we used Oracle ESB (OSB) to expose an internal Services to

the outside. This required security, proxying and message transformation, etc.– A model driven Java version of this code was built in 4 hours (including

transformation, mediation and automated tests)– The OSB version took several days

• IMO that’s not SOA’s fault, it’s not even the ESB patterns fault – it’s due to how some organization approach SOA and the tools they choose to use – which in turn give SOA a bad reputation

All these symptoms can be experienced if you do not understand what you are doing

Your services are not being reusedIn SOA reuse happens the other way around, when a new consumer wants to reuse a service, there is a very low probability that the service will be reused as is.

It generally requires some small changes, when you change the service, what happens is that it is the old consumer which reuses the new version of the services, unless you adopt a “compatible versioning” strategy where new versions of services are compatible with older consumers.

Source Jean-Jacques Dubray

Services are not being reused• I think it is very true that reuse happens the other way around

– I’ve also seen Consumer driven interfaces/contracts yield good results. – Bill Poole has an interesting point on why Consumer Driven Contracts isn’t as necessary

with his style of Autonomous services and Event driven Pub/sub http://bill-poole.blogspot.dk/2008/08/consumer-driven-contracts.html

• IMO there are also other reasons for lack of reuse. Among the most common I’ve seen:– Often we are in reality just talking about a bunch of webservices with very little

governance or even a service registry to help people figure out which services are already available

– Not invented here – I will build it my self because … (bad excuse or good excuse)– It takes too long to get anything across the ESB service team, let’s just bypass them for

now, build what we need and deal with it later– Some don’t see reuse as the end goal. Udi Dahan made a comment sometime ago

stating “For me, I’ve given up on the whole reuse thing. I think it’s a crock. Has been since OO.” – Ref:http://www.udidahan.com/2008/11/01/soa-eda-and-cep-a-winning-combo/

All these symptoms can be experienced if you do not understand what you are doing

“You thought SOA would solve your integration problems, but they’ve gotten worse” Well, following what you say, would certainly create a massive

mess with events flying every where and everyone write code to correlate their content world class encapsulation, cohesion and lose coupling

“No one wants to build or maintain services” That’s why you need a dedicated service team It also fit well the architectural paradigm of intent / consistency with

SoR exposing Integration Points

“System performance is worse” Not quite sure about that. I did some measurements on Synapse for

instance and the added latency is less than 50 ms (measured around 10 ms, but just to be safe, I say 50)

Source: Jean-Jacques Dubray

Create a massive mess with events flying every where

• To detail my perspective further: I favor a composite service approach combined with business events. This doesn’t mean that events are used for everything or that orchestration isn’t used.

• There’s video taped version of an earlier (but similar) talk http://www.tigerteam.dk/talks/IDDD-What-SOA-do-you-have-talk/ (from 16:15) where you can hear me say “I’m not saying it’s the perfect way to do it” and later say “it’s not really the scalable model”

• The entire Matador example was intended to plant the idea that Business Events can play a role in temporally decoupling services, trigger business processes and that they CAN (in a few situations) be used to duplicate data between Services/SoR’s (depending on if you view services can be autonomous and therefore can own their business data)

• I’ve also covered the challenge with using Events in a blog post where I wrote “Using Data duplication over Events is a well known, technology neutral, pattern for slowly separating monoliths into autonomous services, but it’s not the final solution for event based integration.”

“System performance is worse”

• In general the ESB only adds very little latency – no argument there• Most of the performance problems I’ve seen is in solutions where you have services calling

services calling services/systems (layered SOA of sorts) and in the mean time you have a client/user waiting (several seconds) for a response. And if one of these services were down, then “nothing” worked.– Nothing says layered SOA cannot perform, but the results I’ve seen point towards many latency and

stability problems and a general “it’s not my problem that this services isn’t working, my service responded in time, etc.”

• In the name of reuse I’ve seen one service call, from a portal widget, result in 12 additional services/system being called, where they with a different service boundary design could have sufficed with 3-4 services.– Udi has blog post about how he believes Composite UI’s can help you design different service boundaries

http://www.udidahan.com/2012/06/23/ui-composition-techniques-for-correct-service-boundaries/ and http://www.udidahan.com/2012/07/09/ui-composition-vs-server-side-orchestration/

– On page 12 of your interaction document you argue why you believe this is not a good approach https://docs.google.com/document/d/1Vt96dy3HUzSPD42k6lJFGNNequPIQPuRkFNU99CmBlg

• In you have multiple systems/SoR’s with overlapping responsibilities, such as 3 different Life Insurance policy systems, and you want to give the customer an overview of all Life insurance policies in one call, this should definitely happen as Orchestration that insulates any client from knowing anything about how many underlying systems there are and what their contracts might look like.

If you have arrived at the realization that SOA is a pain and

costs too much without enabling business agility or

reducing your costs, you are not the only one.

You will have to change your approach to achieve the benefits of SOA

If only no one would listen to hipsters, pundits and other self proclaimed experts

Jean-Jacques Dubray’s comment in dark blue

Change your approach to SOA• If people are happy with the SOA they have, then I don’t think

there should be any reason to change• If their SOA is a big mess (and thereby most likely broken) – and

they’re unhappy - then searching for other approaches/guidance is IMO a good idea– Then they can hopefully decide who they trust

• I hope we can agree that there are many styles of SOA (it seems each person has his)– We’ve discussed it online and it seems that reaching a consensus is hard

• That fact that you find me a hipster or self proclaimed expert is probably valid to you and I’m not sure I can do more than this to change your mind (if at all possible)

Have our mindset changed over the last 40

years?

Clearly yours has not

Jean-Jacques Dubray’s comment in dark blue

My mind hasn’t changed?• I think having had the soundtrack could have helped with some of the misinterpretations• My mind has changed – Back in the days I viewed Layered SOA as the only option for SOA. Listening to people like Udi

Dahan, Clemens Vaster, Bill Poole and Greg Young brought a different perspective on how Services could be build, how boundaries could be identified – how autonomy of service, command, documents and events play an important role together with existing SOA techniques such as Orchestration and mediation– Udi has written about Layered SOA challenges: http://msdn.microsoft.com/en-us/library/bb245672.aspx– And Bill Poole has also covered it: http://bill-poole.blogspot.dk/2008/05/layered-service-models-are-bad.html

• I think being pragmatic is important – finding the right cost benefit solution depending on existing systems/services/skills/maturity.

• This presentation might not sound pragmatic and it was not intended to be – it was intended to be “provocative” and make people stop and think to see another way to view SOA than the traditional Request/Response style integration and coupling services to services to services…

• The critique of SOA problems in this talk is based on what most companies in reality have: – A bunch of webservices that communicate with other services using Request/Response over HTTP (i.e. RPC) and an

ESB put in the middle as a broker. Business/IT alignment is for most of these companies completely alien and most of them end up creating a bigger and bigger mess every day, experience worse performance and try to mend it using caches (without having proper invalidation mechanism put in place) and other remedies (more hardware)

– Actually Udi puts it quite clear (http://www.udidahan.com/2010/11/15/the-known-unknowns-of-soa/)• Most projects which bandy about the SOA acronym aren’t actually made up of services – they’re made up of

XML over HTTP functions calling other XML over HTTP functions, eventually calling XML over HTTP databases. You can layer as much XML and HTTP as you want on top of it, but at the end of the day, most projects are just functions calling functions calling databases – in other words, procedural programming in the large, and no amount of SOAP will wash away the stink.

• The slides on “have our mind changed” is meant to highlight that back in the days we IMO had something similar to autonomous services (data and logic present together where it was needed) and document based integration (a somewhat forgotten integration form) that Greg has been talking about lately https://skillsmatter.com/skillscasts/4081-greg-young-4081

To be clear – none of the examples represent in my opinion SOA

The 4 tenets of SOA1. Services are autonomous– Encapsulation & Cohesion at a bigger scale. – A service is autonomous if it doesn’t rely on other services to

complete its job2. Services have explicit boundaries– Services should not be too familiar with each other. There’s a clear

boundary to where the responsibilities of each services starts and ends

– Boundaries are made unclear when a service rely on other services to complete its job

3. Services share contract and schema, not class or type– Encapsulation improved. We need not worry about how things are

implemented (languages or platform dependent information)4. Service interaction is controlled by a policy– Controls under which rules/form of technical communication

between services take place (e.g. using encryption incl. algorithm and keys)

Services ensure consistency

Service Interfaces are Intentional

Ok

Whatever

Jean-Jacques Dubray’s comment in dark blue

Service are autonomous• I agree that Services ensure Consistency• Service Autonomy is where I think we probably differ most – I think it’s a good idea to

create autonomous services where the service owns its own data• On InfoQ you wrote “There is no "Autonomy" in SOA, services are not Autonomous.

It's a fact, you can't fight it, you must embrace it.• Here’s how I’ve understood your approach: SoR’s are autonomous and Services are not

autonomous. Service offer services to consumers through intentional interface (interface/impl. separation). Services plays the role of coordinators, which ensure that SoR’s eventually become consistent– Perhaps you’re spot on. I have used a similar approach in different situations (e.g. where I have

multiple SoR’s – like multiple CRM’s, Life Insurance systems or SoR’s that should be replaceable).• What lead me to autonomous Services were the compelling cases and descriptions

people such as Udi Dahan, Clemens Vaster, Bill Poole, etc. presented and the cases they’ve mentioned. It doesn’t just work on paper, but also in real life.

• I especially like Udi’s approach with splitting Services into Business Components and splitting these into Autonomous Components (some call them Business Service and Services)http://www.udidahan.com/2008/04/23/visual-cobol-enterprise-processes-and-soa/

Services are autonomousBill Poole gives a very good description of why he think services should be autonomous (http://bill-poole.blogspot.dk/2008/03/services-are-autonomous-but-why.html):

“The services are autonomous tenet was intended by Don Box to mean that services are deployed and versioned independently from each other. I and some others such as Udi Dahan extend this constraint to also stipulate that services should be able to function autonomously at run time. That is, they should be able to as much as possible function without influence from other services. That is not to say that services live in a vaccuum. Services must of course be able to interact with other services.

The nature of these interactions however should be such that no service should rely on the immediate availability of another service in order to handle a message. That is, we want to eliminate temporal coupling between services by limiting ourselves to asynchronous communication between services.

The more autonomous our services are, the more loosely coupled they are. And loose coupling is what gives us business agility, which is the key business driver for SOA.

Without this kind of autonomy between services, one service failing could cause a domino effect causing many other services across the enterprise to fail. In my opinion this degree of coupling is unacceptable.”

Services are autonomous• Clemens Vasters (Microsoft) has also been talking about Services being

Autonomous (picture borrowed from one of his presentations):

Services are autonomousFrom our joined conversation on Twitter, Clemens explained that Autonomous Services is what allows the Azure teams to scale development or operations

* Link provided by Clemens: http://channel9.msdn.com/Blogs/Subscribe/Wheres-the-ESB

And the discussion continued later

https://twitter.com/clemensv/status/506868256234536960

Are Services autonomous?

As a response to our discussion you (Jean-Jacques) posted a blog post which explains and exemplifies your view on services vs. SoR’shttp://www.chorusjs.com/2014/09/03/everything-i-needed-to-know-about-soa-i-learned-it-at-the-gas-station/

Arnon on Service AutonomyArnon believes that services can be autonomous. If it’s the exact same kind of autonomy as Clemens, Udi and Bill I’m not sure

Are services autonomous or not?

I think each person that presented their view on whether Services are autonomous or not, made compelling cases.

The discussion didn’t get as wide as I had hoped, so there’s feedback from Arnon and Udi Dahan that didn’t get on the table.

A Service is

• A technical authority for a specific business capability – very similar to Bounded Contexts

• The owner of all the data and business rules that support this business capability – like Bounded Contexts

A Service is equivalent to a Bounded Context

When you have 500 apps, can you please show me where the bounded contexts are? There is the “theory” and the practice. Each app is a bounded context?

Jean-Jacques Dubray’s comment in dark blue

A Service is like a bounded context

Adrian Cockcroft (previously of Netflix) also describes Services (or Microservices) as being equivalent to Bounded Contexts*

NetFlix has MANY autonomous services/bounded contexts which seems to work for them. The same seems to be the case for Azure (and as far I can tell also for Amazon).

*See http://www.infoq.com/interviews/adrian-cockcroft-microservices-devops see section 3 and 4.

A Service isDDD’s Bounded Context “concept” is very flexible, so I’d like to present another definition of Service which Udi Dahan uses (see http://www.udidahan.com/2010/11/15/the-known-unknowns-of-soa/) :

“A service is the technical authority for a specific business capability.Any piece of data or rule must be owned by only one service.”

What this means is that even when services are publishing and subscribing to each other’s events, we always know what the authoritative source of truth is for every piece of data and rule.

Also, when looking at services from the lense of business capabilities, what we see is that many user interfaces present information belonging to different capabilities – a product’s price alongside whether or not it’s in stock. In order for us to comply with the above definition of services, this leads us to an understanding that such user interfaces are actually a mashup – with each service having the fragment of the UI dealing with its particular data.

Ultimately, process boundaries like web apps, back-end, batch-processing are very poor indicators of service boundaries. We’d expect to see multiple business capabilities manifested in each of those processes.

A Service is

Bill Poole has a very nice description of why he thing it is important to align services with business capabilities:

“The advantage of business capabilities is their remarkable level of stability. If we take a typical insurance organisation, it will likely have sales, marketing, policy administration, claims management, risk assessment, billing, payments, customer service, human resource management, rate management, document management, channel management, commissions management, compliance, IT support and human task management capabilities. In fact, any insurance organisation will very likely have many of these capabilities.”

See http://bill-poole.blogspot.dk/2008/07/business-capabilities.html

The more autonomous services are, the more loosely coupled

they are.

Loose coupling is what drives business agility – which is why we wanted SOA in the first place

Yes, we all know that autonomy and lose coupling are synonymous to consistency

The CAR theorem postulates that you can’t have Consistency, Agility and a high degree of Relationality at the same time. You have to pick two.

Jean-Jacques Dubray’s comment in dark blue

Loose coupling

Requires good encapsulation and high cohesion

We of course all know that encapsulation and high cohesion drive consistency

Jean-Jacques Dubray’s comment in dark blue

CAR theorem

Google didn’t find anything on CAR theorem

But from your description, I don’t disagree with it.

WebServices and in general synchronous integration

has nothing to do with real SOA

Some call this pattern for SOA 1.0 to distinguish them selves from the old approach and the mess it

causes

Because “real” SOA is a protocol?

Jean-Jacques Dubray’s comment in dark blue

Because “real” SOA is a protocol? • SOA is protocol and messaging pattern agnostic • The reason why I mentioned WebServices and Synchronous

communication (Request/Response over HTTP) specifically is, as mentioned before, because that’s what 95% of all Services are (my experience – yours may vary).

• What I’ve seen in many organizations is that many developers are NOT familiar with the fallacies of distributed computing, eventual consistency, orchestration, compensations, idempotence, etc.

• Their contracts are mostly ad hoc – there’s very little alignment to anything but team structure

• For many of the orchestrations I’ve seen have been built in these environments, with a lack of understanding for how much can go wrong with remote calls, it’s sometimes amazing that they don’t end up with more inconsistent systems/services (or maybe they do and they just don’t spot it).

Layered Architectures typically leaves all orchestration to a central manager (ESB) where

business processes are coordinated through spaghetti code (BPEL)

I am actually the (pre) author of that figure coming from BPMN. There is no ESB and no BPEL in it, this is actually a collaboration diagram, which I can be credited for pushing into BPMN

BPEL is actually the only technology available today to achieve consistency at a reasonable cost. You simply have no freaking clue as to what you are talking about

Jeppe:Yes it’s BPMN

and not BPEL – my apologies

Can we get on?

Jean-Jacques Dubray’s comment in dark blue

You simply have no freaking clue as to what you are talking about

• Because I show a BPMN diagram and say BPEL? – Honest mistake – I hope we can see past it

• It was not the content of the diagram per se that was the point (BPMN or BPEL) – I needed a complex looking diagram to serve a point

• Just to be clear - I have no problem with BPEL and neither with BPMN• The reason why I say BPEL spaghetti is because that’s what I typically see in

companies where they try to mend badly designed services using BPEL (this is not BPEL’s fault – if anything it’s a statement of what can be accomplished using BPEL).

• A centralized ESB’s is one of the problem I see many organizations battle with. In theory it should work (with clustering and all), but I’ve so many times heard “the ESB is slow again”, “none of our services work”, “who forgot to increase the DB quota for the ESB log tables”, etc.

• The point of my talk was to raise awareness to alternatives to Layered SOA and Request/Response style integration.– Just like my presentation(s) on CQRS try to raise awareness that having a single

(relational) model, which is what most choose per default, to fulfill the needs of an entire application is not the ONLY solution – and sometimes it’s directly the worst solution (and other times CQRS is like shooting sparrows using a canon)

These BPEL processes typically break all Service encapsulation as they are data and feature envious

This hurts our coupling an autonomy even further

BPEL is actually the only technology available today to achieve consistency at a reasonable cost. You simply have no freaking clue as to what you are talking about

These past two slides is the reason why I am so mad at Jeppe. It shows that no only he has no understanding whatsoever as to what he is talking about, and he has no restraint in what he is claiming

Jean-Jacques Dubray’s comment in dark blue

You simply have no freaking clue as to what you are talking about

• As I’ve mentioned I believe Autonomous Services combined with events and a composite UI is a good approach to SOA – I know you disagree and that’s okay. – The reason for my comment “breaks all encapsulation” is because autonomous services, as

explained by e.g. Udi Dahan, Clemens Vasters or Bill Poole, favor strong encapsulation and therefore dislikes Commands issued between services as it creates too much expectancy on behavior and often creates a strong temporal coupling. They prefer using events for interacting between services, when possible.

• I’ve also seen the introduction of business events can help battle some of BPEL mess people create – to quote your book* “Events enable even looser coupling mechanisms because independent business processes may be triggered by an event. Otherwise we would have to specify a single business process encompassing several subprocess definitions, making it hard to change and maintain. However the use of events makes it just a “tiny bit” more difficult to understand the overall behavior of the system”.

• I think your split in SoR’s and Services, focus on Business Entity Lifecycle (which I agree with) is interesting – maybe it’s better – maybe it’s not – that was what I was trying to find out on InfoQ before the discussion side railed

• I’m not saying don’t use orchestration or BPEL– In reaction to commands/events you could run an Orchestration using BPEL (in slide 95 I used a

Process Manager – which achieves roughly the same)

* Link http://www.infoq.com/minibooks/composite-software-construction

You simply have no freaking clue as to what you are talking about

• I’m also not saying “do EDA for everything” – be picky about what events you expose (in line with Pat Hellands “Data on the outside vs. data on the inside”*)– EDA is useless if you need authoritative answers or a quick response.

• To me SOA and EDA are twins/two sides of the same coin or to quote your book “I’d like to make the point again that in reality, there is no difference between Service Oriented Architecture and Event Driven Architecture, they are part of the same programming model.”– I guess we agree on that they’re not two architectures separate from each other

• What I was building up to in the original presentation was to present a different way of viewing/building Services that are Autonomous, which combined with event publishing and UI composition can result in a lower degree of coupling than many of the the layered approach I’ve seen– To quote Udi Dahan: “In short, the use of UI composition allows services to collect their own data,

making it so anything outside that service doesn’t depend on those data structures which makes both development and versioning much easier. Technical failure conditions can be mitigated at infrastructure levels in most cases and other business logic concerns can be addressed asynchronously with respect to the data collection.” - http://www.udidahan.com/2012/07/09/ui-composition-vs-server-side-orchestration/

* http://msdn.microsoft.com/en-us/library/ms954587.aspx

What we have with classic layered SOA is a HARD COUPLED

architecture

You have no understanding as to what coupling means

Jean-Jacques Dubray’s comment in dark blue

You have no understanding as to what coupling means• I didn’t quantify which types of coupling I meant – but you make a very general assumption• Coupling is a lot of thing – one of the important ones is dependencies, measured using

– Afferent coupling (Ca) – who depends on you– Efferent coupling (Ce) – who do you depend on

• For services these types of coupling are also interesting– Temporal coupling– Platform coupling– Spatial coupling

• There’s also differences in behavioral coupling between e.g. Command, Event and Document based communication between services– http://bill-poole.blogspot.dk/2008/04/avoid-command-messages.html and

http://bill-poole.blogspot.com/2008/04/when-event-messages-are-not-ideal.html– Ian S Robinson has covered Behavioral coupling vs. Temporal coupling here

http://iansrobinson.com/2009/04/27/temporal-and-behavioural-coupling/– And you also provided interesting insight here http://www.ebpml.org/blog/190.htm– I like Commands because they capture the intention of state transition (which can result in an Event being published).

Excessive use of services calling each other using Command messages is where I think we can get into trouble. Use it when needed and avoid it if the requirements doesn’t mandate it.

– I’ve seen composite UI’s, where each service provides it’s own UI parts, work quite well. These UI parts are prime candidates for sending Commands to their own Service implementation (keeping encapsulation and not exposing Commands outside the service boundary unless necessary). This is the approach that Udi Dahan is promoting (see e.g. http://vimeo.com/44235655).• With multiple channels to support this becomes an increased burden – so it’s not a panacea

• What I’ve seen with many layered service solutions is that they often end up have a high degree of temporal and behavioral coupling (a service depending on many services that depend on my services and most integrated using Request/Response). – Sometimes this is because that’s how reality is and then one must deal with that the best possible (intentional interfaces or

consumer driven interfaces can be good options to decouple service consumers)– IMO, when designing new services we should strive to minimize the degrees of coupling by having autonomous services, use

events (when/where they make sense) and apply the composite application/ui pattern (when it’s possible)– Other times adding layers (or indirection) solve problems better, but they also bring their own issues – sometimes the

tradeoffs favor adding a layer, other times they don’t IMO– A layered approach can have a reasonable coupling, but those I’ve seen/worked with quickly ended up knowing a little too

much and doing too much. Probably mostly due to the lack of events for temporal decoupling

Service reuse multiplies our direct and especially indirect

dependencies which creates high coupling

My new Service that

wants to reuse other

services

Service that is

going to be reused

Reusable

Service

Reusable

Service

Reusable

Service

Reusable

Service

Reusable

Service

DB servic

e

Another Service that is

going to be reused

Could please create the same picture with code duplication? Specially the stateful code that is needed for achieving consistency

Jean-Jacques Dubray’s comment in dark blue

Same with duplicationMy point was that service reuse, like that depicted on the previous slide (services calling services calling services), should be scrutinized. If that’s reality (due to existing systems, fixed boundaries) then that’s different from a situation where you’re defining new service boundaries. Are the boundaries the best or could we introduce a better business domain/solution domain alignment, e.g. by aligning to business capabilities, with perhaps fewer service dependencies as a result?

Bill Poole has some interesting perspective on reusehttp://bill-poole.blogspot.dk/2008/04/soa-and-reuse.htmlUdi has also covered some points to consider http://www.udidahan.com/2009/06/07/the-fallacy-of-reuse/

Business Events help us achieve autonomy, loose coupling and

encapsulation

Encapsulation - because we don’t need to supply our services internal data (because no on else needs to know them) – internal data will only be accessible through the Services UI (which takes part in a Composite UI)

Events only need to explain WHAT happened and what it relates to and very few business data (which are typically only needed for replication scenarios)

And so, how do you do exception handling and roll back with business events? You emit more message events?Shit happens you know. Where is the context?

Jean-Jacques Dubray’s comment in dark blue

Exception handling / rollback• It really depends on the type of business process and the required business consistency requirements

– I think your question was based on the assumption that I didn’t have any coordination (inside my service) to coordinate things – I do

– On slide 95 I use the Process Manager* pattern from the EIP book, which reacts to CustomerBilled (or CustomerBillingFailed). It will most like also use timeouts, to escalate problem if CustomerBilled/CustomerBillingFailed isn’t received within a certain period of time (e.g. NServiceBus’s Saga.RequestTimeOut)

– The alternative to events is handling it with an orchestration• Udi has nice description on an event driven process where not all items are available in the Inventory here

http://www.udidahan.com/2008/11/01/soa-eda-and-cep-a-winning-combo/• Another blog post where Udi describes his Event driven vs orchestration approach here

http://www.udidahan.com/2012/07/09/ui-composition-vs-server-side-orchestration/– He ends with given this explanation for why he uses it: “I’ve spent the better part of 5 years travelling around

the world talking to hundreds of people in quite a few business domains, and every single time I’ve found it possible to apply these principles.”

• I know you and Udi have also “discussed” some of this online herehttp://www.ebpml.org/blog2/index.php/2010/08/31/soa-bpm-rest-udi-s-epiphany and here http://www.udidahan.com/2010/08/31/race-conditions-dont-exist/

• What I like about this using events for this, is that it mimics how business already runs (reactively). Even if we had a requirement that items had to be in stock before we accepted an order, we could still run into “race conditions” where Sam who was pulling the items from the Stock broke the last Vase that we just promised was in Stock. Should we rollback the order or should we handle it reactively and perhaps ship the items already in stock together, wait for the Vase to come back in stock and then ship everything together.

• For me, Events are very useful for capturing/reacting to this type of exception/compensation handling and perhaps trigger other processes (like ordering items, etc.) nicely in combination with a Saga/ProcessManager/etc. Could the same be accomplished using command based interaction and Orchestration – sure – there are many ways to skin a cat (I’ve heard – haven’t actually tried to skin one ;)

What if an exception happens• Here’s a an old blog by Udi that covers some perspectives on this

http://www.drdobbs.com/web-development/dont-eda-between-existing-systems/228700784:

When we use SOA/EDA (the same thing as far as I'm concerned), the top-level building block used is the Service. A service may make use of a number of existing systems to perform its work. The business-level events that we publish (and subscribe to) are done by the service, not the existing systems.If there's any orchestration/workflow that needs to be done as a result of a service receiving an event, it is done entirely internal to that service. Inter-service orchestrations don't really exist, as in there is no orchestration coordinator that is not in a service. And the orchestration coordinators within a service don't touch other services' back-end systems - if anything, they publish other business level events.

Be aware: when just starting out on an SOA, you'll find that multiple Services make use of the same backend systems. This may be necessary, but not a desirable state to stay in for too long since it embodies the most insidious and invisible kind of inter-service coupling there is.

“So what if no one picks the message up? Is that an error?”The answer is mu.If a service publishes a business-event (message) and no other services currently care, that's fine. It's not an error. Actually, you'd probably have some kind of infrastructure "queue" where messages that haven't been received more than X time get sent to, so that the event isn't "lost". On the other hand, within a service - if an existing system sends out a message that needs to arrive at another system, and that message doesn't arrive or isn't picked up "in time", that is an error.

This is one of the advantages SOA brings to the table in terms of EDA (again, the same as far as I'm concerned). You get simple messaging semantics between services, while within the "sphere of control" of a service you need, and more importantly can do more complex messaging and orchestration.

Bottom line: you need higher abstractions than your existing systems to employ EDA effectively.

Business Events Messages

andBusiness Processes

By publishing Events messages from our Services we can communicate with each other and also drive

Business Processes

Events drive business processes? You must be a candidate for the Turing Award.

Jean-Jacques Dubray’s comment in dark blue

Events drive business processes? You must be a candidate for the Turing Award.

• Why thanks! Though you were probably sarcastic ;)• It could be my Danish background which let me to use the word drive – perhaps the word

trigger is more correct in English?• I cannot take credit for this idea – the inspiration among many came from here

http://www.udidahan.com/2012/07/09/ui-composition-vs-server-side-orchestration/ orhttp://www.udidahan.com/wp-content/uploads/Event-Driven-Architecture-SOA-through-the-looking-glass.pdf or http://www.udidahan.com/2008/11/01/soa-eda-and-cep-a-winning-combo/ or http://arnon.me/2010/09/soa-contracts-events-ownership/ or http://www.udidahan.com/2011/04/08/integration-how-and-where/ or http://bill-poole.blogspot.dk/2008/04/avoid-command-messages.html or http://bill-poole.blogspot.dk/2008/05/layered-service-models-are-bad_20.html

• Greg Young has similar material in his online course http://subscriptions.viddler.com/GregYoung

• And I believe you’re saying something along those lines in your book “Events enable even looser coupling mechanisms because independent business processes may be triggered by an event. Otherwise we would have to specify a single business process encompassing several subprocess definitions, making it hard to change and maintain.”

• Arnon Rotem-Gal-Oz also seems to agree

Events trigger processesIMO Bill Pooles description here makes a good point:

http://bill-poole.blogspot.dk/2008/07/business-capabilities.html “Business capabilities are hierarchical in nature. We can decompose a business capability into smaller capabilities that support the broader capability. We only decompose a capability into smaller capabilities where there is sufficient distinctiveness between those sub-capabilities to warrant separate and distinct process maps to describe those sub-capabilities.

Of course there are business processes that span multiple business capabilities. However, those processes are realised as event driven process chains. Each business process that executes within a business capability has the ability to raise business relevant events. Other business processes that execute within other capabilities subscribe to these events in order to trigger the execution of these processes.

As such, end-to-end business processes are realised implicitly by virtue of the event publications and subscriptions between processes defined within each business capability. Note that this notion of business event publication and subscription is not a technology concern. It is simply a stable means of describing how business processes execute within an enterprise.”

Business Events example

Sales Service

Order Accepted

Invoicing Service

Retail System

Order Accepted

Customer Billed

Mess

ag

e C

han

nel

We use the Order Accepted event message published from the Sales Serviceto drive the Invoicing/Billing of the customer.

The billing part of the process also use Business Events, in this case Customer BilledEvent, to indicate that its part of the process is completed.

Because we use asynchronous messaging we can still accept orders in the sales service even though the invoicing services is down. The Order Accepted event message will

remain in the Message Channel until the Invoicing Service is ready to process it.

The problem with SOA discussions is that when you take simplistic examples everything works. You need to “scale” these examples, real world stuff, not just PowerPoint Architectures

Please provide end-to-end sequence diagrams including the implementation of the services, which you will discover is best implemented by … an orchestration that waits on the proper eventsJean-Jacques Dubray’s comment in dark blue

My example (with coordination)*

* Taken from the working document https://docs.google.com/document/d/1Vt96dy3HUzSPD42k6lJFGNNequPIQPuRkFNU99CmBlg

Domain Events can also be used for data replication

This is typically used for read-mostly master data that is needed in many business-domains. This way we avoid having

to call a synchronous services to query the latest value(s)

The response to any business event must be coordinated, Pub/Sub does not work in the business world

Jean-Jacques Dubray’s comment in dark blue

Business Events must be coordinated

• I agree, they need to be coordinated. • Udi Dahan covers some of this in his “Avoid a

failed SOA”.http://www.infoq.com/presentations/SOA-Business-Autonomous-Components

Understanding SOA is actually pretty simple

Services provides a normalized interface to the systems of record

They ensure consistency of both the Queries and Command Services serve data from the Systems of Truth They make sure all SoRs are properly updated (or rolled-back) Services call Integration Points on the SoR (not services)

Service interfaces are intentional, they express the intent of the consumer, how this intent is realized is under the control of the service

Technically there are two software paradigms that are essential to SOA Mediation (e.g. Apache Synapse) Orchestration (e.g. Apache ODE)

None of which are part of Jeppe’s presentation, Yes, events are also part of the picture but only to reflect changes

in the SoR when these changes are not “intentional” The response to events is coordinated … by an orchestrationSource: Jean-Jacques Dubray

Your definition of SOA• First of all: I have no problem with your definition of SOA – I think it

can work quite well• Second: I think that Udi Dahans/Clemens Vasters/Bill Poole’s (and

others) have a SOA definition, involving autonomous services, that is also compelling and also has a proven track record (which has also been my experience).

• IMO too few people know EDA in relationship to SOA – my presentation had as a goal to present business events + EDA and some of the possibilities (and in the voice track add more distinctions and pitfalls)

• I’m not sure that a consensus can be reached – but I applaud every attempt to widen the understanding of the different approaches.– As Clemens wrote:

• I hope we can continue a good discussion forward