Hur möter Microsofts - iBiz Solutions | We are …...Azure Cloud based/external System B Cloud based/external System A The application specific Logic Apps are responsible for system

Hur möter Microsofts integrationsplattformar den

hybrida världen?

Let’s start with a few 101 lessons from 15 years of integration work

Overview & control

Logging, monitoring,

general governance of

integrations

Architecture flexibility

Being able to replace system

and make changes

efficiently

Why use an integration platform in the first place?

Reuse & cost control

Maintain low maintenance/ development cost over time

To achieve these goals that we need to remember a few patterns, concepts and some basic architecture

• Loose versus hard coupling - problemSystem A System B

System DSystem C

• The destination system dictates how an integration is build, all the way from the source system

• Leads to details from the source system being built in to whole integration

• Integration can not be reused – forces need for new integrations instead of reuse

• Creates inflexibility in the architecture – replacement of a system becomes complicated

The integration has built in knowledge about the

destination systems format, transport, address and protocol

New system will need a new integration as the old one is tailored for a specific system.

Lots of integration over time.

Any change in a system will cause a change to the whole integration

System E

System F

System G

System H

System I

System J

System K

Add when finally System A need

changes, updates or is being replaced –

things will be interesting …

• Loose versus hard coupling - solutionSystem A

Integration platform

• New or changed destination system can be added as subscribers to information. Integration only handles requirement directly related to destination system.

• New or changed source systems can be replaced.Only the integration that handles the requirement from that system needs replacement

The publisher integration should only have knowledge on how communicate with the

source system, the integration platform and how to transform from the source format to

a general format

System DSystem C

The subscriber integration should only have knowledge on how communicate with the

destination system, the integration platform, what messages to receive and

how to transfer from a general format to a source specific format

System BNew system can be

added or changed by just adding or changing the

subscribing part of the integration

Microsoft

BizTalk

NOBBEkstern

varemaster

LogiqEDI-

meldinger

Perfion (PIM)Lokal varemaster

AstroWMS (Logistikk)

DAX200ERP-MAXBO

Varer, Priser,

Tilgjengelighet,

Kunder, Ordrer,

Distribusjon

M3ERP-Logistikk

Varer, Priser,

Tilgjengelighet

SentrallagerHybrisB2B

B2C

Eksternt:FTP-utveksling

Filoverføringer

SuperOfficeCRM

proffkunder

KundeByggDoc

CoBuilder

ClarityPrognose innkjøp

AzureB2B

B2C

MagentoB2B

B2C

• Publish subscribe architecture – a key pattern System A

System B

System C• Key technical component in

creating flexible architecture and a loose coupling

• Simplifies scaling – publishers or subscribers can be scaled independently

• Often requires the ability to subscribe to meta data on the messages (content based routing)

The message queue need to be super

stabile – it cannot fail! It’s a major single point of failure.

Needs to have the possibility to add meta

data to messages as that is what we will

subscribe to

For optimal flexibility an simplicity we need the possibility to filter

for only specific messages and not

receive all messages to all subscribers

Needs agents that are responsible for adding the

messages to specific publishers queues

BizTalk messaging engine, details

• Asynchronous versus synchronous – the difference

• Asynchronous messages lets the receiving application handle the message when it’s best suitable

• Asynchronous messaging lets the sending application leave a message without a direct response and the start working on the next one.

Synchronous messaging? Asynchronous messaging?

Our system endpoint

Synchronous messaging has it’s place but should be

used with care.

As it required immediate response it becomes fragile

and require lots of error handling, in addition to put

stress on the responding system

System A System B

Asynchronous versus synchronous –make the correct choice!• Synchronous patterns should

only be used when a intermediate response is needed!

The requirement is really just one way (we don’t care about

the response) – but there is this cool http based API that

we can use!

System A System B

Might seem like a good idea at first – but will put a lot of

stress on the destination system. Failures and resends

needs to be implemented directly in the sending system.

System A System BIntegration component

If there isn’t a requirement for an intermediate response it’s almost always a good idea to try and make the integrations

asynchronous – or at least semi-asynchronous

These have been hard learned lessons …

So how does this knowledge transfer over to cloud based integrations? Or does it?

System A

System B

Logic App

Receive action

Transform action

Send action

Look mom! A one-to-one tightly coupled Logic App!

Logic App

Receive action

Transform action

Send action

System A

System B System C System D

Transform action

Send action

Transform action

Send action

Decide

Look mom! A one-to-one tightly coupled Logic App!

• Changes to any transformation/logic for any of the integrations will require a redeploy of all integration that are part of the Logic App

• New or removed systems will require a redeploy of all integration that are part of the Logic App

• Heavy load on one integration will affect all integrations

• Logging and monitoring becomes more complicated

How can we do things different?

• Azure Service Bus – an its two “modes”

Message Sender

Service Bus Namespace

Message Receiver

Logic App A

Logic App B

BizTalk Server

Topic

Subscription A Logic App D

Filter: InvoiceType = A

Subscription B BizTalk Server

Filter: InvoiceType = A or InvoiceType = B

• Complete pub/system in Azure – packed with features.

• Topic/subscriptions offers content based routing capabilities

So – how can we use all this together?

On premise

Azure Cloud based/externalSystem B

Cloud based/externalSystem A

The application specific Logic Apps are

responsible for system specific communication

and can handle a number of protocols and

transports. Also responsible for

transforming between system specific format

an shared formats.Publish/subscribe - Azure Service Bus

Subscribers are only subscribe to specific messages

using a Service Bus Subscription. Publishers only

publish messages to the Service Bus

with the right properties set.

BizTalk acts very similar to the Logic Apps

Publisher and Subscribers and either publishes or subscribes to messages on the Bus

On-premise data gateway

System specific Publisher

System specific Subscriber



System specific

Publisher

System specific

SubscriberDepending on

previous investments,

strategy etc. Logic Apps might also

be used for communicating with on-premise

applications

On premise basedSystem C

On premise basedSystem D

Microsoft

BizTalk

NOBBEkstern

varemaster

LogiqEDI-

meldinger



DAX200ERP-MAXBO

Varer, Priser,

Tilgjengelighet,

Kunder, Ordrer,

Distribusjon

M3ERP-Logistikk

Varer, Priser,

Tilgjengelighet


B2C


Filoverføringer

SuperOfficeCRM

proffkunder

KundeByggDoc

CoBuilder


AzureB2B

B2C

MagentoB2B

B2C

Microsoft

Azure Service Bus

NOBBEkstern

varemaster

LogiqEDI-

meldinger



DAX200ERP-MAXBO

Varer, Priser,

Tilgjengelighet,

Kunder, Ordrer,

Distribusjon

M3ERP-Logistikk

Varer, Priser,

Tilgjengelighet


B2C


Filoverføringer

SuperOfficeCRM

proffkunder

KundeByggDoc

CoBuilder


AzureB2B

B2C

MagentoB2B

B2C

Azure Event Grid will finally enable event bases architectures!

Publish/subscribe –Azure Service Bus

System A



System specific

Publisher

System specific

Subscriber

System B

Event Grid should make it possible for subscribers to

be “woken up” (using a webhook) when there’s a message on the queue for

them and will hopefully get us away from all polling

need today

Will today constantly check for messages by scheduled

polling. Expensive …

But what about synchronous integrations?

Cloud

On premise

On-premise based systems and data stores

Shouldn’t the same rules and goals apply for a http based synchronous integrations?

HTTP

Clients

HTTP HTTP

• These also need to be monitored and logged!

• Overview and control and general governance is just as important for this type of integration

• Reuse can rarely be achieved without the concept of composite services – exposed data service are usually to granular and will force implementation details in to the client. Again, creating tight coupling

Been here before? What’s the difference between a message in a file and in a http payload?

Cloud

On premise


API Management will solve some problems like overview, control & governance

HTTP

Clients

HTTP HTTP

Azure API Management

Authentication, security, policies,logging and monitoring

• Central API Management gives overview, control and general governance

• Also offers the possibility of abstracting away complexity in underlying implementation –improves reuse

Reuse! Not possible when having tight coupling

On premise

Micro/compositeservices architecture


Cloud based systems and data stores

Virtual ServicesAzure API Management

Micro Service/Composite serviceLogic App/Function/Api App

HTTP

Clients

On premise data gateway


HTTP

This over simplified!

- Cache/performance- Synchronous requirement, asynchronous behavior in

back end system- Client system callbacks

and push to client patterns- Full stack logging and

monitoring

All shortcuts will cost money over time!

One-to-on, tightly coupled integrations

will always cost money and time in

the long run

Synchronous integrations should

have the same requirements as an

asynchronous integrations

Resuable, stabile maintainable

synchronous are however hard to

achieve

ConclusionAn efficient

architecture requires much more that good

technical platform

Requires good architecture, planning

and governance

No problem building one-to-one, tightly

coupled integrations on a perfectly

designed technical platform

When possible -Always choose a pizza delivery over a swat team at your door!

Documents

Hur möter Microsofts - iBiz Solutions | We are …...Azure Cloud based/external System B Cloud based/external System A The application specific Logic Apps are responsible for system