Speedy perception trumps speedy reception–smart asynchronous interactions - Lucas Jellema

Speedy perception trumps speedy reception –

Smart Asynchronous Interactions

Lucas Jellema (AMIS)

5th November 2014

Key Take Aways

• Performance (perceived)

• Scalability

• Availability

• Security

• Flexibility (design time)

Conclusions (to be substantiated)

• Asynch improves [perception of] performance– And availability

• Asynch lowers load on resources and enhances security• Asynch decouples logically• Asynch consists of three steps

– Call and hang-up (free up calling thread)– Process in the background– Return the response (using push and/or

poll & correlation) – to the original tier (& session | client | thread)

• Typical Java (EE) mechanisms: JMS, CDI Events, Executor, WebSocket/SSE, (Job) Scheduler

Terms and Technology

• Multi Tier architecture, Event, Transaction, Pub/Sub, queues, job scheduling, non blocking

• thread | session | client

• HTML(5), JavaScript, AJAX, JSON

• WebSocket, Servlet, CDI (Events)

• EJB (timer), MDB, JMS

• JDBC, Database Job

Who said this?

• I sneezed and the sun set…

• God

• Superman

• Larry Ellison

• Henk

Confusion

• Between

– Correlation

– Causality

We [human brains] think

• Synchronously

– Sequentially Ordered

– Looking for the narrative

When it is not synchronous..

• Out of the blue

• Act of God

• Deus ex Machina

When and Why Synchronous?

In real life, when you tell something to a person, you don't mind waiting for answer (beyond some form of acknowledgement that your message was heard and understood) if and when:• You asked a question that you want an answer to• You cannot meaningfully continue with your life until

you receive the response• You do not have to wait too long (a little subjective)• You will lose the opportunity to receive

the answer at all if you do not wait for it• You can afford to wait (because of the time

or cost of resources involved with waiting)

Asynchronous… in real life

• “I will get back to you”

• Daily newspaper delivery

• DHL delivery of our Amazon order of last Friday

• Death Notification of very old uncle

• Send application letter & CV

• TV presenter talking into camera

• That pretty guy or girl in the barwho said (s)he is surely goingto call you…

I wonder: if I were to ask a rethorical question

on asynchronous conversations, would I receive answer?

Why Asynchronous?

• You have better things to do than wait for a reply

• You do not need the reply (any time soon)

• The ‘other side’ cannot spare you the time or resources to continue to wait

• You trust the reply will make itsway to you

In IT: Why Synchronous all the time?

• Easy – it is usually the default

• How to do asynchronous processing – How to hang up the phone?

– How to return faults in asynchronous cases?

• When asynch – how to establish communication (howdoes invokee find right point of contact in caller)– What to do when not found?

– Responsibility has shifted(and probably rightly so)

• Note: Asynchronous “Event Storms” can overwhelm systems

Consumption of scarce resources has to be done very carefully

• Threads are expensive and scarce– It is a waste to have them blocked – waiting and

doing nothing while tying up physical resources

– Many jobs, connections and conversations have tobe handled by limited number of threads

• Other scarce resources– JDBC Connections

– HTTP Browser-Server Connections

– Memory, CPU, Battery

Some things have changedover the years

• Multi core => parallellism• JavaScript – single threaded – you have to use asynch!• Better facilities in programming languages and platforms

– Java Future– HTML5 workers– SSE/WebSockets– Java EE CDI, Asynch EJB, JMS– Database HTTP Call-out & Query Result

Change Notifcation

• Understanding of user experiencerequirements– I am working on it can be just as good

as it is done – compare ordering in a web shop

Wait for… what exactly?

Web Application ESB

Document Management

ProcessEngine

One way (fire and forget) with queue for save keeping

Web Application ESB

Document Management

ProcessEngine

Q

First wait… then potentially fail(availability chain)

Portal ESB

ERP system

Asynchronous (background) processing resolves issue

Portal ESB

ERP system

Background process

Wait with so many consumers(resource depletion)

Web ShopBusiness Tier

(EJB)

Background Database Jobs and/or Front-End/Write Behind Cache

Web ShopBusiness Tier

(EJB)

Cache

jobs

Web ApplicationClient (Browser)

Middle Tier

Database

RestaurantDinner Table

Waiting Staff

Kitchen

The Synchronous Restaurant

• Hold your breath (and wait)once the order has been placed

• The waiter is exclusivelyassigned to the table and the order

• The waiter goes to the kitchen,hands over the meal order and…waits until the meal items are complete

• Note: one cook works only on a single table’s meal order

• The waiter returns to the table withthe meal – and the guests can exhale

Synchronous Restaurant as Web AppClient (Browser)

Middle Tier

Database

HTML

Servlet

EJB

TableStored

Procedure

Form Post

Synch Call

Synch CallTransaction Boundary

Synch Call

screenshot

Turning the restaurant into an asynchronous place

• Guests can continue conversation after placingtheir order

• Waiters can continue waiting on other tablesonce they have handed off the order– And prepared and delivered drinks

• Cooks can work on multiple meal orders• Meal items can be delivered when prepared

– Appetizers ahead of main course

• Fewer resources are needed, while better service is provided

Browser

Java Middle Tier

Foreground(User Interface)

Background(blocking UI)

WebWorker(non blocking)

timer

Worker

SynchronousHTTP GET/POST

AsynchronousXMLHTTPRequest

AsynchronousXMLHTTPRequest

Web Client makes Asynchronous Call –Callback function updates UI

Client (Browser)

Middle Tier

HTML

Servlet

AJAX Call

Lots of SynchronousProcessing

Web Tier makes Asynchronous CallClient (Browser)

Middle Tier

HTML

Servlet

AJAX Call

MDBAsynch

EJB

JMS

How do you end the synchronous part of a conversation in Java

• Hand off to Executor– Run on a different thread

• Publish to JMS– Consume on different thread

• Invoke Asynchronous EJB– Execute on a different thread

• Start a Java Batch Job• Database Commit• Send Web Socket message

– Send email, write file, put on grid

• Invoke asynchronous Web Service• Note: not CDI events

Browser

Java Middle Tier

HTTP GET/POST

executor

QuartzScheduler

WorkManager

JMS MDB

MDB

EJB Timer

Java Batch (Java EE 7)

jobJobOperator

EJB

One Way WebService

grid

CDI Bean(Java EE 7) CDI events

vert.x

Business Tier processing is AsynchronousMiddle Tier

Servlet

MDB EJB

JMS

Database

Table

StoredProcedure

Synch CallSchedule

Job

Java Middle Tier

WorkManager

JMS MDB

grid

MDB

One Way WebService

DatabaseJDBC HTTP

Emb

edd

ed

PL/SQ

L Ga

tewa

y

AQ

Job Scheduler

FTP/WEBDAV

XML DB

XML DB Repository

events Event Handler

Email

Stored Java (poll job)

File

File Watcher

Na

tive Da

tab

ase

Web

Services

tableasynchcommit

pipe

Global appcontext

Synchronous Meal Ordering

Middle TierServlet

EJB

Database

Table

Synch CallSchedule

Job

Client (Browser)

HTML

AJAX Call

Schedule Database Job To Cook

Table

Synch CallSchedule

Job

Cook Job to ExecuteTable

StoredProcedure

Asynchronous Response up the tiers

Returning the Asynchronous Response

• Push when ready– Contact the requesting

tier• Against the grain

– Deliver to the requestingcomponent• Which session, thread, client

• Poll periodically until the response is available– Get hold of the right response – outside

the scope of the original conversation

Business Tier processing is AsynchronousMiddle Tier

Timer EJB

JMS

Database

Table

StoredProcedure

poll for ready meal items

Pushing & Polling – DB => Java EE

Java Middle Tier

JMS

MDB

MDB

DatabaseJDBC HTTP

AQ

StoredProcedure

FTP/WEBDAV

XML DB Repository

Email

Mail sending

File

File Writer

table

pipe

Global appcontext

HTTP Caller

DB

Qu

ery Resu

ltC

ha

ng

e No

tificatio

n

po

lling

threa

d

table

TCP/IP

Socket

servlet

AsynchronousWeb Service Response

HTTP

WebSocket

TCP

Web SocketEndpoint

Correlation: “I am calling back regarding…”

• Correlation

– Response contains an identifier that is known tothe polling thread

– or that can be used to connect to the originalcaller (session, client, thread)

Asynchronously delivered meal item comes with table number

Middle Tier

Timer EJB

JMS

Database

Table

StoredProcedure

poll for ready meal items

Timer EJB and Polling Kitchen

Timer EJB

poll for ready meal items

Timer EJB, Polling Kitchen andPublishing Meal Items

Timer EJB

JMS

poll for ready meal items

Java Middle Tier

LongPoll

Repeated/Long Poll

Web Socket

Web SocketEndpoint

ServerSent Event

Cache Event Handler

HTTP Session

listeners

JMS

MDBEJB

PiggyBack

A waiter delivers the meal itemsClient (Browser)

Middle Tier

HTML

WebSocketServer

WebSockethandler

MDBJMS

CDI Event

Consume JMS and Publish CDI EventsMiddle Tier

WebSocketServer

MDBJMS

CDI Event

Initialization of WebSocketClient (Browser)

Middle Tier

HTML

WebSocketServer

WebSockethandler

Send Meal Item from CDI Event toWeb Socket channel for table

Client (Browser)

Middle Tier

HTML

WebSocketServer

WebSockethandler

CDI Event

Demo of Asynchronous Restaurant

Demo of Asynchronous Restaurant

Demo of Asynchronous Restaurant

Demo of Asynchronous Restaurant

Cut off points, thread jumps, ends-of-conversations, asynchronization points

Middle Tier

Database

Client (Browser)

AJAX

JMS

WebSocket

Job

JMS

WebSocket

CDIEvent

Conclusions

• Asynch improves [perception of] performance– And availability

• Asynch lowers load on resources & enhances security• Asynch decouples logically• Asynch consists of three steps

– Call and hang-up (free up calling thread)– Process in the background– Return the response (using push and/or

poll & correlation) – to the original tier (& session | client | thread)

• Typical Java (EE) mechanisms: JMS, CDI Events, Executor, WebSocket/SSE, (Job) Scheduler

Code on GitHub:https://github.com/lucasjellema/DinnerAtSpeedyJoes

Slide on Slideshare and NLJUG websitehttp://www.slideshare.net/lucasjellema/speedy-perception-trumps-

speedy-receptionsmart-asynchronous-interactions-j-fall2014lucasjellema

Contact me:lucas.jellema @ amis.nl

Blog: http://technology.amis.nl