QEWD.js: Have your Node.js Cake and Eat It Too

Copyright © 2014 M/Gateway Developments Ltd

QEWD.js

Have Your Node.js Cake and Eat It Too

Rob TweedM/Gateway Developments Ltd

Twitter: @rtweed

http://www.mgateway.com

mailto:[email protected]



QEWD.js

• Just another Node.js framework?• What's wrong with the ones already

available and used by thousands of developers?– Express.js– Koa– Meteor– Hapi– Sails, etc etc


Two reasons

• Achieving the best interface to Cache & GT.M from JavaScript– JavaScript as a first-class language and direct

alternative to Cache ObjectScript / Mumps– Integrating with legacy Cache ObjectScript /

Mumps code that was inherently synchronous


Two reasons




– I've been working with Node.js & Cache/Mumps since 2011 and have tried everything you can imagine to figure this out!


Two reasons




• Addressing some of the deficiencies in Node.js that are a direct consequence of its architecture


Node.js Architecture

• Everything executes in a single process– You might have 1000's of users doing stuff

concurrently• They're all sharing the same process!


By Design

• Ryan Dahl, the inventor of Node.js, wanted to create a highly-scalable, high-performance architecture for network programs– Web servers– Chat servers

• Showed that the threaded architecture of Apache had performance limits and heavy memory usage


Node.js Architecture

• Everything executes in a single process• Non-blocking I/O

– No user activity must block the process, or everyone grinds to a halt

• Event-driven, asynchronous logic– Fire off the request, but don't wait for the results

• Carry on with the next task

– When results come back, handle them at the next opportunity


Node.js = server-side JavaScript

• Ryan Dahl wasn't actually a big JavaScript fan• However, he realised that it was a convenient

language for such an environment, as it's designed for exactly the same kind of way of working in the browser

• Google's V8 JavaScript engine was Open Sourced, so he used it to provide the language for Node.js– (which is actually mostly written in C++)


Asynchronous Syntax in Other Languages?

• Node.js isn't unique in supporting asynchronous logic and non-blocking I/O

• Available also in most other modern languages– Java– Python, etc

• Allows efficient use of resources when making parallel requests for resources– Files, external web services etc


Asynchronous Syntax in Node.js

• Node.js is unique in that you have no option but to use Asynchronous logic


Node.js Popularity

• Launched in 2009• Now hugely popular and continuing to

grow in popularity– Server-side of web / mobile applications

• One reason is that just one language – JavaScript – is needed for all parts of an application– Single skill-set


Drawbacks of Node.js

• Even its greatest exponents will tell you not to use Node.js for certain things

• In every other language, zealots will encourage you to use it for everything!



• Even its greatest exponents will tell you not to use Node.js for certain things– CPU-intensive logic– Complex database manipulation

• Particularly relational database handling



• Even its greatest exponents will tell you not to use Node.js for certain things– CPU-intensive logic

• Tie up the CPU and everyone else using the process will be blocked

– Complex database manipulation• Particularly relational database handling



• Even its greatest exponents will tell you not to use Node.js for certain things– CPU-intensive logic– Complex database manipulation

• Particularly relational database handling– Due to the limitations of asynchronous logic, eg you can't

chain functions


Usual Solutions

• Use a third-party queue to offload CPU-intensive work to some other environment– RabbitMQ– ZeroMQ

• Use another language such as Rails for database-intensive work


The down-sides

• Heterogeneous environment• Multiple skill-sets• Multiple moving parts

Node.jsRabbitMQ

Rails

Java

Database


The down-sides

• Heterogeneous environment• Multiple skill-sets• Multiple moving parts

Node.jsRabbitMQ

Rails

Java

DatabaseReminds you of EHMP?


Cache/Mumps integration

• Ideally using synchronous logic– Especially if you have to elegantly and easily

handle locking in legacy code

• Ideally abstract Cache/Mumps databases in a way that makes intuitive sense to a JavaScript developer

• Asynchronous logic for complex database manipulation is painful, time-consuming and difficult to understand/maintain


Async is the New Sync?

• The Node.js community are well aware of the need to avoid "callback hell" caused by asychronous logic

• Next version of Node.js will support Async/Await which greatly improves the syntax– Very like synchronous logic


Async/Await Doesn't solve:

• Node.js concurrency– Still need to avoid CPU-intensive code

• High-level database abstractions– Proper chaining of functions requires

synchronous logic


I want my Node.js Cake & Eat it

• I like JavaScript• I want just one language for everything

– 1 skill set– 1 set of moving parts

• No extra technologies, thank you– Keep it simple


The Problem is Concurrency

• All concurrent users in Node.js share the same process


The Problem is Concurrency

• All concurrent users in Node.js share the same process

• As it happens, Amazon Web Services accidentally created a solution– And nobody (including AWS) seems to have

realised the consequences of what they've done


AWS Lambda

• "Function As A Service"• AKA "Serverless"


AWS Lambda

• "Function As A Service"• AKA "Serverless"

• You upload functions• AWS will execute them

– You don't worry about how or on what physical machine(s)

• You pay per invocation of your function(s)


AWS Lambda

• The first technology they supported was Node.js


AWS Concurrency?

• Your function will be invoked in a private computation container of some sort

• Your function has that container all to itself for the duration of its execution

• No competition with other concurrent users


Lambda Node.js Examples

• Examples show use of asynchronous APIs• Node.js users of Lambda use

asynchronous APIs• They're even excited about the possibility

of using Async/Await in Lambda


Lambda Node.js Examples

• Examples show use of asynchronous APIs• Node.js users of Lambda use

asynchronous APIs• They're even excited about the possibility

of using Async/Await in Lambda

• But unless your Lambda function really needs to be asynchronous, why use asynchronous logic?


Asynchronous Syntax in Other Languages?

• Available also in most other modern languages– Java

– Python, etc

• Allows efficient use of resources when making parallel requests for resources– Files, external web services etc

• But no programmer in those languages would use async logic if they didn't have to



• Why would Node.js developers use asynchronous logic in Lambda functions if they don't have to?– Partly dogma



• Why would Node.js developers use asynchronous logic in Lambda functions if they don't have to?– Partly dogma– Partly because almost no synchronous APIs

exist, particularly for:• Database integration

• Web/REST service access


Such APIs are possible


Such APIs are possible


Any other way…

• To have your Node.js cake and eat it?– Not everyone will want to use Lambda

• Would it be possible to create a locally-available environment where my Node.js code runs in an isolated container where concurrency isn't an issue?



QEWD.js

Node.js Worker Process

Master Node.js Process

Queue

Queueprocessor/dispatcher

qewd

ewd-qoper8-gtm

Cache/GT.M

ewd-document-store

GlobalStorage

ewd-session

Express

HTTP(S)Interface

Bro

wse

r /

RE

ST

Clie

nt

Custom WorkerModule

ewd-qoper8

Constructed from a number of EWD 3 modules

LegacyCode

WebSocketInterface


QEWD.js



Queue


qewd

ewd-qoper8-gtm

Cache/GT.M

ewd-document-store

GlobalStorage

ewd-session

Koa.js

HTTP(S)Interface

Bro

wse

r /

RE

ST

Clie

nt

Custom WorkerModule

ewd-qoper8

Can also use Koa.js – next generation Web Server

LegacyCode

WebSocketInterface


QEWD.js



Queue


qewd

ewd-qoper8-redis

Redis

ewd-document-store

GlobalStorage

ewd-session

Koa.js

HTTP(S)Interface

Bro

wse

r /

RE

ST

Clie

nt

Custom WorkerModule

ewd-qoper8

Can even use Redis as if it was a Global Storage DB

WebSocketInterface


QEWD's Architecture

• Master Process– Handles all incoming REST requests

• Via Express

• Queues and dispatches requests to worker processes

• Worker Processes– You define how large a pool of workers you

require– Each worker handles just one request at a

time (unlike Node.js Cluster)



Queue


IncomingRequest

RESTHTTP

WebSocket

QEWD Architecture

Every incoming requestis passed from Expressand placed in a queue

No further processingof requests occurs inthe master process


QEWD Architecture


Queue


Queue dispatcher isinvoked whenever arequest is added tothe queue


QEWD Architecture



Queue


Worker processstarted if none

available


QEWD Architecture



Queue


qewd-rippleHandler module

Redis/Cache/GT.M

QEWD &application-specific

Modules loaded

and connected todatabase:

Redis,Cache, GT.M


QEWD Architecture



Queue


CustomWorkerModule

Node.js Worker ProcessCustomWorkerModule

Request passedto worker

Redis/Cache/GT.M


QEWD Architecture



Queue


CustomWorkerModule

Worker flagged as Unavailable


Begin processing message

Redis/Cache/GT.M


QEWD Architecture


Queue


Unavailable / processing

Another incomingrequest Node.js Worker Process

CustomWorkerModule


Redis/Cache/GT.M


QEWD Architecture



Queue


CustomWorkerModule





If worker pool size not exceeded,another worker is startedand request passed to it

Redis/Cache/GT.M

Redis/Cache/GT.M


QEWD Architecture


Queue





Redis/Cache/GT.M

If entire Worker Pool is busy:




Redis/Cache/GT.M




Redis/Cache/GT.M


QEWD Architecture


Queue


If entire Worker Pool is busy:

New requests remain

in queue




Redis/Cache/GT.M




Redis/Cache/GT.M




Redis/Cache/GT.M


QEWD Architecture


Queue





Redis/Cache/GT.M

As soon as a worker is available again,a queued message can be passed to it




Redis/Cache/GT.M

Available


Redis/Cache/GT.M


QEWD Architecture


Queue


Finished




Redis/Cache/GT.M

A user's handler function signalscompletion using the function:

finished(responseObject);

This returns the responseobject to the master

process


QEWD Architecture


Queue


Finished




And the response ispassed to Express which

returns it to the clientthat sent the original request

Redis/Cache/GT.M


QEWD Architecture


Queue


Available


The finished() functionalso automatically returnsthe worker process back

to the available pool

So it can now handlethe next queued request

Redis/Cache/GT.M


QEWD Architecture


Queue


Available


Worker processes, once started, are persistent

No start-up / tear-down cost

Workers will automatically close themselves down if they are

inactive for more than a pre-setthreshold time period

Redis/Cache/GT.M


QEWD Architecture


Queue


Available


Worker processes only handlea single request at a time

Completely isolated run-timeenvironment for handler functions

No need for concerns aboutNode.js concurrency, so

synchronous APIs can be used

Redis/Cache/GT.M


QEWD Architecture


Queue


Available


Redis/Cache/GT.M

Long-running or CPU-intensivelogic has no direct impact on

other worker processes


QEWD Architecture


Queue


Node.js concurrency is handledby the master process.

100% asynchronous logic

The master process doesalmost nothing

No CPU-intensive or long-running tasks, so very

high-performance

All the work happens in theisolated worker processes

MultipleConcurrentIncomingrequests


Performance?

• ewd-qoper8 module in isolation:– Handles the queue/master/worker architecture

• Raspberry Pi 3 Model B– 4 core CPU– Readily-available commodity item– Low-cost

• Easily-replicable benchmark


Performance

• ewd-qoper8 benchmark script provided:– How many workers? 3– How many messages? 500,000– Create a steady state of messages added to

the queue as they're being processed:• Add a batch of 622 messages at a time

• Wait 100ms between each batch

– Messages are sent to workers which echo them straight back


Performance

• 5,800 messages/second sustained throughput

• Limiting factor is master process hitting 100% CPU

• Workers only 30% CPU, so plenty of capacity to do real work at this rate


QEWD.js

• For the Java, Python or .Net developer moving to Node.js, QEWD makes it a much simpler, much less weird and scary environment– Asynchronous code only needed when

necessary for parallel execution:• eg accessing multiple remote web services

• Quick and simple to install, configure and deploy


Unique features of QEWD.js

• Somewhat comparable to Apache Tomcat:– Application container

• Can host multiple applications from one single instance of QEWD.js– Can be a mixture of websocket, HTTP and

REST applications


Unique features of QEWD.js

• High-level abstraction of Redis/Cache/GT.M:– Flexible Document Database– Persistent JavaScript Objects

• Built-in very high-performance Session database– Using the above databases


QEWD.js

• Front-to-back JSON• Browser communicates via JSON

messages– Ajax or Web-Sockets

• Even the integrated database is abstracted as persistent JSON– Cache, GT.M or Redis– Identical APIs, so apps can be effortlessly

migrated between databases


QEWD.js

• Looks after all the critically important parts of a Web Application's back-end for you, eg:– Sessions– Database cacheing– Security

• Using tried and tested solutions– Based on the previous-generation EWD

technology


QEWD.js

• Resilient / Audit mode– When enabled, permanent record kept in the

database of:• Queued requests

• Processing status

• Response(s)

– If QEWD restarted, database is examined for queued, unprocessed requests

• Automatically re-queued

– Can specify retention period of database records


QEWD.js

• Supports all browser-side JavaScript frameworks

• Some cool tooling and support available for React


Ripple-OSI

• Open Source project in UK & Ireland– Open Source stack for healthcare

• Based on OpenEHR back-end systems

– Browser-based, REST-ful UI• Orientated around the needs of clinicians

• Initiative led by Dr Tony Shannon

• Middle tier was originally Java-based– Now entirely replaced using QEWD

• Choice of GT.M or Redis as embedded database– Used for cacheing OpenEHR data


Federated Access to OpenEHR

Browser QEWD

GT.M orRedis

ewd-qoper8queue

qewd-rippleModule

Express

OpenEHRServer

AQL overHTTP(S)

Worker

Pul

seT

ileU

I

OpenEHRServer


Few Moving Parts

• Unlike EHMP, RippleOSI has almost no other moving parts apart from Node.js and QEWD.js

• Simple installer script• Can even install and run on a Raspberry

Pi!


Ripple Foundation

• http://ripple.foundation/

http://ripple.foundation/


Software

QEWD.js: Have your Node.js Cake and Eat It Too