Automated Scaling of Microservice Stacks

for JavaEE Applications

Why I am here?

I want to discuss with you:

• General autoscaling concept for JavaEE applications

• Real issues and Tricky things

My goals are:

• Save your time if you decide you need to scale your app

• Share our own experience

I am sure you’ve heard those words many times …

I just deploy my application to Docker, set a number of replicas and

… have it running … bla bla bla … super easy! Done!

Yes, pretty easy …

If you need a super scalable and clustered Hello World application …

In real life with JavaEE application

we have architecture similar to this

and its autoscaling becomes not a very trivial task…

In real life with JavaEE application we have architecture

similar to this

Theoretical Part

At least

applications scalable

At least 3 key things to make your

applications scalable

Adapted runtime(Container + JVM + JavaEE Server)

micro

FIRST:

Preferably, thin versions of application servers

NO! YES!

There are various versions adapted

for lightweight containers

micro

GlassFish

There are various versions adapted for lightweight

containers

Any specific reason to use VMs for microservices?

NO!

Well…it might be, but …Well…it might be, but …

One container - one microservice instance

NO!YES!

What should we use instead?What should we use instead?

LXD

YES!

APP

MSMS

MS MS

Applicationprepared for microservices

SECOND:

SORRY,

Applicationprepared for SCALABLE microservices

Next few slides can be a bit boring for those

who know…

BUT

Still important!!!!!!

Next few slides can be a bit boring for those who know

horizontal

ve

rtic

al

Scaling vectors

Decomposition into microservice

At least

Decomposition into microservice

At least 2 key approaches

Logical division

1. Analyze metrics (NewRelic /

JavaVisualVM)

2. Find a weak point (!)

3. Move this part to microservice

4. Repeat

Splitting bigger parts into smaller

Orchestration

software

THIRD:

CONTAINERS

PROVISIONING

HEALTHCHECKS

METRICS

GATHERING

APPLICATION

LIFECYCLE

MANAGEMENT

AUTOMATIC

LOAD BALANCING

SERVICE

DISCOVERY &

CONNECTIVITY

SCALING

CUSTOM

SCRIPTING*

What it typically does?

CPU

HDD

(disk I/O)

SERVICE

FAILURE

RAM

NETWORK

SCHEDULER

Typical triggers that are used for scaling

Practical PartTricky things. Typical issues. Examples

horizontal

ve

rtic

al

In which direction to scale?

Tricky thing #1

TOPOLOGY

Java EE Server

Business Tier

Web Tier

EJB App1 EJB App2

JSP App1

Java EE

Micro

Instance

JSP App1

Java EE

Micro

Instance

JSP App2

JSP App1

Java EE

Micro

Instance

JSP App1

Java EE

Micro

Instance

Scaling type

Java EE

Micro Instance

Java EE

Micro Instance

Example #1

JSP App2 JSP App2JSP App1

JSP App1

Java EE

Micro

Instance

JSP App1

Java EE

Micro

Instance

JSP App1

Java EE

Micro Instance

JSP App1

Java EE

Micro

Instance

JSP App1

Java EE

Micro

InstanceJava EE

Micro Instance

JSP App1 JSP App2

Legacy JavaEE application

Example #2

CPU NO

YES

YES

NO

NO

RAM

HDD (disk I/O)

NETWORK

SCHEDULER

Scaling type VERTICAL

Databases Triggers configuration

(dedicatee)

Takes much time to sync data

Temporary locks

Temporary performance degradation

RESUME

Hard to use auto horizontal scaling

Automatic vertical scaling can be used just fine

It’s better to keep spare instances underloaded

(dedicated storage per instance)

Typical issues while scaling of database instances

Events subscription

Usually, just spinning up an extra instance is NOT enough

Tricky thing #2

onAfterScaleOut[nodeGroup:ejb-app1] {

“call”: “registerNodeInMonitoringSystem”,

“call”: “addMemberToHazelcast”,

“call”: “call3rdPartyApiService”

}

onAfterScaleIn[nodeGroup:jsp-app2] {

“call”: “removeNodeFromMonitoringSystem”,

“call”: “removeMemberFromHazelcast”

}

Example #1

Typical ScaleIn/ScaleOut events and cases

THE FULL WORKING AND MORE COMPLEX CODE EXAMPLE CAN BE FOUND HERE:

https://github.com/jelastic-jps/payara/blob/master/payara-micro-cluster/manifest.jps

1. Decrypt encrypted volume, that was attached on-fly

onAfterVolumeAttached[nodeGroup:instance] {{

“call”: “decryptVolume”

}

2. Get auth keys by instance before joining DAS

onBeforeStartService[nodeGroup:instance] {

“cmd”: “/root/scripts/pullAccessKeys.sh ${token}”

}

3. Deprovision node from DAS if cluster was shrinked

onAfterScaleOut[nodeGroup:das] {

“cmd”: “./asadmin_proxy remove ${instance[@last].ip}”

}

Example #2

THE FULL WORKING AND MORE COMPLEX CODE EXAMPLE CAN BE FOUND HERE:

https://github.com/jelastic-jps/glassfish/blob/master/manifest.jps

Proper triggers configuration

Tricky thing #3

This tuning is a long process, so be patient

Typical problem #1

Changing the value in one place can be reflected in other places

Load Testing

Detect deviations in testing loop

How to deal with this?

The triggers can become outdated with new code

Typical problem #2

COMMIT BUILD TESTINGPERFORMANCE

ANALYSISDELIVERY

The fix is easy here

Inject metrics collection as a part of your CI/CD step

Getting rid of spare things

Tricky thing #4

Some of JavaEE Servers are designed for VMs, not for containers

On example of Weblogic

Typical problem #1

Just remove everything spare

What is the right way to go?

Network is full of useless traffic

Total bandwidthUseful traffic

Typical problem #2

Multicast | Heartbeats

Avoid multicast for members detection

Instance 1

Orchestrator

Instance 2

Instance 3

What we can do?

Use events and unicast

Real use case

On-Demand Scalability and Easy Management of Java EE Project: Hybrid Cloud for Miele

• Migrated from VMs to Containers

• Migrated from GlassFish to WildFly

• Improved scaling and HA

• Implemented Hybrid Cloud

Java in Containers

https://jelastic.com/java-cloud-hosting/

Jelastic.cominfo@jelastic.com

@jelastic