Embed Size (px)
Citation preview
Automated Scaling of Microservice Stacks for
JavaEE Applications
by Ihor Kolodyuk
Few words about me …
Driving technology direction at Jelastic
Like to solve unsolvable problems
Checking technology concepts on practice
Why I am here?
I want to talk to you about:
General autoscaling concept for JavaEE applications
Real use cases. Real issues. Tricky things
My goals are:
Save your time if you decide you need to scale your app
Share our own experience
I just deploy my application to Docker, set a number of replicas and … have it running … bla bla bla … super easy! Done!
The myth
Yes, pretty easy …
If you need a super scalable and clustered Hello World application …
In real life with JavaEE application we have architecture similar to this
and its autoscaling becomes not a very trivial task…
The truth
Legacy Application
Scalable Cluster
Autoscalable Cluster
Theoretical part
At least 3 key things to make your applications scalable
First:Adapted runtime
(Container + JVM + JavaEE Server)
micro
Preferably, thin versions of application servers
YES!NO!
microglassfish
There are various versions adapted for lightweight containers
Any specific reason to use VMs for microservices?
NO!
Well…it might be, but …
What should we use instead?
YES!
One container - one microservice instance
YES! NO!
Second:Application
prepared for microservices
APPMS
MS MS
MS
SORRY, Application
prepared for SCALABLE microservices
Next few slides can be a bit boring for those who know…
BUT
Still important!!!!!!
Ideal scaling and decomposition
horizontal
verti
cal
topolo
gy
Decomposition into microservice
At least 2 key approaches
Logical division
Splitting bigger parts into smaller
1. Analyse metrics (NewRelic / JavaVisualVM)
2. Find a weak point (!)
3. Move this part to microservice
4. Repeat
Third:Orchestration software
What it typically does?
CONTAINERSPROVISIONING
HEALTHCHECKS
METRICSGATHERING
APPLICATION LIFECYCLEMANAGEMENT
SCALING
AUTOMATIC LOADBALANCING
SERVICE DISCOVERY&CONNECTIVITY
CUSTOM SCRIPTING*
Typical triggers that are used for scaling
CPU RAM
HDD(disk I/O) NETWORK
SERVICEFAILURE SCHEDULER
Practical part
Tricky things
Typical issues
Examples
Tricky thing #1
In which direction to scale?
horizontal
verti
cal
topolo
gy
Example #1Message Queue Service
Triggers configuration
CPURAMHDD (disk I/O)NETWORKSCHEDULER YES
NOPROBABLY NO
GENERAL SETGENERAL SET
Scaling type HORIZONTAL
Tricky place
Example #2Databases
CPURAMHDD (disk I/O)NETWORKSCHEDULER
YES
VERTICAL
NO
NONO
Scaling type
Triggers configuration
YES
Typical issues while scaling of database instances
Takes much time to sync data
Temporary locks
Temporary performance degradation
Hard to use auto horizontal scaling Automatic vertical scaling can be used just fine It’s better to keep spare instances underloaded
(dedicated storage per instance)
RESUME
Example #3Legacy JavaEE application
TOPOLOGY
Java EE Server
Business Tier
Web Tier
JSP App1 JSP App2 JSP App2
EJB App1 EJB App2
Java EE Micro Instance
JSP App1
Java EE Micro Instance
JSP App1
Java EE Micro Instance
JSP App1Java EE
Micro Instance
JSP App1
Java EE Micro Instance
JSP App1
Java EE Micro Instance
JSP App2
Java EE Micro Instance
JSP App1
Java EE Micro Instance
JSP App1
Java EE Micro Instance
EJB App2Java EE
Micro Instance
JSP App1
Java EE Micro Instance
JSP App1
Java EE Micro Instance
EJB App2
Scaling type
Tricky thing #2
Events subscription
Usually, just spinning up an extra instance is NOT enough
Example #1Typical ScaleIn/ScaleOut events and cases
onAfterScaleOut[nodeGroup:ejb-app1] { “call”: “registerNodeInMonitoringSystem”, “call”: “addMemberToHazelcast”,
“call”: “call3rdPartyApiService”
}
onAfterScaleIn[nodeGroup:jsp-app2] { “call”: “removeNodeFromMonitoringSystem”, “call”: “removeMemberFromHazelcast” }
THE FULL WORKING AND MORE COMPLEX CODE EXAMPLE CAN BE FOUND HERE
Example #2
2. Get auth keys by instance before joining DASonBeforeStartService[nodeGroup:instance] { “cmd”: “/root/scripts/pullAccessKeys.sh ${token}” }
THE FULL WORKING AND MORE COMPLEX CODE EXAMPLE CAN BE FOUND HERE
1. Decrypt encrypted volume, that was attached on-fly
onAfterVolumeAttached[nodeGroup:instance] { “call”: “decryptVolume” }
3. Deprovision node from DAS if cluster was shrinked
onAfterScaleOut[nodeGroup:das] { “cmd”: “./asadmin_proxy remove ${instance[@last].ip}” }
Tricky thing #3Proper triggers configuration
This tuning is a long process, so be patient
Typical problem #1
Changing the value in one place can be reflected in other places
How to deal with this?
Load Testing
Gathering
Metrics
Detect deviations in a testing loop
Analyz
ing
Metrics
Edit
Trigg
ersCompare previous
results
Typical problem #2The triggers can become outdated with new code
The fix is easy here
Inject metrics collection as a part of your CI/CD step
COMMIT BUILD TESTING PERFORMANCEANALYSIS DELIVERY
Tricky thing #4Getting rid of spare things
Typical problem #1Network is full of useless traffic
Total bandwidth Multicast | HeartbeatsUseful traffic
What we can do?
Avoid multicast (for members detection) Use events and static members lists
Static members list Static members list Static members list
Orchestrator
Typical problem #2Some of JavaEE Servers are designed for VMs, not for containers
On example of Weblogic
VM VM VM VM
What is the right way to go?Just remove everything spare
LIVE demo comes here
QUESTIONS
