Embed Size (px)
Citation preview
Performance ofMicroservice frameworkson different JVMs
Agenda
1. Introduction
2. Microservice frameworks
3. JVMs
4. Test setup
5. Results
6. Recommendations
Who am I?
Who is Maarten?
• Principal Consultant Software Architect at AMIS / Conclusion
• Several certificationsSOA, BPM, MCS, Java, SQL, PL/SQL, Mule, AWS, etc
• Enthusiastic presenter and bloggerhttp://javaoraclesoa.blogspot.com
@MaartenSmeetsNL
https://nl.linkedin.com/in/smeetsm
What is the CJIB
• The Central Judicial Collection Agency part of the Ministry of Justice and Security in the Netherlands
• The CJIB is responsible for collecting a range of different fines, such as traffic fines and punitive orders.
• Works together with EU Member States when it comes to collecting fines.
• Plays a key enforcement role in decisions relating to criminal matters, such as• court rulings• decisions made by one of the Public Prosecution Service’s public
prosecutors
• Located in Leeuwarden, Friesland
Where do I work?
Traffic fines
9.223.477Fees cost awards
3.846
CJIB: Key figures 2017
Fines imposed by the court
57.900
Compensation measures
13.563Aid with problematic debts
Transaction proposals
4.575PrincipalCustodial sentences
13.485
Administrative premium
2.081.270
Public ProsecutorService settlements
284.642
Coordination of community service
36.630
Incoming European fines
1.038Outgoing European fines
49.766Confiscation measures
1.690
Conditional release
1.043
Administrative fines
40.608
Supervision
15.021
Converted communityservice sentences
7.657
Youth authority
5.258
The CJIB and Java
• 1400 people. ICT department of around 325 people. 100 Java developers
• 30 teams using Scrum and SAFe. Tight integration between business and IT
• Solid CI/CD pipelines and release train• Automated testing using Cucumber, Gherkin• Code quality checks using SonarQube• Bamboo, Puppet, Git, Maven, Vault
• Running on Redhat 7, OpenJDK 8 with Spring Boot Microservices on Jetty
• Innovation lab• Blockchain• Machine Learning
CJIB ICT
Disclaimer
The performance tests mentioned in this presentation were conducted with intention to obtain information on what performance differences can be expected from running various frameworks in various JVMs using various garbage collection algorithms. A best effort has been made to conduct an unbiased test. Still, performance depends on many parameters such as hardware, specifics of the framework implementation, the usage of different back-ends, concurrency, versions of libraries, OS, virtualization and various other factors. I cannot provide any guarantees that the same values will be achieved with other than tested configurations. The use of these results is at your own risk. I shall in no case accept liability for any loss resulting from the use of the results or decisions based on them.
Let yourself be inspired and perform your own tests!
Spring Fu
Microservice frameworks
Code testedWhich framework parts are tested
• HTTP server
• Routing. Mapping of HTTP request to Java method
• Parsing of the URL for a GET request
• Creating a response object based on data from the request
• Serialize the response object to JSON
HTTP server
Routing
Request parsing
Response generation
Response serialization
Microservice frameworks
No framework
• Minimal implementation using Java SE code only
• No reflective code and no frameworks
• Easy native compilation
• Of course the fastest option which uses least resources
Microservice frameworks
Spring Boot
• Spring Boot makes it easy to create stand-alone, production-grade Spring based Applications that you can "just run".
• An opinionated view of the Spring platform and third-party libraries so you can get started with minimum fuss. Most Spring Boot applications need very little Spring configuration.
• Supported by Pivotal / VMware
Microservice frameworks
Spring Boot Reactive / WebFlux
• Spring WebFlux is
– fully non-blocking
– supports Reactive Streams back pressure
– and runs on such servers as Netty, Undertow, and Servlet 3.1+ containers
• Supported by Pivotal / VMware
Microservice frameworks
Spring Fu
• Spring Fu is
– an incubator for Kofu (Ko for Kotlin, fu for functional)
– provides a Kotlin API to configure Spring Boot applications programmatically
– allows for native compilation on GraalVM while ‘regular’ Spring does not
Microservice frameworks
Quarkus
• A Kubernetes Native Java stack tailored for GraalVM & OpenJDK HotSpotcrafted from the best of breed Java libraries and standards
• Extensions configure, boot and integrate a framework or technology into your Quarkus application. These extensions also provide the right information to GraalVM for your application to compile natively
• Supported by Red Hat / IBM
Microservice frameworks
Eclipse Vert.x
• Eclipse Vert.x is event driven and non blocking.
• You can use Vert.x with multiple languages including Java, JavaScript, Groovy, Ruby, Ceylon, Scala and Kotlin.
• Vert.x is flexible and unopiniated
• Quarkus provides a Vert.x implementation
• Supported by the Eclipse foundation
Microservice frameworks
Akka
• Akka is a toolkit for building highly concurrent, distributed, and resilient message-driven applications for Java and Scala
• Actors and Streams let you build systems that scale up, using the resources of a server more efficiently, and out, using multiple servers
• Supported by Lightbend
Microservice frameworks
Micronaut
• A modern, JVM-based, full-stack framework for building modular, easily testable microservice and serverless applications.
• Micronaut uses no reflection. This makes it easier for Micronaut applications to run on GraalVM.
• You can run Spring applications as Micronaut applications making native compilation possible (Micronaut for Spring)
• Supported by Object Computing
Microservice frameworks
Helidon
• Helidon SE
– Microframework
– Functional style
– Reactive
– Can be natively compiled
• Helidon MP
– MicroProfile
– Declarative style
– CDI, JAX-RS, JSON-P
– Cannot be natively compiled (CDI)
• Supported by Oracle
Microservice frameworks
Open Liberty. A Microprofile implementation
• An open forum to optimize Enterprise Java for a microservices architecture by innovating across multiple implementations and collaborating on common areas of interest with a goal of standardization.
• Specifies the least number of Java Enterprise specifications APIs required to build a microservice.
JVMs differ
• Licensing / support
• Memory usage / NUMA support
• Garbage collection algorithms
• Start-up caching
• Other features
JVMs
Test setupFramework versions used Hardware used
• Intel Core i7-8700• hexa-core• 12 threads• Max 4.6GHz
• 32Gb DDR4L (2400MHz)
OS used
• Running Linux Mint 19.1 (Tessa)based on Ubuntu 18.04LTS (Bionic Beaver)
• Docker server 18.06.1-ce client 18.09.2
Framework Version HTTP server
Spring Boot 2.1.4 Tomcat
Spring Fu 0.0.5 Reactor Netty
WebFlux 2.1.4 Reactor Netty
Akka 2.12 10.1.8 Akka HTTP
Open Liberty 19.0.0.4+ Open Liberty
Vert.x 3.7.0 Vert.x
Quarkus 0.15.0 Reactor Netty
Helidon 1.1.1 Helidon webserver
My laptop
What did I do?
• Create minimal but comparable implementations for every frameworkJava, Kotlin
• Create a script to loop over scenariosJVMs, Microservice implementations, GC-algorithms, etcBash, Python
• Create multithreaded load generators and compare resultsPython, Node/JavaScript
• Containerize the implementations; makes testing JVMs and resource isolation easyDocker
• Summarize results (determine average, standard deviation, Prometheus results)Bash (awk, sed, curl), Python, Apache Bench
• Run the setup under various conditions First priming followed by 15 minutes per framework per JVM per variable (weeks of data)
• Visualize resultsPython / Jupyter: pandas, numpy, pyplot
Test setup
Test setup response times15m per test
Docker base image with JVM
Microservice framework fat JAR
Load generator(Python)
Build and run containerJVM + framework
Load generation
runtest.shLoop over JVMs and frameworks
Start processes, clean up and summarize results
outputfile.txt
groupedbarplot.py
Data validation
Data description
Data visualization
Generate and summarize data Validate and visualize
results.txt(measures per JVM
per framework)
messages
Test setup throughput + feature importance20s per test (many variables)
• No more containersEffects of containers are slower startup and Docker NAT overheadNo more need to create and clean up containers -> simpler code
• Apache Bench for testingNo more split individual measures and summary dataProven simple to use tool (also used to test Apache HTTPD)
• Python for running the tests / manage the processesEasier than Bash for complex testing
• Jupyter for visualization instead of plain PythonDisplaying graphs and developing the script in small bits is easier
• Looked at many variables Average response time and throughput
– Concurrent requests1,4,8
– CPU cores assigned to the JVM1,4,8
– Memory512mb, 128mb, 50mb
– Frameworks10x
– GC algorithmsall available for every JVM
– JVMs OpenJDK, Oracle JDK, OpenJ9, Zing
– JVM version8,11,12,13
Test setup throughput + feature importance20s per test (many variables)
JVM
Microservice framework fat JAR
Apache Bench
Build and runJVM + framework
Load generation
run_test.pyLoop over all variables
Start processes, clean up
feature importance.ipynb
Data validation
Data description
Data visualization
Generate and summarize data Validate and visualize
results.txt
messages
Resource isolation
User processes
Load generators
JVM process
1 32
• Images available on Docker Hub
– OpenJDK
– AdoptOpenJDK
– Oracle GraalVM
– Amazon Corretto
– Eclipse OpenJ9
– Azul Zulu
• Images not available on Docker Hub (due to license restrictions)
– Oracle JDK
– Azul Zing
Running in a container
ContainersResults
• Hosting on Docker gives worse response times than hosting locally• Docker NAT is expensive!
• Application startup in Docker is worse• Shared libraries need to be loaded• JVM caching features are not available or difficult to implement
• Docker to Docker is not faster than local to Docker
• Everything outside a container is fastest
Microservice frameworksWhich framework gives best response times
• Akka gives worst performance. Vert.x best.
• Reactive frameworks (Akka, Vert.x, WebFlux)
do not outperform non-reactive frameworks (Microprofile, Quarkus, Spring Boot, Spring Fu)
Java versionsWhat happens when migrating from Java 8 to Java 11?
• Java 8 and 11 behave very similarly for every frameworkJDK 11 is slightly slower than 8
• OpenJ9 benefits most from going to JDK 11Especially for Spring Boot and Akka
Res
po
nse
tim
e [m
s]
Java version
JVMsWhich JVM gives best response times?
• OpenJDK and Oracle JDK perform similarly for every framework (no consistent winner)
• Substrate VM (native compilation) does worse than Oracle JDK and OpenJDK
• OpenJ9 does worst for every framework followed by Zing
Application startup
Outside a container1Gb heap
Application startupNative compilation and startup
• Native compilation (Substrate VM) greatly reduces start-up time
• OpenJ9 is fastest to start outside a containerShared classes cache (SCC) and dynamic ahead-of-time (AOT) compilation (?)
• Zing and OpenJ9 are slow to start in a containerHave not looked at Zing Compile Stashing and ReadyNow! features
• Quarkus, Helidon SE, Vert.x start approximately 5x faster than Spring Boot and approximately 10x faster than Open Liberty!
• OpenJ9 BalancedDivide memory in individually managed blocks. Good for NUMA
• OpenJ9 MetrononeGarbage collection occurs in small interruptible steps
• OpenJ9 OptAvgPauseUses concurrent mark and sweep phases. Reduces pause times
• OpenJ9 OptThruPutOptimizes on throughput but long pause times (app freezes)
• OpenJDK Shenandoah GC (Java 12)A low pause time algorithm which does several tasks concurrentlyNo increased pause times with a larger heap
• OpenJDK / Oracle JDK ZGC (Java 12)Scalable (concurrent) low latency garbage collector
• Zing C4 GCContinuously Concurrent Compacting CollectorPauseless garbage collection
• OpenJDK / Oracle JDK G1GC (default Java 9+)Compact free memory space without lengthy pause times
• OpenJDK / Oracle JDK Parallel (default Java 8)High-throughput GC which does not allow memory shrinking
• OpenJDK / Oracle JDK ConcMarkSweep (EOL?)Designed for lower latency / pause times than other parallel collectors
• OpenJDK / Oracle JDK Serial GCSingle threaded GC which freezes the application during collection
• OpenJ9 Generational Concurrent policyMinimize GC pause times without compromising throughput
GC algorithms
GC algorithmsHow do GC algorithms influence response times (2Gb heap)
• OpenJ9 did worst (with and without sharedclasses)Metronome does best for OpenJ9
• Every JVM (OpenJ9, Zing, OpenJDK) achieves similar performance for every available GC algorithm (at 2Gb heap!)
• OpenJDK Serial GC did best
OpenJDK
OpenJ9
Zing
GC algorithms 20mb heap
AdoptOpenJDK Eclipse OpenJ9
How do GC algorithms influence response times (20Mb heap)
• GC algorithms influence the minimal required memoryOpenJ9 Metronome GC gave out of memory
• When memory is limitedDo not use Shenandaoh (30ms) or parallel GCOpenJ9 does better than AdoptOpenJDK
• Azul Zing cheated!‘Warning Maximum heap size rounded to 1024 MB’
• OpenJ9 with OptThruPu GC is the winner!produces best performance on limited memory
• For AdoptOpenJDK Serial GC does best
• Which feature is most important in determining response times and throughput?
– Garbage collection algorithms
– Concurrency
– JVM supplier
– JVM version
– Microservice framework
– CPUs
– Memory
• Determine feature importance using Random Forest Regression
Feature importance
ThroughputResponse time
Feature importance
• Framework choice is very important when considering performance (~70%)Quarkus, Vert.x, Helidon SE
– are fast to start
– can be natively compiled
– give best throughput and response times
• Low on memory: consider OpenJ9 and think about garbage collection
• Is performance important? Don’t run in a container!
– Docker NAT is overhead (responsetime, throughput)
– JVM caching features (Zing, OpenJ9) are hard to use (start-up)
Recommendations
• JDK 11 has slower start-up times and slightly worse performance than JDK 8
• OpenJDK variants (GraalVM, Corretto, AdoptOpenJDK, Zulu) and Oracle JDK perform pretty similarly. Does not matter much which one you use when looking at performance
• Native images have way faster startup time but worse response times. Many frameworks and libraries do not support this yetCloud / Serverless environments are good use cases to consider native images
Recommendations
Choices the CJIB made
Framework: Spring BootQuick for development and can run standalone
Jetty servlet engineEfficient in memory usage and performance
OpenJDK 8 on RedHat 7 (moving to Kubernetes)
CJIB already has RedHat licenses for supportRedHat is steward for OpenJDK 8 and 11Spring Boot runs well on OpenJDK
Garbage CollectionDefault as long as no performance issues
Choices the CJIB made
Suggestions
• Sources:https://github.com/MaartenSmeets/jvmperformance
• Get help from the JVM and framework suppliersSeveral got in touch and provided valuable feedback
• Do your own tests. Your environment and application is unique. Results might differ
Questions?
@MaartenSmeetsNL
https://nl.linkedin.com/in/smeetsm
