Asynchronous programming using CompletableFutures
Agenda
• What is asynchronous programming?
• Why should I care?
• How to use CompletableFuture for asynchronous calls?
• Does it worth it?
What is asynchronous programming?
Latency Numbers
0.5
25
100
0 20 40 60 80 100 120
ns
Main memory reference
Mutex lock/unlock
L1 Cache Reference
Latency Numbers
100
10,000
150,000
0 20000 40000 60000 80000 100000 120000 140000 160000
ns
Read 4K randomly from SSD
Send 1K over 1 Gbps
Main memory reference
Latency Numbers
150,000
250,000
500,000
10,000,000
0 2,000,000 4,000,000 6,000,000 8,000,000 10,000,000 12,000,000
ns
Disk seek
Round trip within samedatacenter
Read 1 MB sequentially frommemory
Read 4K randomly from SSD
The Hollywood Principle
Don’t call us,
we’ll call you
Why should I care?
Why should I care?
• Process scheduling in modern kernels is pretty mature
• Re-implementing it in user-space is an anti-pattern
• Java relies on kernel process scheduling
It’s about
• Controlling thread lifecycle
• The Nr. of threads and context switches
• I/O operation optimization
Non blocking I/O
Why should I map number of threads to number of connections?
It doesn’t make sense.
The Selector
Client 1
Client 2
Selector
Handler
Handler
Anatomy of a typical Java application
? ?
Your code goes here
How to use CompletableFuturefor asynchronous calls?
A Simple Example
Combining Asynchronous Calls
Using Executors Independently
We Have Exception Handling!
Tomcat 8
HelloService
Small I/O
Scenario 1
Tomcat 8
HelloService
125ms
Scenario 2
In-memory
Tomcat 8
HelloService
125ms
Scenario 3
Tomcat 8
HelloService
Small I/O
Scenario 4
4 threads
Pros & Cons
Pros
• Combinable executors
• Exception handling
• Fast enough execution
• Java language support
• Easy
Cons
• You have to know what you’re doing
• Can be blocking
• Can be slower if Executor scheduling is a bottleneck
Design Considerations
• Blocking 1000 threads because of service failure? Not a smart decision
• Queueing 1000 request-timeouts because of same reasons? Bad idea.• Bit better choice: less pressure on downstream services
• Choose threading and scheduling mechanism wisely• ForkJoinPool is most of the time a bad choice
• Monitor and purge request queues in case of a specific failure rate• Hystrix offers similar mechanism
Questions?
References
GitHub exampleshttps://github.com/gitaroktato/completablefuture-examples
CPU Schedulinghttps://www.cs.uic.edu/~jbell/CourseNotes/OperatingSystems/5_CPU_Scheduling.html
CompletableFuture & Spring Examplehttps://spring.io/guides/gs/async-method/
Non-blocking IO in Javahttps://dnhome.wordpress.com/2012/06/28/java-non-block-io-new-io/
References
CompletableFuture API Examples
https://www.slideshare.net/jpaumard/asynchronous-api-in-java8-how-to-use-completablefuture
Grizzly NIO frameworkhttps://grizzly.java.net/documentation.html
Scalable IO in Javahttps://www.slideshare.net/giridhar510/scalable-io-in-java
Non-blocking IO in Javahttps://dnhome.wordpress.com/2012/06/28/java-non-block-io-new-io/
RxJava – Multithreadinghttps://praveer09.github.io/technology/2016/02/29/rxjava-part-3-multithreading/