Upload
dmitriy-dumanskiy
View
459
Download
0
Embed Size (px)
Citation preview
Vanilla java or handling 10k req/sec per core
Dmitriy DumanskiyBlynk
Co-Founder / Java / Android
Java blog : http://habrahabr.ru/users/doom369/topicsDOU : http://dou.ua/users/DOOM/articles/
Experience
mGage - 3k req/secmobclix - 6k req/secxxxx - 11k req/sec
Visa
Load : ~2000 req/sec
Twitter : new tweets
Load : ~6000 req/sec
Facebook messenger
Load : 300k req/sec
IoT Market
● 4.4 billion devices right now (70$ b)● 25 billion devices per 2020 (270$ b)
Makers problem
+ = ?
Problem
● Thousands of connections● Keep-alive● Security● Pulling / polling● Simplicity
IoT Protocols
HTTP, MQTT, XMPP, AMQP, CoAP
Minimum packet size
● HTTP - 26 bytes (~70 bytes)● MQTT - 2 bytes● Blynk - 3 bytes
Blocking IO
connection new Socket() new Thread() read / writeServer Socket
Blocking IO
● 256kb stack● 1k threads == 250 MB of RAM● Сontext switching
Non-Blockingnew Socket()
read / write
Selector
Thread
new Socket()
read / write
new Socket()
read / write
Non-Blocking
● Few threads. Usually - n cores x 2.● No context switching● No memory consumption
Non-blocking frameworks● Apache MINA● Grizzly● Xnio● Vert.x● Akka● Netty
Non-blocking frameworks● Apache MINA● Grizzly● Xnio● Vert.x● Akka● Netty
Non-blocking frameworks● Apache MINA● Grizzly● Xnio● Vert.x● Akka● Netty
Non-blocking frameworks● Apache MINA● Grizzly● Xnio● Vert.x● Akka● Netty
Non-blocking frameworks● Apache MINA● Grizzly● Xnio● Vert.x● Akka● Netty
Netty
Netty cons
Netty is hard. 2-3 weeks onboarding
Netty cons
Bad documentation
Netty cons
Too many abstractions
Netty cons
Easy shot in the foot
Netty cons
ByteBufin.readBytes(length) - memory leak
in.readSlice(length) - ok
Netty cons
Still has bugs, 300 issues
Netty pros
Full control
TCP_FASTOPEN, TCP_MD5SIG, TCP_CORK, TCP_KEEPCNT, TCP_KEEPIDLE, TCP_KEEPINTVL,
SPLICE
Netty pros
Supports everything
TCP, UDP, UDT, SCTP, HTTP, HTTPS, HTTP/2, SPDY, Memcached, MQTT, etc
Netty pros
A lot of default Handlers
Netty pros
Fast
Netty pros
Super quick community help
Blynk architecture
https://github.com/blynkkk/blynk-server
Open source
https://github.com/blynkkk/blynk-library
Blynk stats
● ~2000 of hardware always online● ~300 users always online● ~1000 local Blynk Server installations● ~3 Mbps in traffic● ~7 billions requests per month
Blynk load
● 6% CPU with 2500 req/sec on 4 cores● 10k req/sec per core● 90% of requests - 3 handlers● Only 3% requests are HTTP
Why so fast?
No blocking IO
Request
BE
Blocking IO Workers
Response
Batches
Request
BE
Response
Collector1 min batch File System
What is your DB?
try (Writer writer = Files.newBufferedWriter(file)) { writer.write(user.toString());}
No synchronization
Client 1connect
Selector Thread 1assign
Client 2connect
Selector Thread 2assign
Client 2login
Logic Thread 1reregister
No GC pressure
ctx.write(new Response(mId));vs
ByteBuf buf = ctx.alloc().directBuffer(1);buf.writeByte(mId);
ctx.write(buf);
In memory
Set<User> users = JsonParser.parseUsers(dir);
+ 1 min batches
Native transports
● Native epoll● Native OpenSSL
Version 1
Blynk monolith
80$
Version 1. Tech stack
● Vanilla Java 8● Netty (HTTP, WebSockets, TCP/IP)
Version 1. Pros
Simple single jar deploymentjava -jar server.jar
100ms start
Version 1. Pros
Simple backup script
Version 1. Pros
Single JVM, no cross-JVM shared state
Version 1. Pros
Entry level
Version 1. Cons
Single point of failure, single datacenter
Version 1. Cons
Latency is bad for distant locations
Version 1. Cons
Need to drop connections on redeploy
Version 2
ClientGet DNS record EUROPE
USA
ASIAGeo DNS
50$
Version 2
● Improved latency● No common storage anymore
Version 2
USA
Clients
DC 1
EU
Clients
DC 2
ASIA
Clients
DC 3
Version 3
USA
Clients
DC 1
EU
Clients
DB
ASIA
Clients
DC 3
Version 3. DB Cassandra
Node 1
Node 3
Node 2
Node 3
Node 2
Node 1
Node 3
Node 2
DC 1 DC 2
Node 1
15$
Version 3. DB Cassandra
● No typical batches● Minimum requirement 1GB of RAM● Performs poorly on low-end VMs● Much slower than Postgres
Version 3. Final
USA
Clients
DC 1
EU
Clients
Postgres
ASIA
Clients
DC 365$
Version 3. Final
● 60k req/sec for 65$ (25x for current load)● + 10x growth to 20 CPU● Samsung cloud - 1 req/sec per 6$
Q & A