Dynamic Routing at 1 Million Messages per Second with Spring Integration

Dynamic Routing at 1 Million Messages per Second with Spring Integration

John Davies, CEO, Incept 5 @jtdavies !Josh Long, Spring Developer Advocate, Pivotal @starbuxman

!2

SPRING INTEGRATION 101

Copyright © 2013 Incept5 Ltd. http://www.incept5.com

what is “EAI”?

EAI encompasses approaches, methodologies, standards, and technologies allowing very diverse but important systems to share information, processes, and behavior in support of the core business.

-David Linthicum

“ ”

http://www.incept5.com


what is “EAI”?§ file transfer § shared database § remote procedure call § messaging



what is “EAI”?



Spring Integration

§ At the core, an embedded Message Bus § Also, an Application Integration

Framework § Support Enterprise Integration Patterns*

via the Spring programming model § Build on the Spring Platform

n (n-1)2

A

C

B

D



Spring Integration§Payload can be any object §Header values are stored in a read-only java.util.Map<K,V> §Similar to JMS messages, email mime envelopes, XMPP messages, etc.

public interface Message<T> { MessageHeaders getHeaders(); T getPayload(); }



Spring Integration§Decouples producers from consumers § provide extension points for interceptors §messages may be queued and buffered

<channel id="async-p2p"> <dispatcher task-executor=”simpleAsyncThreadPool" /> <queue capacity=“10” /> </channel>

producer consumermessage channel receive(Message)send(Message)

point to point

publish-subscribe<publish-subscribe-channel id="async-pubsub” task-executor="someThreadPool" />

producer message channel

receive(Message)

send(Message)

consumer

consumer

consumer

!!!{ !

!!!{ !



Spring Integration

transformers: convert payloads and modify headers

filter: discard messages based on test

router: determine next channel based on message and test

splitter: generate multiple messages from one

aggregator: assemble a single message from multiple messages

service activator: delegates processing to a backend service call



Spring Integration

<channel id = “customerChannel” /> !<jms:outbound-channel-adapter channel = “customerChannel” destination-name=“stocks” connection-factory = “connectionFactory” /> !

producer

jms:outbound-adapter

send(Message) jmsTemplate.send(javax.jms.Message)ActiveMQ



Spring Integration

<channel id = “fileChannel” /> !<file:inbound-channel-adapter channel = "fileChannel " directory= "#{systemProperties['user.home']}/Desktop/in "> <int:poller fixed-rate="1000"/> </file:inbound-channel-adapter> ! file:inbound-channel-adapter

new File(directory).list() Message<File> mf = fileChannel.receive() consumerDocumentDocumentDocument



Spring Integration

!MessageChannel channel = ... Message<Customer> customerMessage = MessageBuilder.withPayload(customer) .setHeader("customer-id", 10) .build(); channel.send(customerMessage);


!13

DEFINING PERFORMANCE


We need a good safety margin

• If we can process 200k/sec and we get a peek of 400k for an hour (e.g. Christmas, Thanksgiving, breaking news etc.) • We have a backlog of 200k/sec for 3600 seconds, if the volume drops to

150k/sec it will take 4 hours to catch up

• And we’ll need 360 GB of RAM to queue it! And that’s just 500 bytes/msg !

• RabbitMQ was used at a large “fruit” vendor • Queue sizes were critical in the planing of the deployment !

• Even working at 500k/sec and a peak of 550k for 30 minutes dropping to 400k • 15 minute’s delay and 45 GB of RAM !

• We need “REALTIME”



• In order to do any work on a message we need to parse it • For XML that’s a SAX parser, DOM or Java Binding (JAXB, JIBX etc.) !

• If you’re going to store, search or deliver messages these are the basic steps… • Parse the message or part of the message you need

• Query and route the message or…

• Query to search the message

• - Possibly serialise and de-serialise the message

• - Possibly enrich or transform the message

• Write the message out (if it’s been bound to Java) !

• All this takes time :-(

Where’s the problem?



Java Binding

• Keeping the example simple a CSV generates something like this…

public class Row extends ComplexDataObject {! private static final long serialVersionUID = 1L;! private String name;! private String cardNumber;! private String expiryDate;! private double amount;! private boolean isamountSet;! private String currency;! private Date transactionDate;! private double commission;! private boolean iscommissionSet;! private long vendorID;! private boolean isvendorIDSet;! private String country;

Name,Card Number,Expiry Date,Amount,Currency,Transaction Date,Commission,Vendor ID,Country!Stephen Hawkins,4325-6486-3757-2674,10/06,100,GBP,26-09-2006,2,14988603,UK!Tim Berners-Lee,4724-7345-4725-7833,11/07,258,USD,21-12-2006,5,15688632,UK!Bill Clinton,4924-7264-1264-8532,04/09,1250.6,USD,13-09-2006,15,66846035,US!Angela Merkel,4457-4356-0087-0107,05/08,350,EUR,13-11-2006,7,93440252,DE!Richard Branson,4724-7345-4725-7833,11/06,250,USD,22-02-2006,5,14988103,UK!Tim Cook,4924-7264-1264-8532,04/09,12250,USD,16-09-2006,1.3,67434435,US!Alan Turing,4325-6486-3757-2674,10/06,50,GBP,23-02-2006,1,14119663,UK



Message Parsing

• Taking a message with about 50-60 attributes and parsing them so that they can be available to Spring Integration/Batch is not exactly complex !

• But providing the ability to manage the changes and dozens of standards starts to add complexity • Things change over time, we need to manage that change too

• We’d also like a consistent API so Java-Binding is still ideal !

• Two relatively simple examples from the Telco and Financial Services industry are RADIUS (rfc-2865) and FIX (from FPL) • Others include ASN.1 & ISO-8583



Radius (rfc-2865)

• Remote Authentication Dial In User Service (RADIUS) • 77 pages of binary spec...



Modelling in C24 first

•The RADIUS standard...



What we can now do

• The spec may say that bit 13 of a 32 bit field represents the presence of a field ABC (later in the message) • Programatically we can test bit 5 with a mask 0x00002000

• Using the generated code we can simply call isAbcSet() !

• It may say that bits 4-6 represent the version ID • Programatically we can mask it and then shift it...

• mask 0x00000070

• shift >> 4

• Using the generated code we can simply call getVersionId() !

• We now have a nice API that hides the complexity of the binary implementation



Bits and Bytes to…

• We can now go from this… !

jd-server:Radius TestData jdavies$ hexdump -C radius.dat 00000000 01 02 00 74 ea d5 7c 62 1f d0 f6 fe a3 bf 36 4c |...t..|b......6L| 00000010 35 25 e5 8c 1a 17 00 00 28 af 01 11 32 33 34 34 |5%......(...2344| 00000020 35 37 30 36 32 37 38 38 35 33 36 01 11 32 33 34 |57062788536..234| 00000030 31 35 39 30 36 32 35 38 38 35 33 36 1f 11 33 35 |159062588536..35| 00000040 33 34 32 31 30 32 30 39 34 35 35 36 38 5e 0e 34 |3421020945568^.4| 00000050 34 37 30 30 34 31 38 38 36 37 33 1a 0d 00 00 28 |47004188673....(| 00000060 af 08 07 32 33 34 33 35 06 06 00 00 00 02 37 06 |...23435......7.| 00000070 4e 97 57 a8 |N.W.| 00000074

!

• To being able to use it in Spring • Or Mule, Fuse, Camel etc…



To Spring Integration...

• Test bits and binary data in native Spring…

!

• <filter input-channel="filter-message-channel" output-channel="process-message-channel" ref=”payload” method=”isAbcSet"/>

!

• <filter input-channel="filter-message-channel" output-channel="process-message-channel" expression="payload.versionId == 5"/>



A good start but…

• We now have a pretty cool API and can use Spring but it’s still pretty slow as it generates LOTS of objects !

• With our telco standards binary to Java binding was “slow” due to the number of objects, we got around 20k/sec/core • Fine for most purposes and with an 8 core machine potentially good

for 100k/sec but we wanted better !

• We needed a better solution, it wasn’t the parsing but the complexity of the objects we were creating and parsing into



ByteBuffer

• Java 1.4 added java.nio.ByteBuffer

• Basically a wrapper for a byte[]



Lazy parsing

• Rather than parse all of the elements from the message every time we retrieve the elements only when we need them • Similar to comparing a DOM with a SAX parser

• We assume that we will only need to filter/sort/query on a limited number of fields so we save a lot of redundant parsing !

• Performance goes from ~20k/sec (50µs per message) to ~1 million/sec (1µS per message), some 50 times faster • Even parsing the entire message is significantly faster



Here’s where it gets interesting...

• So we now have a Java object with a ByteBuffer holding the message data and dozens of get() methods to get the content • Message call = new Message(data);

• call.getDuration();

• This works pretty fast, plus the JIT compiler kicks in after 10,000 iterations and optimises the method

• Now the bottleneck moves to the SpEL queries • <filter input-channel="filter-message-channel" output-channel="process-message-channel"

expression="payload.duration lt 0.1"/>



Reflection

• What’s happening under the hood is reflection, very powerful but sadly still rather “slow” • expression="payload.duration lt 0.1"

• Turns into something like...

!

!

!

• This adds about 700nS to each message • A few of these and we’ve more than halved the performance



The Java Compiler to the rescue!

• New from Java 1.6: ToolProvider.getSystemJavaCompiler();

• We can create a generic accessor for double values...

!

!

• Can now write a class on the fly that implements this method for the getter we want e.g. getDuration()



Java on the fly...



Compile it...



Running it...

• Load up the compiled class (or byte-code) and run it

!

!

• Note however that the first two lines (above) are done only once, outside of the loop

!

• The result is “native” performance as if it was code • Which of course it is



Back to spring...

• SpEL expressions are currently interpreted

!

• There is some basic caching • java.lang.reflect.Field

• java.lang.reflect.Method objects are cached once discovered and reused on subsequent evaluations !

• But overall they are “slow” (for what we need) • We need to look at avoiding reflection by compiling the parsed

expression into a class



SpEL compiler

• Currently proposed by Andy Clement (Spring guru) is a SpEL compiler • With the proposed changes SpEL has a Mixed Mode Interpreter

(MMI) system in place

• Mixed means it mixes the current interpreter with a real expression compiler • When an expression is evaluated X number of times SpEL compiles it to byte-code and

uses that going forward

• User does nothing, evaluations just accelerate



Does it make a difference?

• Property access:foo.bar.boo



Does it make a difference?

• Method invocation:hello()

!

!



Advantages - Performance

• We can now parse, compare and filter in around 1 µSec (1 million/sec) per core • Make decisions on what you want early on and drop unwanted

messages

• Provide “on-the-fly” aggregation

• Around 90-95% less CPU per message - $€£¥₹ !

• Disadvantages • Less error checking

• Hardware vendors make less money

• Machine rooms become uncomfortably cold

• Power bills go down



Other Advantages

• Messages in memory are binary encoded in ByteBuffers and so occupy a fraction of the size of a bound Java object • Typically around 1/12 of the size, using only 8% of the memory !

• We can queue around 10 times more messages per unit of memory !

• We get around 1000% capacity advantage for in-memory grids - GigaSpaces, Hazelcast, GemFire, Coherence !

• We can provide custom serialisation of the ByteBuffer for all of the above - boosting node to node replication



So - In a nutshell

• We’ve gone from 20k per second to around 800k • 40 times faster, sometimes better !

• We’ve gone reduced memory footprint by around 12 times • Meaning you can store 12 times more data in your grid

• Or use 1/12th of the hardware/memory !

• You can route this with Spring at “native” JIT-compiled Java code speeds • So far no other framework can offer this level of efficiency, dynamic

flexibility and performance



The Future

• SpEL to native Java compilation to be included in Spring release • A few issue remain to be resolved in the short term but early versions

look good !

• Full support for standards like ASN.1 • This is a standard wire-level implementation option for ISO-20022

• ASN.1 supports multiple encoding rules, Basic, XML, Canonical, Packed etc. !

• XML messages encoded as packed ASN.1 (PER) would be almost as performant as raw binary • Complex XML as packed binary in a ByteBuffer



More Spring

• Read more on the tools we used... !

• Spring XD - http://projects.spring.io/spring-xd/

!

• Spring Integration - http://projects.spring.io/spring-integration/

!

• Spring Batch - http://projects.spring.io/spring-batch/

!

• Spring AMQP - http://projects.spring.io/spring-amqp/


http://projects.spring.io/spring-xd/

http://projects.spring.io/spring-integration/

http://projects.spring.io/spring-batch/

http://projects.spring.io/spring-amqp/


Thank you!

John Davies

Twitter: @jtdavies

LinkedIn: http://linkedin.com/in/jdavies/

E-Mail: [email protected]

C24: http://www.c24.biz

+Josh Long

Twitter: @starbuxman

GitHub: http:// github.com/joshlong

E-Mail: [email protected]

Spring & Spring Integration: http://spring.io


http://linkedin.com/in/jdavies/

http://www.c24.biz

http://linkedin.com/in/jdavies/

http://www.c24.biz

http://www.c24.biz

Technology

Dynamic Routing at 1 Million Messages per Second with Spring Integration