Embed Size (px)
DESCRIPTION
In many applications, there is a tension between how much you can log without slowing down your application, and how much information you would like to have. Chronicle provided a number of solutions which allow you to record millions of events per second, with micro-second latencies in a persisted way without contributing to your garbage. How does this simplify the design, help you increase the determinism and vertical scalability of your application?
Citation preview
Peter Lawrey
CEO and Principal Consultant
Higher Frequency Trading. Presentation to JDD 2014 Krakow, October 2014.
High Performance Logging Good bye debug logging
Higher Frequency Trading is a small consulting and software development house specialising in: • Low latency, high throughput software • 6 developers + 2 staff in Europe and USA.
• Sponsor HFT related open source projects • Core Java engineering
About Us…
• CEO and Principal Consultant • 3rd on Stackoverflow for Java, most Java Performance answers.
• Founder of the Performance Java User's Group
• An Australian, based in the U.K.
About Me…
"I want to be able to read and write my data to a persisted, distributed system, with the speed of in memory data structures"
What Is The Problem We Solve?
Chronicle scales Vertically and Horizontally If you could log everything fast enough would you need logging levels? Would there be such a thing as Debug level logging?
l Shares data structure between processes l Replication between machines l Build on a low level library Java Lang. l Millions of operations per second. l Micro-second latency. No TCP locally. l Synchronous logging to the OS. l Apache 2.0 available on GitHub l Persisted via the OS.
Chronicle scaling Vertically and Horizontally
l Logging Adapters for Chronicle Queue. l Synchronous logging avoids lost messages on a crash. l Similar speed to Log4j2 using asynchronous logging l Designed to be read in real time by other JVMs (with a micro-seconds) l Can be distributed over TCP and aggregated for centralized monitoring. l Apache 2.0 open source library
What is Chronicle Logger?
l Low latency journaling and logging. l Low latency cross JVM communication. l Designed for reactive programming l Throughputs up to 40 million/second. l Latencies between processes of 200 nano-seconds. l Sustain rates of 400 MB/s, peaks much higher. l Replication via TCP. l Apache 2.0 open source library. l Pure Java, supported on Windows, Linux, Mac OSX.
What is Chronicle Queue?
l Low latency persisted key-value store. l ConcurrentMap or Set interface l Latency between processes around 200 ns. l In specialized cases, latencies < 25 ns. l Designed for reactive programming l Throughputs up to 30 million/second. l Sustain rates of 400 MB/s, peaks much higher. l Replication via TCP and UDP. l Apache 2.0 open source library. l Pure Java, supported on Windows, Linux, Mac OSX.
What is Chronicle Map/Set?
Chronicle monitoring a legacy application
Chronicle journalling multiple applications
slf4j.chronicle.base = ${java.io.tmpdir}/vanilla/${today}
slf4j.chronicle.type = vanilla
slf4j.chronicle.path = ${slf4j.chronicle.base}/root
slf4j.chronicle.level = debug
slf4j.chronicle.shortName = false
slf4j.chronicle.append = false
slf4j.chronicle.format = binary
slf4j.chronicle.binaryMode = formatted
What is Chronicle Logger?
How does Chronicle perform?
Logging code log.info(format string, int, long, double) Chronicle Queue code logger.startExcerpt(); logger.writeLong(System.currentTimeMillis()); logger.writeUTFΔ(getThreadName()); logger.writeUTFΔ(fmt); logger.writeInt(i); logger.writeLong(i * 7L); logger.writeDouble(i / 16.0); logger.finish();
How much difference does it make?
Tested with 32 threads. Chronicle Queue shows better concurrency giving a lower average latency. Chronicle Logger can give you much of the benefit of Chronicle Queue without having to change your code.
How much difference does it make?
Without Logger With Logger slf4j 1.6 µs 1.4 µs logback 55 µs 1.7 µs Log4j v1 7.6 µs 4.6 µs Log4j v2 2.7 µs 1.4 µs Chronicle Queue 32 threads
1 thread 0.026 µs 0.3 µs
