JVM: need for speed - tooling overview for jdk.io 2016

01

About me02

Andrey AdamovichJava/Groovy developer

(Sometimes) application performance analyst

Co­author of Groovy 2 Cookbook

Co­organizer of @latcraft and @devternity

••••

03

Contact detailsE­mail: andrey@aestasit.com

Linkedin: http://www.linkedin.com/in/andreyadamovich

Lanyrd: http://lanyrd.com/profile/andrey­adamovich

GitHub: https://github.com/aadamovich

SO: http://stackoverflow.com/users/162792/andrey­adamovich

Twitter: @codingandrey, @aestasit

••••••

04

What's in thispresentation?

05

The tools06

The game07

RulesPicture is shown

You guess what the next topic is

First one to guess gets a sticker!

•••

08

Let's try!09

10

Java's(Hash)Map11

12

BigData13

Let the gamesbegin!

14

15

GC Pauses16

What are thegood

parameters?17

So manyoptions....

18

So many options....> java ‐XX:+PrintFlagsFinal ‐version | wc ‐l

java version "1.8.0_45"

Java(TM) SE Runtime Environment (build 1.8.0_45‐b15)

Java HotSpot(TM) 64‐Bit Server VM (build 25.45‐b02, mixed mode)

714

01.

02.

03.

04.

05.

19

So many options....> java ‐XX:+PrintFlagsFinal ‐version | wc ‐l

java version "1.7.0_80"

Java(TM) SE Runtime Environment (build 1.7.0_80‐b15)

Java HotSpot(TM) 64‐Bit Server VM (build 24.80‐b11, mixed mode)

668

01.

02.

03.

04.

05.

20

So many options....> java ‐XX:+PrintFlagsFinal ‐version | wc ‐l

java version "1.6.0_45"

Java(TM) SE Runtime Environment (build 1.6.0_45‐b06)

Java HotSpot(TM) 64‐Bit Server VM (build 20.45‐b01, mixed mode)

670

01.

02.

03.

04.

05.

21

So many options....> java ‐XX:+PrintFlagsFinal ‐version | grep GC | wc ‐l

65

01.

02.

22

Use casesWeb application

Streaming application

Background job

•••

23

Memory tuning?‐Xms<heap size>[g|m|k] 

‐Xmx<heap size>[g|m|k]

‐XX:PermSize=<perm gen size>[g|m|k] 

‐XX:MaxPermSize=<perm gen size>[g|m|k]

‐Xmn<young size>[g|m|k]

‐XX:SurvivorRatio=<ratio>

01.

02.

03.

04.

05.

06.

24

Leverage multi­core‐XX:+UseConcMarkSweepGC 

‐XX:+UseCMSInitiatingOccupancyOnly 

‐XX:+ScavengeBeforeFullGC 

‐XX:+CMSScavengeBeforeRemark

‐XX:+CMSConcurrentMTEnabled

‐XX:CMSInitiatingOccupancyFraction=70

01.

02.

03.

04.

05.

06.

25

GC logging‐XX:+PrintGCDetails 

‐XX:+PrintGCDateStamps 

‐Xloggc:<file‐path> 

‐XX:+PrintGCApplicationConcurrentTime 

‐XX:+PrintSafepointStatistics 

‐XX:+PrintHeapAtGC

01.

02.

03.

04.

05.

06.

26

Log analysis

27

Open source: GCViewer

28

Comercial: Censum

29

Next riddle30

31

Memorydumps

32

How to take?On out of memory:

‐XX:+HeapDumpOnOutOfMemoryError 

‐XX:HeapDumpPath=<path to dump>`date`.hprof

Or manually:

jmap ‐dump:file=<path to dump> <java process id>

01.

02.

01.

33

jhat34

Eclipse MAT

35

OQL

36

Useful characteristicsObject Retained Size

Object Shallow Size

Stale Objects (not connected to GC roots)

•••

37

Flight Recording

38

ComercialYourKit

jProfiler••

39

Next riddle40

41

Threaddumps

42

SituationsDead locks

100% CPU load

Hanging I/O

•••

43

jstackjstack <java process pid>01.

44

JMXUse MBean with name  java.lang:type=Threading

Call  getAllThreadIds

Loop through all retrieved IDs and call  getThreadInfo

•••

45

The cheapest profilerfor ((a=1; a <= 60; a++))

do 

  jstack 7789 > thread‐dump.$a

  sleep 1

  echo $a

done

01.

02.

03.

04.

05.

06.

46

Thread dumpanalysis

47

Thread dump structure

48

Thread states IA thread can be in one of the following states:

NEW  A thread that has not yet started is in this state.

RUNNABLE  A thread executing in the Java virtual machine is in this

state.

BLOCKED  A thread that is blocked waiting for a monitor lock is in this

state.

••

•

49

Thread states IIWAITING  A thread that is waiting indefinitely for another thread to

perform a particular action is in this state.

TIMED_WAITING  A thread that is waiting for another thread to perform

an action for up to a specified waiting time is in this state.

TERMINATED  A thread that has exited is in this state.

•

•

•

50

TDA

51

SamuraiTDA

52

IBM TMDA

53

Use customscripts

54

Next riddle55

56

Debugging57

We need it when we have...Non­reproducable issues

Hard­to­understand code••

58

Enable debugging‐agentlib:jdwp=transport=dt_socket,server=y,address=133701.

59

Use your IDE (Eclipse)

60

Use your IDE (Intellij IDEA)

61

jdb62

jdb

63

YouDebug64

YouDebugYouDebug is a non­interactive debugger scripted by Groovy to assist

remote troubleshooting and data collection to analyze failures.“65

YouDebughttp://youdebug.kohsuke.org/

https://github.com/kohsuke/youdebug••

66

YouDebug (buggy class)public class SubStringTest {

  public static void main(String[] args) {

    String s = someLengthComputationOfString();

    System.out.println(s.substring(5));

  }

  private static String someLengthComputationOfString() {

    ...;

  }

}

01.

02.

03.

04.

05.

06.

07.

08.

09. 67

YouDebug (SubStringMonitor.ydb)breakpoint("com.acme.SubStringTest",3) {

  println "s="+s;

}

01.

02.

03.

68

YouDebug (run debugger)> java ‐jar youdebug.jar ‐socket 1337 SubStringMonitor.ydb

s=test

01.

02.

69

Next riddle70

71

Decompiling72

Java Decompiler

73

Java Decompiler

74

IssuesLack of support for latest JVM features

Lack of support for other JVM langauges

Obfuscation is always an issue

•••

75

Next riddle76

77

Profiling78

Types of profilingMemory allocation profiling

CPU profiling

Lock profiling

•••

79

VisualVM

80

Honest Profiler

81

ComercialYourKit

jProfiler••

82

Next riddle83

84

Benchmarks85

JMHMH is a Java harness for building, running, and analysing

nano/micro/milli/macro benchmarks written in Java and other

languages targetting the JVM.“86

JMH: examplepackage org.openjdk.jmh.samples;

import org.openjdk.jmh.annotations.Benchmark;

import org.openjdk.jmh.runner.Runner;

import org.openjdk.jmh.runner.RunnerException;

import org.openjdk.jmh.runner.options.Options;

import org.openjdk.jmh.runner.options.OptionsBuilder;

01.

02.03.

04.

05.

06.

07.

87

JHM: examplepublic class JMHSample_01_HelloWorld {

  @Benchmark

  public void wellHelloThere() {

    // this method was intentionally left blank.

  }

01.

02.

03.

04.

05.

88

JMH: example  public static void main(String[] args) throws RunnerException {

    Options opt = new OptionsBuilder()

            .include(JMHSample_01_HelloWorld.class.getSimpleName())

            .forks(1)

            .build();

    new Runner(opt).run();

  }

}

01.

02.

03.

04.

05.

06.

07.

08.

89

JMH: example> mvn clean install

> java ‐jar target/benchmarks.jar JMHSample_01_HelloWorld

01.

02.

90

JMH: annotations@Benchmark @BenchmarkMode @CompilerControl 

@Fork @Group @GroupThreads

@Level @Measurement @Mode

@OperationsPerInvocation @OutputTimeUnit

@Param @Scope @Setup @State

@TearDown @Threads @Timeout @Warmup

01.

02.

03.

04.

05.

06.

91

JMH: output

92

Load testing toolsjMeter

ab

gatling

•••

93

Next riddle94

95

Leakdetection

96

Open sourceLeak Prevention Library

Leak Canary••

97

ComercialPlumbr

AppDynamics

NewRelic

•••

98

Next riddle99

100

Monitoring101

JMX is yourfriend!

102

VisualVM (again)

103

Mission Control

104

Metricscollection

105

JMXTransThis is effectively the missing connector between speaking to a JVM

via JMX on one end and whatever logging / monitoring / graphing

package that you can dream up on the other end.“106

jHiccupjHiccup is a non­intrusive instrumentation tool that logs and records

platform "hiccups" ­ including the JVM stalls that often happen when

Java applications are executed and/or any OS or hardware platform

noise that may cause the running application to not be continuously

runnable.

“107

jHiccup> java ‐javaagent:jHiccup.jar MyProgram01.

108

jHiccup

109

Drop Wizard MetricsMetrics provides a powerful toolkit of ways to measure the behavior of

critical components in your production environment.“110

Netflix Hysterix

111

Stagemonitor

112

113

Single JVM?114

Really?115

116

Dead JVM117

118

Many JVMs119

Open­source toolsGraphite

Graphana

Logstash

Kibana

Graylog2

Fluentd

••••••

120

Comercial toolsNewRelic

Plumbr

AppDynamics

Dynatrace

Takipi

Hyperic

••••••

121

Final riddle122

123

Links124

Links: GChttp://spin.atomicobject.com/2014/09/03/visualizing­garbage­

collection­algorithms/

http://techblog.netflix.com/2013/05/garbage­collection­

visualization.html

http://blog.sokolenko.me/2014/11/javavm­options­production.html

https://blog.codecentric.de/en/2013/10/useful­jvm­flags­part­7­cms­

collector/

•

•

••

125

Links: GChttp://blog.ragozin.info/2011/09/hotspot­jvm­garbage­collection­

options.html

http://blog.tier1app.com/2014/11/19/understand­garbage­collection­

log/

http://www.tagtraum.com/gcviewer­vmflags.html

•

•

•

126

Links: GChttp://fasterj.com/tools/gcloganalysers.shtml

https://github.com/chewiebug/GCViewer

http://www.jclarity.com/censum/

•••

127

Links: Thread dumpshttp://www.javacodegeeks.com/2012/03/jvm­how­to­analyze­thread­

dump.html

http://www.slideshare.net/buzdin/thread­dump­analysis

http://docs.oracle.com/javase/7/docs/api/java/lang/Thread.State.html

•

••

128

Links: Toolshttps://github.com/akullpp/awesome­java

https://github.com/aragozin/jvm­tools

http://openjdk.java.net/projects/code­tools/jmh/

•••

129

Links: Toolshttp://youdebug.kohsuke.org/

https://java.net/projects/tda

http://samuraism.jp/samurai/en/index.html

•••

130

Links: Toolshttps://github.com/jmxtrans/jmxtrans

http://www.stagemonitor.org/

http://jamonapi.sourceforge.net/

https://github.com/Netflix/Hystrix

••••

131

Links: Toolshttp://www.eclipse.org/tptp/home/documents/tutorials/profilingtool/profilingexample_32.html

https://github.com/mjiderhamn/classloader­leak­prevention••

132

Readingmaterial

133

Java Performance

134

Java Performance: The Defenetive Guide

135

Java Performance And Scalability

136

This is it!137

Questions?138

Thank you!139

Devternity1­2 of December, 2016, Riga

Software Excellence Conference

4 tracks, 30+ speakers, 6 workshops

http://devternity.com

••••

140