The Java HotSpot VM Under the Hoodcr.openjdk.java.net/~thartmann/talks/2016-Hotspot... · – Generated at VM startup (before program execuRon) – Maps a well-deﬁned machine code

Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.

TheJavaHotSpotVMUndertheHood

TobiasHartmannZoltánMajóHotSpotCompilerTeamOracleCorpora6on


Aboutus•  TobiasHartmann–  MScETHZurich,Switzerland–  LivesinRheinfelden,Germany

•  ZoltánMajó–  PhDETHZurich,Switzerland–  GrewupinCluj,Romania

•  Bothofus:@Oraclesince2014–  CompilerteamfortheJavaHotSpotVirtualMachine

2


SafeHarborStatementThefollowingisintendedtooutlineourgeneralproductdirecRon.ItisintendedforinformaRonpurposesonly,andmaynotbeincorporatedintoanycontract.Itisnotacommitmenttodeliveranymaterial,code,orfuncRonality,andshouldnotberelieduponinmakingpurchasingdecisions.Thedevelopment,release,andRmingofanyfeaturesorfuncRonalitydescribedforOracle’sproductsremainsatthesolediscreRonofOracle.

3


Atypicalcompu6ngplaJorm

4

Hardware

Opera6ngsystem

JavaVirtualMachine

UserApplica6ons

JavaEEJavaSE

Applica'onSo,ware

SystemSo,ware



5

Hardware

Opera6ngsystem

JavaVirtualMachine

UserApplica6ons

JavaEEJavaSE

Applica'onSo,ware

SystemSo,ware



6

Hardware

Opera6ngsystem

JavaVirtualMachine

UserApplica6ons

JavaEEJavaSE

Applica'onSo,ware

SystemSo,ware


Outline• Whyvirtualmachines?•  TheJavaHotSpotVM–  Just-in-RmecompilaRon–  OpRmisRccompileropRmizaRons–  TieredcompilaRon–  Recentprojects:SegmentedCodeCache,CompactStrings–  Future:AOT,JVMCI

•  Conclusions

7


Programminglanguageimplementa6on

C

Windows

Intelx86

8

Opera6ngsystem

Languageimplementa6on

Hardware

Programminglanguage

CompilerStandardlibrariesDebuggerMemorymanagement

Linux

Intelx86


Linux

ARM


Solaris

SPARC


Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved. 9

(Language)virtualmachine

Java

Windows

Intelx86

Opera6ngsystem

Virtualmachine

Hardware

Programminglanguage

HotSpotVM

PPC ARM SPARC

MacOSX SolarisLinux

JavaScript Scala Python


TheVM:Anapplica6ondeveloper’sview

Javasourcecodeinti=0;do{i++;}while(i<f());

Bytecodes0:iconst_01:istore_12:iinc5:iload_16:invokestaticf9:if_icmplt212:return

compileHotSpotJavaVMexecute

• Ahead-of-6me• Usingjavac

• Instruc6onsforanabstractmachine• Stack-basedmachine(noregisters)

10


TheVM:AVMengineer’sview

11


HotSpotJavaVM

Garbagecollector

manage

Interpreter

executeHeap

Stack

access

access

Compila6onsystem

compile

C1

C2

Compiledmethodproduce

Machinecode

Debuginfo

Objectmaps


MajorcomponentsofHotSpot•  Run6me–  Interpreter–  Threadmanagement–  SynchronizaRon–  Classloading

•  Heapmanagement–  Garbagecollectors

•  Compila6onsystem

12


Interpreta6onvs.compila6oninHotSpot•  Template-basedinterpreter–  GeneratedatVMstartup(beforeprogramexecuRon)–  Mapsawell-definedmachinecodesequencetoeverybytecodeinstrucRon

–  OpRmizaRon:cachetop-of-stackvalueinaregistertoreduce#ofmemoryaccesses

•  Compila6onsystem–  SpeeduprelaRvetointerpretaRon:~100X–  Twojust-in-'mecompilers(C1,C2)–  AggressiveopRmisRcopRmizaRons

13


Machinecodemov-0x8(%r14),%eaxmovzbl0x1(%r13),%ebxinc%r13mov$0xff40,%r10jmpq*(%r10,%rbx,8)

Loadlocalvariable1

DispatchnextinstrucRon


Ahead-of-6mevs.just-in-6mecompila6on•  AOT:Beforeprogramexecu6on•  JIT:Duringprogramexecu6on

•  Tradeoff:Resourceusagevs.performanceofgeneratedcode

14

Performance

Amountofcompila6onInterpreta'on Compileeverything

Badperformanceduetointerpreta6on

Badperformanceduetocompila6onoverhead

Goodperformanceduetogoodselec6onofcompiledmethodsandofappliedop6miza6ons


Balancingresourceusageandperformance•  Gecngtothe“sweetspot”•  Carefullyselec6ng

1.  Methodstocompile2.  AppliedcompileropRmizaRons

15


1.Selec6ngmethodtocompile•  Hotmethods(frequentlyexecutedmethods)•  Profilemethodexecu6on–  #ofmethodinvocaRons,#ofbackedges

•  Amethod’slife6meintheVM

16

Interpreter Compiler(C1orC2) Codecache

GatherprofilinginformaRon CompilebytecodetonaRvecode Storemachinecode

#methodinvocaRons>THRESHOLD1#ofbackedges>THRESHOLD2

DeopRmizaRon

Compiler’sopRmisRcassumpRonsprovenwrong


Virtualcallinlining

17

classA{voidbar(){S1;}}

classBextendsA{voidbar(){S2;}}

voidfoo(){Aa=create();//returnAorBa.bar();}

Classhierarchy Methodtobecompiled

loaded

notloaded

Compiler:Inlinecall?Yes.


Virtualcallinlining

•  Benefitsofinlining–  Virtualcallavoided–  Codelocality

•  Op6mis6cassump6on:onlyAisloaded–  Notedependenceonclasshierarchy–  DeopRmizeifhierarchychanges

18



voidfoo(){Aa=create();//returnAorBS1;}


loaded

notloaded

Compiler:Inlinecall?Yes.


Virtualcallinlining

19





loaded

notloaded


Virtualcallinlining

20





loaded

loaded


Virtualcallinlining

21



voidfoo(){Aa=create();//returnAorBa.bar();}


loaded

loaded

Compiler:Inlinecall?No.


False

Controlflowgraph Generatedcode

Hotpathcompila6on

S1;S2;S3;if(x>3)

S4; S5;S6;S7;

S8;S9;

10’000 0

guard(x>3)S1;S2;S3;S4;S5;

Uncommontrap

22

True


Deop6miza6on•  Compiler’sop6mis6cassump6onprovenwrong–  AssumpRonsaboutclasshierarchy–  ProfileinformaRondoesnotmatchmethodbehavior

•  Switchexecu6onfromcompiledcodetointerpreta6on–  ReconstructstateofinterpreteratrunRme–  CompleximplementaRon

•  Compiledcode–  Possiblythrownaway–  Possiblyreprofiledandrecompiled

23


Performanceeffectofdeop6miza6on•  Followthevaria6onofasinglemethod’sperformance

24

Performance

TimeVMStartup VMTeardown

Interpreted Compiled Interpreted Compiled

Compila>on Deop>miza>on Compila>on


2.Selec6ngcompilerop6miza6ons•  C1compiler–  LimitedsetofopRmizaRons–  FastcompilaRon–  Smallfootprint

•  C2compiler–  AggressiveopRmisRcopRmizaRons–  Highresourcedemands–  High-performancecode

•  Graal–  Experimentalcompiler–  NotpartofHotSpot

25

ClientVM

ServerVM

Tieredcompila6on(enabledsinceJDK8)


Balancingresourceusageandperformance1.   Selec6ngmethodstocompile–  “Hot”methods–  ControlledbyinvocaRonandbackedgethreshold

2.   Choosingcompilerop6miza6ons–  C1:moderatelyop'mizingandfastcompiler–  C2:highlyop'mizingandslowcompiler–  LimitaRon(beforeJDK8):SinglecompilerintheVM(clientorserver)–  StarRngwithJDK8:BothcompilersenabledatthesameRme(ReredcompilaRon)

26



•  Conclusions

27


Tieredcompila6on•  Combinethebenefitsof–  Interpreter:Faststartup–  C1:Fastwarmup–  C2:Highpeakperformance–  SRllwithinthesweetspotofresourceusage/performancetradeoff

28


Benefitsof6eredcompila6on(ar6st’sconcept)

29

Performance


Interpreted C1-compiled

Methodwarm-upRme

ClientVM(C1only)



30

Performance



Methodwarm-upRme

ServerVM(C2only)



31

Performance



Methodwarm-upRme

Tieredcompila6on

C2-compiled


Tieredcompila6on•  Combinedbenefitsofinterpreter,C1,andC2•  Addi6onalbenefits–  MoreaccurateprofilinginformaRon

32


Moreaccurateprofiling

Rme

Interpreter C1(profiled) C2(non-profiled)

Interpreter

Profilingwithout6eredcompila6on

Profilingwith6eredcompila6on

C2(non-profiled)

300samples

100samples 1000samples

100samples 200samples

w/6eredcompila6on:1’100samplesgatheredw/o6eredcompila6on:300samplesgathered

33


Tieredcompila6on•  Combinedbenefitsofinterpreter,C1,andC2•  Addi6onalbenefits–  MoreaccurateprofilinginformaRon

•  Drawbacks–  CompleximplementaRon–  CarefultuningofcompilaRonthresholdsneeded–  Morepressureoncodecache–Tobiaswilltellyoumoreaboutthat

34


Amethod’slife6me(w/6eredcompila6on)

Interpreter C1 C2

Codecache

CollectprofilinginformaRon GeneratecodequicklyConRnuecollecRngprofilinginformaRon

Generatehigh-qualitycodeUseprofilinginformaRon

DeopRmizaRon

35


Performanceofasinglemethod(w/6eredcompila6on)

36

Performance


Interpreted C1compiled Interpreted C2compiled

Compila>on

Deop>miza>on

Compila>on

Interpreted C2compiled

Deop>miza>on

Compila>on


Compila6onlevels(detailedview)

Interpreter

C1:noprofiling

C1:limitedprofiling

C1:fullprofiling

C2

0

1

2

3

4

CompilaRo

nlevel

TypicalcompilaRonsequence

Associatedthresholds:Tier3InvokeNoRfyFreqLogTier3BackedgeNoRfyFreqLogTier3InvocaRonThresholdTier3MinInvocaRonThresholdTier3BackEdgeThresholdTier3CompileThreshold

Associatedthresholds:Tier4InvocaRonThresholdTier4MinInvocaRonThresholdTier4CompileThresholdTier4BackEdgeThreshold

37



•  Conclusions

38


Part1:SegmentedCodeCacheImprovingthelayoutofJITgeneratedcode

39


ProgramAgenda

Background

Challenges

Design

Evalua6on

Conclusion

1

2

3

4

40

5


ProgramAgenda

Background

Challenges

Design

Evalua6on

Conclusion

1

2

3

4

41

5


Whatisacodecache?•  StorescodegeneratedbyJITcompilers•  Con6nuouschunkofmemory–  Fixedsize-XX:ReservedCodeCacheSize–  BumppointerallocaRonwithfreelist

• Memorymanagedbysweeper–  Coldmethodsareevicted–  Hotmethodsremain

• WhyshouldIcare?–  EssenRalforperformance

42

RunRme

Debugging

Compiler

GC

Serviceability

Sweeper


Codecacheusage:JDK6and7

freespaceVMinternalscompiledcode

43


Codecacheusage:JDK8(TieredCompila6on)

44

freespace

VMinternals

C1compiled(profiled)

C2compiled(non-profiled)


Codecacheusage:JDK9

45

freespace

VMinternals




ProgramAgenda

Background

Challenges

Design

Evalua6on

Conclusion

1

2

3

4

46

5


Challenges•  Tieredcompila6onincreasesamountofcodeby2-4X•  Allcodeisstoredinasinglecodecache–  DifferenttypeswithdifferentcharacterisRcs–  Differentusagefrequencies(hotness)–  AccesstospecificcoderequiresfulliteraRon

•  Highfragmenta6onandbadlocality

47


Proper6esofcompiledcode

Op6miza6onlevel Size

Costofcompila6on Life6me

48


Typesofcompiledcode

Non-methodcode opRmized small cheap immortal

Profiledcode(C1) instrumented medium cheap limited

Non-profiledcode(C2) highlyopRmized large expensive long

49


Codecacheusage

50

freespace

VMinternals




Codecacheusage:Reality

profiledcodenon-profiledcode

freespace


Codecacheusage:Reality


freespace

hotness


ProgramAgenda

Background

Challenges

Design

Evalua6on

Conclusion

1

2

3

4

53

5


• WithoutSegmentedCodeCache • WithSegmentedCodeCache

54

Design

CodeCache

non-profiledmethods

profiledmethods

non-methods


non-profiledmethodsprofiledmethods

55

SegmentedCodeCache:Reality


freespace


SegmentedCodeCache:Reality

non-profiledmethodsprofiledmethods hotness


ProgramAgenda

Background

Challenges

Design

Evalua6on

Conclusion

1

2

3

4

57

5


Evalua6on:Codelocality•  Instruc6onCache(ICache)–  14%lessICachemisses

•  Instruc6onTransla6onLookasideBuffer(ITLB1)–  44%lessITLBmisses–  9%speedupwithmicrobenchmark

58

1cachesvirtualtophysicaladdressmappingstoavoidslowpagewalks


Evalua6on:Sweeper

59

0

5

10

15

20

25

30

35

40

#fullsweeps CleanuppauseRme SweepRme

Redu

c6on

in%


Evalua6on:Run6me

60

0

2

4

6

8

10

12

14

SPECjbb2005 SPECjbb2013 JMH-Javac Octane(Typescript) Octane(Gbemu)

Improvem

entin%


ProgramAgenda

Background

Challenges

Design

Evalua6on

Conclusion

1

2

3

4

61

5


Conclusion•  Codelayoutmaters–  Significantimpactonperformance–  CodelocalityreducesiTLBmisses

•  SegmentedCodeCachehelps–  Lesssweeperoverhead–  ReducedfragmentaRon

•  Baseforfutureextensions–  Newcodetypes–  SeparaRonofcodeandmetadata

62


Part2:CompactStringsImproveVMinternalhandlingofStrings

63


ProgramAgenda

JavaStrings

ProjectGoals

Design

Evalua6on

Conclusion

1

2

3

4

64

5


ProgramAgenda

JavaStrings

ProjectGoals

Design

Evalua6on

Conclusion

1

2

3

4

65

5


publicclassHelloWorld{publicstaticvoidmain(String[]args){

StringmyString="HELLO";System.out.println(myString);

}}

JavaStrings

66




}}

JavaStrings

publicfinalclassString{privatefinalcharvalue[];...}




}}

JavaStrings

publicfinalclassString{privatefinalcharvalue[];...}

charvalue[]=

H0x0048 0x0045 0x004C 0x004C 0x004F

2bytes

E L L OUTF-16encoded


“PerfecRonisachieved,notwhenthereisnothingmoretoadd,butwhenthereisnothingmoretotakeaway.”–  AntoinedeSaintExupéry

69


Thereisalottotakeawayhere..•  UTF-16encodedStringsalwaysoccupytwobytesperchar• WastedmemoryifonlyLa6n-1(one-byte)charactersused:

70

charvalue[]=

H0x0048 0x0045 0x004C 0x004C 0x004F

2bytes

E L L O


Thereisalottotakeawayhere..•  UTF-16encodedStringsalwaysoccupytwobytesperchar• WastedmemoryifonlyLa6n-1(one-byte)charactersused:

•  Butisthisaprobleminreallife?

71

charvalue[]=

H0x0048 0x0045 0x004C 0x004C 0x004F

2bytes

E L L O


Reallifeanalysis:char[]footprint•  950heapdumpsfromavarietyofapplica6ons–  char[]footprintmakesup10%-45%oflivedata–  Majorityofcharactersaresinglebyte–  75%ofStringsaresmallerthan35characters–  75%ofCharactersareinStringsoflength<250

•  Predictedfootprintreduc6onof5%-10%

72


ProgramAgenda

JavaStrings

ProjectGoals

Design

Evalua6on

Conclusion

1

2

3

4

73

5


ProjectGoals• Memoryfootprintreduc6onbyimprovingspaceefficiencyofStrings• MeetorbeatthroughputperformanceofbaselineJDK9

•  Fullcompa6bilitywithrelatedJavaandna6veinterfaces

•  FullplaJormsupport–  x86/x64,SPARC,ARM–  Linux,Solaris,Windows,MacOSX

74


ProgramAgenda

JavaStrings

ProjectGoals

Design

Evalua6on

Conclusion

1

2

3

4

75

5


Design•  Stringclassnowusesabyte[]insteadofachar[]

•  Addi6onal'coder'fieldindicateswhichencodingisused

76

publicfinalclassString{privatefinalbytevalue[];privatefinalbytecoder;...}

H E L L Obytevalue[]= 0x00 0x48 0x00 0x45 0x00 0x4C 0x00 0x4C 0x00 0x4F

bytevalue[]= 0x48 0x45 0x4C 0x4C 0x4F

UTF-16encoded

La6n-1encoded

H E L L O


Design•  Ifallcharactershaveazeroupperbyte

→ StringiscompressedtoLa6n-1bystrippingoffhighorderbytes

•  IfAcharacterhasanon-zeroupperbyte→ StringcannotbecompressedandisstoredUTF-16encoded

77

bytevalue[]= 0x00 0x48 0x00 0x45 0x00 0x4C 0x00 0x4C 0x00 0x4F

bytevalue[]=0x48 0x45 0x4C0x4C 0x4F

UTF-16encoded

La6n-1encoded

Infla6onCompression


Design•  Compression/infla6onneedstofast•  RequiresHotSpotsupportinaddi6ontoJavaclasslibrarychanges–  JITcompilers:IntrinsicsandStringconcatenaRonopRmizaRons–  RunRme:Stringobjectconstructors,JNI,JVMTI–  GC:StringdeduplicaRon

•  KillswitchtoenforceUTF-16encoding(-XX:-CompactStrings)–  ForapplicaRonsthatextensivelyuseUTF-16characters

78


ProgramAgenda

JavaStrings

ProjectGoals

Design

Evalua6on

Conclusion

1

2

3

4

79

5


Evalua6on•  Newandexis6ngunitests• MicrobenchmarksattheStringAPIlevel

•  Largebenchmarkstomeasureoverallperformance

80


publicclassLogLineBench{intsize;Stringmethod=generateString(size);publicStringwork()throwsExceptions{return"["+System.nanoTime()+"]"+Thread.currentThread().getName()+"Callinganapplicationmethod\""+method+"\"withoutfearandprejudice.";}

Microbenchmark:LogLineBench


LogLineBenchresultsPerformancens/op Allocatedb/op

1 10 100 1 10 100

Baseline 149 153 231 888 904 1680

CSdisabled 152 150 230 888 904 1680

CSenabled 142 139 169 504 512 904

82

•  Killswitchworks(noregression)•  27%performanceimprovementand46%footprintreducRon


Largeworkloads•  SPECjbb2005–  21%footprintreducRon–  27%lessGCs–  5%throughputimprovement

•  SPECjbb2015–  7%footprintreducRon–  11%criRcal-jOpsimprovement

• Weblogic(startup)–  10%footprintreducRon–  5%startupRmeimprovement

83


ProgramAgenda

JavaStrings

ProjectGoals

Design

Evalua6on

Conclusion

1

2

3

4

84

5


Conclusion

CompactStringshelpsourapplica6onsalot.

Ongoingeffort:IndifyStringConcat,FusedStrings

TryoutJDK9earlyaccess:jdk9.java.net/download/

..andtellushowitperformswithyourapplica6ons!

85


Future•  AOT:Ahead-of-6mecompila6on–  CompiletonaRvecode(nottoJavabytecodes)–  MoreinformaRon:hvps://www.youtube.com/watch?v=Xybzyv8qbOc(45-minutetalkfromJVMLS’15)

•  JVMCI:JavaVirtualMachineCompilerInterface–  CurrentcompilerswriveninC/C++–  JVMCI:InterfacetoallowJavacodetointerceptJVMacRvityandplug-innaRvecode–  Experimentalfeature,GraalandSubstrateVMuseit

86


Conclusions•  Java–avibrantplaJorm–  Newfeatures:SegmentedCodeCache,CompactStrings,JVMCI–  …andmanyotherfeaturestobereleasedwithJDK9–  Staytuned!

•  ThefutureoftheJavaplaJorm

"OurSaaSproductsarebuiltontopofJavaandtheOracleDB—that’stheplaMorm.”LarryEllison,OracleCTO

87


Thankyouforyouraten6on!

88


Backupslides

90

Documents

The Java HotSpot VM Under the Hoodcr.openjdk.java.net/~thartmann/talks/2016-Hotspot... · – Generated at VM startup (before program execuRon) – Maps a well-deﬁned machine code