It’s Dangerous to GC alone. Take this!

The OMR GC talk

Who are those handsome guys?

Robert Youngrwyoung@ca.ibm.com

@rwy0717

Craig Lehmanncraigl@ca.ibm.com

@CraigLehmann

2

IBM Runtime Technologies

3

4

VisualAge Smalltalk

J9 JVM

1988

?

1993

2016

5

VisualAge Smalltalk

J9 JVM

1988

OMR

1993

2016

An open source toolkit for language runtime

technologies.

OMR

Execution Environment

7

Language-Agnostic Components

Platform Abstraction Layer

Garbage Collector

Diagnostic Services

Source Code Bytecode/ASTCompiler

Just-In-Time Compiler

InterpreterSource BytecodeCompiler

Interpreter

Execution Environment

8

Language-Agnostic Components

Platform Abstraction Layer

Garbage Collector

Diagnostic Services

Source Code Bytecode/ASTCompiler

Just-In-Time Compiler

InterpreterSource BytecodeCompiler

Interpreter

9

A Runtime Toolkit for Many Languages

JVM

OMR

Java App

Python

OMR

Python App

MRI

OMR

Ruby App

In Ruby

OMR

13

Getting started

14

How do we start?

http://thefeedingdoctor.com/wp-content/uploads/2013/10/Kid-Confused.jpg

15

How do we start?

16

Read the source

17

Read the source

18

19

Craig! Robert!

Tue Oct 5th - 1993

• Fantastic worldwide community

• Ruby has changed a lot over 20 years

• MRI is a complex virtual machine with a lot of history behind

it’s design decisions.

20

What we noticed

Objects

22

RVALUE• Direct object pointers• Conservative GC

Conservative

collection

23

Dynamic memory

string arrayhash data …

Managed Memory

• Fixed sized objects• What about variable sized objects

Dynamic memory

string arrayhash data …

Managed Memory

malloc malloc malloc malloc

Malloc Memory

24

Dynamic memory

string arrayhash data …

malloc malloc malloc malloc

25

• No heap fragmentation• Easy conservative marking

Dynamic memory

string arrayhash data …

malloc malloc malloc malloc

26

• Heavyweight allocation

• Malloc fragmentation

• Poor cache locality

• Expensive sweeping

Ruby has an incremental GC

Mark Sweep

Ruby

27

• GC work is interleaved with execution• Minimizes mutator pause times• Single threaded• Serialized with the mutator

What we did

Goals

Make allocations faster

Improve GC Time

Improve mutator performance

100% compatibility

29

100% compatibility

No C-extensions API changes

Pass all extended tests

Run Rails - unmodified

30

Let’s make it work

31

32

Starting out with Mark Sweep

static intgc_start(rb_objspace_t *objspace …){…

OMR_GC_SystemCollect();...

}

static intnewobj_of(rb_objspace_t *objspace …){

...obj = rb_omr_get_freeobj(th, sizeof(RVALUE));...

}

33

Starting out with Mark Sweep

Need to support conservative collection

Conservative collection

Object Map (bit map)

• Object map allows us to support conservative collection

• Bit Map used to keep track of objects• Results in ~ 1.6% memory overhead

Let’s make it parallel

34

Multithreaded GC (Stop the world)

35

mark sweep

• Introducing Parallel Global GC• Aggressively parallelized• New APIs for parallelism• Support for thread pooling and task

synchronization

Marking in parallel

36

mark sweep

Marking:• Scan roots in parallel• Break VM roots into subsets• Complete marking in parallel

Sweeping in parallel

37

mark sweep

Sweeping:• Free malloc space in parallel• Clean up objects in parallel• Move work out of finalization

Hey… It works!

38

New tools!

39

Garbage Collection Memory Visualizer for Ruby MRI with zero changes to the tool

40

Health Center for Ruby MRI: Live GC visualization –No code changes required

41

Health Center for Ruby MRI: Method profiling

42

How can we make CRuby faster?

43

Allocations!

44

Making allocation fast

Thread-local heaps:

• Cache regions of the heap per-thread

• Allocate without locking

• Better locality

45

Thread local heap

Object

TLH

Heap

Next Alloc

47

TLH

Thread local heap

TLH

Object

TLH

Heap

Next Alloc

48

Thread local heap

TLH

Object

TLH

Heap

Next Alloc

49

How can we make it faster?

50

Lets look at off heap memory!

51

How do we replace malloc/free?

• malloc/free callouts are expensive

• Rely on system for memory management concerns

• Still susceptible to fragmentation and concurrency

• Black box implementation

• Replace malloc and free with a new allocator

52

New built-in type!

55

New built-in type OMRBuffers

• Create a new, variable sized object type

• Allocate all buffers on the heap as objects

typedef struct OMRBuffer {VALUE flags;long size;

} OMRBuffer;

56

buffer

58

OMRBuffers on the heap!

string arrayhash data …

Managed Memory

buffer buffer buffer buffer

Malloc Memory

59

Shorter sweeping with OMRBuffers

mark sweep

GC Before

GC After

mark

sweep

Getting user defined types on heap

• RDatas are used to create C-extension types.

• Typed RDatas have a new flag:

RDATA_HEAP_ALLOCATED

• Automatically heap allocates the data buffer at allocation

• No free method results in no object finalization

• Allows for heap allocation in extensions

• Extremely easy to use

60

OMRBuffer wins

Improvements:

• Eliminated obj_free()

• Fast allocations

• Better cache locality

Challenges:

• Heap fragmentation

• Conservative collection

• Concurrency

61

Multithreaded allocations

• MRI does some work in backgrounded threads

• Use OMRBuffers in background threads

• Introduce finer grained locking than the GVL

• Allows for:

• multithreaded allocations

• GCing from a background thread

62

Is Ruby Fast Yet?

63

8% better throughput

122.57112.84

0

20

40

60

80

100

120

140

2.2-omr 2.2.3

Experiments

64

Generational GC in MRI

• Experimenting with non-copying generational GC

• Marking is already fast

• Heap fragmentations issues

• High memory overhead

65

Segregated heap in MRI

• Introduced a segregated heap into ruby

• Heap divided into regions of fixed sized objects

• Bounds maximum heap fragmentation

66

67

Concurrent GC in MRI

• Adding Concurrent GC to Ruby

• Ruby threads incrementally mark

• Background thread scans

• Parallelized sweep (STW)

sweepmark

What’s next

70

OMR GC in Ruby – what’s next

• Ongoing experimentation

• Balanced

• Semispace copying generational collection

• compaction

• Real time garbage collection

71

What’s next for OMR?

• Open source OMR

• Make our Ruby experiments available

• We want to hear from the experts (you)

• Lets make OMR and Ruby the best they can be

72

Ruby+OMRTech Preview

Download @

goo.gl/P3yXuy

• Releasing a

preview so you

can start test-

driving

• Send us

feedback!

73

74

75

We want to help

Thank You!

76

John DuimovichCTO IBM Runtimesduimovic@ca.ibm.com@jduimovich

Ask us Anything!

Mark StoodleyOMR Project Leadmstoodle@ca.ibm.com@mstoodle

Robert YoungOMR Developerrwyoung@ca.ibm.com@rwy0717

77

Charlie GracieOMR GC Architectcrgracie@ca.ibm.com@crgracie

Craig LehmannOMR Developercraigl@ca.ibm.com@craiglehmann

The end!

Who were those handsome guys again?

Robert Young

rwyoung@ca.ibm.com

@rwy0717

Craig Lehmann

craigl@ca.ibm.com

@CraigLehmann

82

Trademarks, Copyrights, Disclaimers

88

IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion. Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision. The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion.

IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corp., registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of other IBM trademarks is available on the web at "Copyright and trademark information" at http://www.ibm.com/legal/copytrade.shtml

Other company, product, or service names may be trademarks or service marks of others.

THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY. WHILE EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. IN ADDITION, THIS INFORMATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT NOTICE. IBM SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION. NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, NOR SHALL HAVE THE EFFECT OF, CREATING ANY WARRANTIES OR REPRESENTATIONS FROM IBM (OR ITS SUPPLIERS OR LICENSORS), OR ALTERING THE TERMS AND CONDITIONS OF ANY AGREEMENT OR LICENSE GOVERNING THE USE OF IBM PRODUCTS OR SOFTWARE.

© Copyright International Business Machines Corporation 2015. All rights reserved.

Additional Important Disclaimers

• THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY.

• WHILST EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED.

• ALL PERFORMANCE DATA INCLUDED IN THIS PRESENTATION HAVE BEEN GATHERED IN A CONTROLLED ENVIRONMENT. YOUR OWN TEST RESULTS MAY VARY BASED ON HARDWARE, SOFTWARE OR INFRASTRUCTURE DIFFERENCES.

• ALL DATA INCLUDED IN THIS PRESENTATION ARE MEANT TO BE USED ONLY AS A GUIDE.

• IN ADDITION, THE INFORMATION CONTAINED IN THIS PRESENTATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM, WITHOUT NOTICE.

• IBM AND ITS AFFILIATED COMPANIES SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION.

• NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, OR SHALL HAVE THE EFFECT OF:

• - CREATING ANY WARRANT OR REPRESENTATION FROM IBM, ITS AFFILIATED COMPANIES OR ITS OR THEIR SUPPLIERS AND/OR LICENSORS

89