Maximum length: 1 line Syncopation - IBMresearcher.watson.ibm.com/.../us-bacon/Bacon05SyncopationSlides.pdf · Syncopation: Generational Real-time Garbage Collection in the Metronome

LCTES’05 Presentation 17 June 2005

• Presentation subtitle:20pt Arial Regular,teal R045 | G182 | B179

Recommendedmaximum length: 2 lines

• Confidentiality/date line: 13pt Arial Regular, whiteMaximum length: 1 line

• Information separated by vertical strokes,with two spaces on either side

• Disclaimer information may also be appear in this area. Placeflush left, aligned at bottom, 8-10pt Arial Regular, white

• IBM logo must notbe moved, addedto, or altered inany way.

Indications in green = Live content

Indications in white = Edit in master

Indications in blue = Locked elements

Indications in black = Optional

elements

• Presentation title:28pt Arial Regular, black

Recommended maximumlength: 2 lines

• Group name:17pt Arial Regular, white

Maximum length: 1 line

• Copyright: 10pt ArialRegular, white

Template release: Oct 02For the latest, go tohttp://w3.ibm.com/ibm/presentations

For clientpresentations,client’s logo maygo in this area

Syncopation:Generational Real-timeGarbage Collection in

the MetronomeDavid F. Bacon Perry Cheng David Grove

IBM T.J. Watson Research CenterMartin T. Vechev

Cambridge University

Syncopation: Generational Real-time Garbage Collection in the Metronome• IBM logo must not

be moved, addedto, or altered inany way.

• Background shouldnot be modified,

• Title/subtitle/confidentiality line: 10pt Arial Regular, whiteMaximum length: 1 line

Information separated by vertical strokes,with two spaces on either side

• Slide heading:28pt Arial Regular,blue R120 | G137 | B251

Maximum length: 2 lines

• Slide body:18pt Arial Regular, black

Square bullet color:teal R045 | G182 | B179

Recommended maximumtext length: 5 principalpoints



• Copyright: 10ptArialRegular, white

Optional slide number:10pt Arial Bold, white

Template release: Oct 02For the latest, go to http://w3.ibm.com/ibm/presentations





elements

David F. Bacon LCTES’05 17 June 2005

The Metronome Project Make standard Java suitable for real-time programming

Real-time Garbage Collection [POPL’03]• Solves biggest obstacle for real-time Java• Prototype in Jikes RVM• Uniprocessor algorithm• Worst-case pause 12ms; utilization at least 45%

Metronome v2• Worst-case pause 1.5ms; utilization at least 70%• For periodic scheduled tasks up to 200 Hz, jitter is 4us• Multiprocessor algorithm• Implementation in IBM’s J9 virtual machine product• Alpha versions in use by customers and universities




















elements


IBM Real-Time Java (J9 Virtual Machine)

Metronome Real-time Garbage Collection• Provides real-time without changing the programming model

Ahead-of-Time Compilation• Ahead-of-time (AOT) compilation and JXE Linking• Removes JIT non-determinism, allows code ROMification

RTSJ (Real-Time Specification for Java) – standard• Scheduling• Scopes (yuck)




















elements


Why is real-time collection different?

Why is real-time collection different from other GCs?• Must break work into increments (while heap changes)• Must bound individual work increments• Must guarantee space bounds and timely termination

Why is real-time collection different from other tasks?• Deadline for overall task is low-frequency• Allows re-distribution of work over individual increments




















elements


Redistributing Collector Work

Application Collector

Time

Spa

ce

a = allocation rate

c = collection rate

Example Application

Allocates half as fast asthe collector can collect

c = -2a

Resulting Schedule

Base Application Memory




















elements


Allocation Rate Determines Utilization

Application

Timea =

alloc

ation

rate

Resulting Schedule: Utilization = 50%

Base Application Memory

Collector

c = collection rateSpa

ce

Utilization = 66%




















elements


View of Metronome Collector Schedule

Application

allocates auΔt

Collector

collects part of m

Δt

uΔt (1-u)Δt




















elements


Generational Metronome

Goal:

Increase utilization and reduce memory consumption




















elements


Nursery Reduces Allocation Rate

Nursery Heapaηa

Heapa

High allocation rate

Lower allocation rate - but slows main heap collection




















elements


Synchronous Nursery Collection Main heap collection is concurrent (in even increments)

• Concurrent nursery collection would be extremely complex Small nursery can be collected in a very short time

• Greatly simplifies design How much time?

• Nursery size N, survival rate η, collection rate RN

• Nursery collection work is proportional to live data

• Time to collect is Nη/RN

NNη




















elements


Quantifying the Scheduling Tradeoff

Application

allocates auΔt

Collector

part of m

Δt

uΔt (1-u)Δt

Heapcoll. part of m

Nurserycoll. aηuΔt

aηuΔt / RN




















elements


When to use Generational Collection?

0

2

4

6

8

10

12

14

16

18

20

0 0.2 0.4 0.6 0.8 1Utilization

Rel

ativ

e H

eap

Size

Non-generationalGen (_ = 0.50)Gen (_ = 0.30)Gen (_ = 0.15)Gen (_ = 0.05)Genl (_ = 0.01)




















elements


Handling Non-uniformProgram Behavior




















elements


Can Application be Modeled So Simply?

Collector

Time

Spa

ce

a = allocation rate?? c = collection rate




















elements


Maximum Mutator Allocation Rate (MMAR)

0

100

200

300

400

500

600

700

100 800 1500220029003600430050005700640071007800850092009900

Time Interval _ t (_s)

MM

AR

cryptokXMLPNG




















elements


Complications due to non-uniformity

Cost of collection is dependent on varying parameters• High allocation rate: frequent nursery collection• High survival rate: expensive individual collection

Tension• Survival rate drops as nursery gets larger (more time to die)• But WCET of nursery collection increases

Uniform aη Non-uniform a Non-uniform η




















elements


Syncopation: Move work to unstressed interval

Spike in allocation rate forces frequent nursery GC’s• Degenerates into work-based collection

Syncopation: pre-tenure all objects• During allocation spikes, allocate directly into heap• No nursery collection needed

Problem: we don’t find out until it’s too late• Once collector quantum is used, can’t GC for a mutator quantum




















elements


Scheduling: Multiple Beats per MeasureΔt

u = 2/3

12 beats/measure3/12 time; u = 3/4

u = 8/12 = 2/3

u = 9/12; reserve restored

u = 9/12 = 3/4

pre-tenuring




















elements


Cost of Syncopation

During syncopation, heap consumed at rate a, not aη• No benefit from generational collection during this time• Even if program obeys the generational hypothesis

Space bound requires characterization of spikes• Which requires more detailed characterization of application• Tradeoff: more detail, tighter bound

• See Mann et al (next paper in session)




















elements


Increasing the EffectiveNursery Size




















elements


Reducing the Survival Rate

Multiple beats per measure means smaller nurseries• Which leads to higher survival rates• Which reduces effectiveness of generational collection

Large objects contribute disproportionately• Arraylet pre-tenuring solves this problem




















elements


What Are Arraylets?

Large arrays create problems for real-time collection• Fragment memory space• Can not be moved in a short, bounded time

Solution: break large arrays into arrayoids & arraylets• Access via indirection; move one arraylet at a time• Optimize: opportunistic contiguity, strip-mining, CSE

A1 A2 A3

A

A1 A2

A3




















elements


Nursery

Heap

Arraylet Pre-tenuring

A1 A2 A3

Exploit arraylet indirection• Arrayoid allocated in nursery• Arraylets allocated in heap• Nursery consumption:

– Array Size / Page Size + Header Size On nursery collection

• If array is live, copy– Arraylet pointers unchanged

• If array is dead, reclaim arraylets Issues for pointer arrays

• Arraylets could ref. nursery• Requires extra fixup work• Must factor into GC time




















elements


Reduction in Allocation Rate (32K nursery)

0

5

10

15

20

25

30

35

40

45

50

Ches

s

Ches

s Gen

Ches

s Alet

Cryp

to

Cryp

to Ge

n

Cryp

to A

let

kXM

L

kXM

L Gen

kXM

L Alet

Para

llel

Para

llel G

en

Para

llel A

let

PNG

PNG

Gen

PNG

Alet

Rege

x

Rege

x Gen

Rege

x Alet

Allo

catio

n (M

B/s)

Large Objects

Small Objects




















elements


Conclusions

Syncopation allows RT scheduling of generational GC• Modest over-provisioning required (extra beat)

Generational collection can improve performance• Space and utilization

Arraylet pre-tenuring increases effectiveness• Increases virtual nursery size; more time to die

Cost: more precise (complex) application characterization




















elements


Ongoing Work Help!

Handlers [w/Dan Spoonhower, CMU]• Allow low-latency (microsecond) reponse for aperiodic tasks• Safe – checked at load time; eliminate the need for scopes

E-code support: Logical Execution Time [w/Kirsch & Henzinger]• Provide deterministic timing across multiple platforms• Scheduling is a first-class virtual machine primitive• Supported in product code!

Trace analysis and visualization [w/Matthias Hauswirth, Lugano] Probabilistic real-time methodology Verifiable concurrent collectors [w/Martin Vechev, Cambridge] Wait-free scalable SMP collector Porting to Stargate nodes (and other embedded ARMs)




















elements


Questions?

http://www.research.ibm.com/metronome




















elements


PNG

0

5000000

10000000

15000000

20000000

25000000

30000000

35000000

40000000

1 25 49 73 97 121 145 169 193 217 241 265 289 313 337 361 385 409 433 457 481 505 529 553 577

100us

1000us 10000us




















elements


Allocation Stability vs. Time Scale

Peak Allocation Rate

0

50

100

150

200

250

300

100 500 1000 5000 10000 Average

Time Window (microseconds)

MB

/s (p

eak)

Documents

Maximum length: 1 line Syncopation - IBMresearcher.watson.ibm.com/.../us-bacon/Bacon05SyncopationSlides.pdf · Syncopation: Generational Real-time Garbage Collection in the Metronome