Performance Pathologies in Hardware Transactional Memory

Performance Pathologies in Hardware Transactional Memory

Jayaram Bobba, Kevin E. Moore, Haris Volos, Luke Yen, Mark D. Hill, Michael M. Swift, and David A. Wood

Multifacet Project (www.cs.wisc.edu/multifacet)

Computer Sciences Dept., University of Wisconsin Madison

http://www.cs.wisc.edu/multifacet

© 2007 Multifacet Project University of Wisconsin Madison

• Multiple Hardware Transactional Memory Designs– Best Performance ?

• Depends !

Executive Summary (1/2)

0

0.5

1

1.5

2

2.5

Sp

eed

up

Norm

alized

to E

E

Cholesky Radiosity Raytrace

EE

EL

LL

EE_HP

EL_T

LL_B

Liv

elo

ck


0

0.5

1

1.5

2

2.5

Sp

eed

up

Norm

alized

to E

E


EE

EL

LL

EE_HP

EL_T

LL_B

Executive Summary (2/2)

• Why?

– Performance Pathologies

• What to do?– Better Conflict Resolution

Performance Pathologies Performance

Liv

elo

ck

pa●thol●o●gy: any deviation from a normal, healthy or efficient condition


Talk Outline

• Motivation• Base HTM Results

• Performance Pathologies• Enhanced HTM Systems


Motivation

• Which HTM system performs best?

• Different Assumptions– Hardware

• Broadcast vs Directory Based

– Software:

• Continuous Transactions vs Critical Section Transactions

• How to compare?


HTM Design Space

Version ManagementVersion Management

Co

nflict D

etection

Co

nflict D

etection

Lazy Eager

Lazy

Eager

Herlihy/Moss TM

MIT LTM

Intel/Brown VTM

Stanford TCC

Illinois Bulk

MIT UTM

Wisconsin LogTM

No HTMs (yet)

Conflict ResolutionConflict Resolution: Adhoc

“Better By Design” ?


Experimental Setup

• Base HTM Systems– – –

• Common Hardware Platform– 32 core CMP– Directory-Based Coherence

• Common TM Programming Model– Critical Section Based Transactions

• Common Workloads– 5 SPLASH + 2 Microbenchmarks (Btree, LFUCache)


And the winner is…

• Low Contention: Similar• High Contention: Depends

0

0.2

0.4

0.6

0.8

1

1.2S

peed

up

Norm

alized

to E

E


EE

EL

LL

Liv

elo

ck


Talk Outline

• Motivation• Base HTM Results

• Performance Pathologies• Enhanced HTM Systems


Performance Pathologies

• StarvingElder• SerializedCommit• RestartConvoy

• FriendlyFire

• DuelingUpgrades• FutileStall• StarvingWriter


System

• Conflict Detection: LazyDetect conflicts at commit time (Validation)

• Version Management: LazyNew value elsewhere, Update on commit

+ Abort

- Commit

• Conflict Resolution: Committer Wins

compare with TCC, Bulk


COMMITING

STALLED

ABORTING

CAUSE

Committer Wins

EFFECT(S)

• Starvation

• Load Imbalance

FIX

Elder gets Priority ?

StarvingElder

store A

load A

ABORT

store A

load A

ABORT

store A

load A

ABORT

Tim

e


SerializedCommit

CAUSE

Serial Validation

EFFECT(S)

• Unnecessary Stall

FIX

Parallel Validation ?

Tim

e


RestartConvoy

load A load A load A

CAUSE

Fast Abort +

Immediate Restart

EFFECT(S)

• Resource Contention

• Serialized Execution

≈ Lock Convoys

FIX

Stagger Restarts ?

load A load A load A

load A store A

ABORTABORTABORT

load A

load A load A load Astore A

ABORT ABORT ABORT


Pathologies

• Pathology (% ExecutionTime)

• Enhancement– Linear Backoff on Abort– Eliminate RestartConvoys

StarvingElder

Serialized Commit

Restart Convoy

StarvingElder

SerializedCommit

Restart Convoy

Raytrace 45 27 5.2 0.1 0.3 1.0


0

0.5

1

1.5

2

2.5

Raytrace

Sp

ee

du

p N

orm

ali

ze

d t

o E

E

LL

LL_B

Enhanced LL ( )


• Performance

StarvingElder

Serialized Commit

Restart Convoy

StarvingElder

SerializedCommit

Restart Convoy

Raytrace 45 27 5.2 0.1 0.3 1.0




• FriendlyFire



System

• Conflict Detection: Eager– Detect conflicts immediately

• Version Management: Lazy– New Value elsewhere, Update on commit

+ Abort

- Commit

• Conflict Resolution: – Requester Wins– Exponential Backoff on Abort

compare with HMTM, LTM


FriendlyFire

load Astore A

load A

store A

ABORT

ABORT

load A

store A

load A

store A

ABORT

ABORT

ABORT

CAUSE

Requester Wins

EFFECT(S)

• Unnecessary Aborts

• Livelock

FIX

Priority Based Conflict Resolution ?


Pathologies


• Enhancement– Timestamp Based Conflict Resolution– Eliminate FriendlyFire

Friendly Fire

FriendlyFire

Raytrace 73 0.2


Enhanced EL ( )


• Performance

0

0.5

1

1.5

2

2.5

Raytrace

Sp

ee

du

p N

orm

ali

zed

to

EE

EL

EL_T

Friendly Fire

FriendlyFire

Raytrace 73 0.2




• FriendlyFire



System

• Conflict Detection: Eager– Detect conflicts immediately

• Version Management: Eager– New Value in-place, Restore on Abort- Abort+ Commit

• Conflict Resolution– Stall Requester– Abort Requester on possible deadlock (Conservative Deadlock Avoidance)

compare with LogTM


DuelingUpgrades

load A

load Astore A

store A

load A

store A

STALL

ABORT

CAUSE

Conservative Deadlock Avoidance

EFFECT(S)


• Unnecessary Abort

FIX

Acquire Store Permissions Early ?


FutileStall

store Aload A

ABORT

CAUSE

Stall Requester

EFFECT(S)


FIX

No simple fix ?

STALL


StarvingWriter

store Aload A

load A

load Aload A

load A

CAUSE

Stall Requester

EFFECT(S)

• Starvation

• Load Imbalance

≈ Reader/Writer Locks

FIX

Writer Priority ?


Pathologies


• Enhancements– DuelingUpgrades: Store-Set Predictor– StarvingWriter: Writer Aborts Readers immediately

% Execution Time Aborts /Transaction


FutileStall StarvingWriter

Dueling Upgrades

Futile Stall

StarvingWriter

Dueling Upgrades

Raytrace 1.0 4.6 4.2 <0.1 0.3 <0.1


Enhanced EE ( )


• Performance

0

0.5

1

1.5

2

2.5

Raytrace

Sp

ee

du

p N

orm

ali

zed

to

EE

EE

EE_HP



FutileStall StarvingWriter

Dueling Upgrades

Futile Stall

StarvingWriter

Dueling Upgrades

Raytrace 1.0 4.6 4.2 <0.1 0.3 <0.1


Caveats / FutureWork

• High Contention Scenarios– Better Conflict Resolution– Unidentified Pathologies

• Better TM Workloads

Restart Convoy

StarvingElder

Serialized Commit

RestartConvoy

StarvingElder

SerializedCommit

Mp3d 21 36 30 9.0 28 25


0

0.5

1

1.5

2

2.5

Sp

eed

up

Norm

alized

to E

E


EEELLLEE_HPEL_TLL_B

Executive Summary

• Performance Pathologies• Enhanced HTM systems

Liv

elo

ck

DuelingUpgrades

SerializedCommit StarvingElder

StarvingWriter

FriendlyFire

FutileStall

RestartConvoy

Documents

Performance Pathologies in Hardware Transactional Memory