Targeted Path Profiling : Lower Overhead Path Profiling for Staged Dynamic Optimization Systems

Targeted Path Profiling: Lower Overhead Path Profiling for Staged Dynamic Optimization Systems

Rahul Joshi, UIUC

Michael Bond*, UT Austin

Craig Zilles, UIUC

2

Path information is useful

Enlarges scope of optimizations– Superblock formation– Hyperblock formation

Improves other optimizations– Code scheduling and register allocation– Dataflow analysis– Software pipelining– Code layout– Static branch prediction

3

Overhead vs. accuracy

0

10

20

30

40

50

75 80 85 90 95 100

Accuracy (%)

Ove

rhea

d (

%)

Edge profiling(SPEC 95 INT)

4


0

10

20

30

40

50

75 80 85 90 95 100

Accuracy (%)

Ove

rhea

d (

%)


Ball-Laruspath profiling(SPEC 2000 INT)

5


0

10

20

30

40

50

75 80 85 90 95 100

Accuracy (%)

Ove

rhea

d (

%)



Targetedpath profiling(SPEC 2000 INT)

6


0

10

20

30

40

50

75 80 85 90 95 100

Accuracy (%)

Ove

rhea

d (

%)



Targetedpath profiling(SPEC 2000 INT)

Profile-guided profiling

7

Outline

Background– Staged dynamic optimization and

profile-guided profiling– Ball-Larus path profiling– Opportunities for reducing overhead

Targeted path profiling Results

– Overhead and accuracy

8

Staged dynamic optimization

Staticoptimizations

Stage 0

9


Staticoptimizations

Edgeprofile

Stage 0

Hardwareedge profiler

10


Staticoptimizations

Edgeprofile

Stage 0

LocalOptimizations(code layout)

Stage 1


11


Staticoptimizations

Edgeprofile

Stage 0


Path profilinginstrumentation

Stage 1


12


Staticoptimizations

Edgeprofile

Stage 0



Stage 1

Pathprofile


13


Staticoptimizations

Edgeprofile

Stage 0



GlobalOptimizations(superblockformation)

Stage 2 Stage 1

Pathprofile


14

Profile-guided profiling

Staticoptimizations

Stage 0



GlobalOptimizations(superblockformation)

Stage 2 Stage 1

Pathprofile


Edgeprofile

15

Ball-Larus path profiling

Acyclic, intraprocedural paths Handles cyclic CFGs

– Paths end at loop back edges

Each path computes unique integer

16


4 paths

CB

D

A

FE

G

17


2

1

4 paths Each path computes

unique integerCB

D

A

FE

G

18


2

1


unique integer

Path 0

CB

D

A

FE

G

19


2

1


unique integer

Path 0 Path 1

CB

D

A

FE

G

20


2

1


unique integer

Path 0 Path 1 Path 2

CB

D

A

FE

G

21


2

1


unique integer

Path 0 Path 1 Path 2 Path 3

CB

D

A

FE

G

22


r=r+2

r=0

r=r+1

count[r]++

r: path register

count: array of path frequencies CB

D

A

FE

G

23

Overhead in Ball-Larus path profiling

SPEC 95 SPEC 2000

gcc 96% 87%

INT Avg 41% 43%

FP Avg 12% 22%

Overall Avg 28% 37%

24

Overhead in Ball-Larus path profiling

SPEC 95 SPEC 2000

gcc 96% 87%

INT Avg 41% 43%

FP Avg 12% 22%

Overall Avg 28% 37%

Opportunities for reducing overhead?– When there are many paths– When edge profile gives perfect path profile

25

Routines with many paths

Many possible paths– Exponential in number of edges– Can’t use array of counters

Number of taken paths small– Ball-Larus uses hash table– Hash function call expensive

Hashed path ~5 times overhead

26

Edge profile gives perfect path profile

27


28


An obvious path contains an edge that is only on that path– Path uniquely identified

by edge– Path freq = edge freq

If all paths obvious, edge profile gives perfect path profile

29

Outline





30

Targeted path profiling

Profile-guided profiling– Use existing edge profile

Exploits opportunities for reducing overhead– When there are many paths

Remove cold edges– When edge profile gives perfect path profile

Don’t instrument obvious routines and loops

31

Removing cold edges

Examine relative execution frequency of each branch

if (relFreq < threshold)

edge is cold

3 97

32

Removing cold edges

4060

397

1000

5050



edge is cold

3 97

33

Removing cold edges

4060

397

1000

5050



edge is cold

3 97

34

Removing cold edges

4060

397

1000

5050

A path that contains a cold edge is a cold path

Removing an edge may halve number of paths

35

Removing cold edges

4060

97

100

5050



Number of paths: 16 4

36

Removing cold edges

4060

97

100

5050



Number of paths: 16 4

Goal: hashed non-hashed

37

Removing cold edges

Remaining paths potentially hot

4 paths [0, 3]

2

1

38

Removing cold edges

r=r+2

r=0

r=r+1

count[r]++

Remaining paths potentially hot

4 paths [0, 3]

39

Removing cold edges

What if cold edge taken? r=r+2

r=0

r=r+1

count[r]++

40

Removing cold edges

What if cold edge taken?

Cold edges poison path

r=r+2

r=0

r=poison

r=poison

r=r+1

count[r]++

41

Removing cold edges

What if cold edge taken?

Cold edges poison path

Instrumentation checks for poisoned path

r=r+2

r=0

r=poison

r=poison

r=r+1

if (r poisoned) cold_counter++else count[r]++

42

Checking for poison

if (r poisoned) cold_counter++else count[r]++

43

Obvious routines

All paths obvious We don’t instrument

obvious routines Edge profile gives

perfect path profile

44

Obvious loops

Loop with obvious body Don’t instrument

obvious loops with high average trip counts

Edge profile yields high-accuracy path profile

…

…

45

Obvious loops

Loop with obvious body Don’t instrument

obvious loops with high average trip counts

Edge profile yields high-accuracy path profile

…

…

46

Summary of our techniques

Remove cold edges– Eliminates many cold paths– Count paths with array (instead of hash table)

Don’t instrument obvious routines and loops– Edge profile derives path profile

47

Outline





48

Implementation

Static profiling PP: tool for path profiling TPP: tool for targeted path profiling Tools instrument native SPARC executables

– SPEC 95 ref– SPEC 2000 ref

49

Results: SPEC 2000 INT

0

10

20

30

40

50

60

70

80

90

100

Ov

erh

ea

d/A

cc

ura

cy

Ball-Larus PP overhead TPP overhead Accuracy

50

Where does benefit come from?

Cold path elimination alone: 60% Add obvious path elimination: + 40%

Little benefit from obvious path elimination alone

51

Related work

Dynamo [Bala et al. ‘00]

– Successful online path-guided optimization– “Bails out” when no dominant path

Instrumentation sampling [Arnold & Ryder ‘01]

– Orthogonal to targeted path profiling

Selective path profiling [Apiwattanapong & Harrold ’02]

– Useful when only a few paths of interest

52

Summary

Profile-guided profiling in a staged dynamic optimization system

Two synergistic techniques– Remove cold paths– Don’t instrument obvious routines and loops

Reduces overhead by half (SPEC 95) to two-thirds (SPEC 2000)

High accuracy: ~99%

53

Remaining slides not part of talk

54

Future work

Targeted path profiling in a staged dynamic optimization system– Jikes RVM

55

Future work

Targeted path profiling in a staged dynamic optimization system– Jikes RVM

Pseudo-obvious subgraphs Maintaining path profiles across

program transformations

56


Edgeprofiler

Edgeprofile

Stage 0:Staticoptimizations


Pathprofile

Stage 2:Globaloptimizations

Stage 1:Localoptimizations

57

Accuracy

Our techniques lose path information– For removed cold paths (cold counter)– For paths that enter or exit disconnected loops

Accuracy of targeted path profiling: ~99%

Accuracy of edge profiling: 80% SPEC 95 (76% INT, 84% FP)

58

Why not edge profiling?

Edge profile is “point” profile

Correlation between edge frequencies ambiguous

CB

D

A

FE

G

50 50

50 50

59

Edge profile limitations



CB

D

A

FE

G

50 50

50 50

60

Edge profiling limitations



CB

D

A

FE

G

50 50

50 50

61


Dynamic optimization system decides if profiling likely to be beneficial

Staged dynamic optimization system applies more powerful and expensive optimizations at each stage

62

Cyclic graphs

2 pathsA

C

E

F

B

D

63

Cyclic graphs

2 paths 8 paths Acyclic paths

– Start at A or B– End at E or F

A

C

F

B

D

E

64

Cyclic graphs



A

C

F

B

D

count[r]++

r=0

E

65

Cyclic graphs



A

C

F

B

D

count[r]++r=0

count[r]++

r=0

E

66

Cyclic graphs



Paths enter and/or exit loop body

A

C

F

B

D

count[r]++r=0

count[r]++

r=0

E

Documents

Targeted Path Profiling : Lower Overhead Path Profiling for Staged Dynamic Optimization Systems