Computer & Information Sciences - University of Delaware Colloquium / 55 Exhaustive Phase Order...

Computer & Information Sciences - University of Delaware Colloquium / 55

Exhaustive Phase Order Search Space Exploration and

Evaluation

Prasad Kulkarni

(Florida State University)

2Computer & Information Sciences - University of Delaware Colloquium / 55

Compiler Optimizations

• To improve efficiency of compiler generated code

• Optimization phases require enabling conditions– need specific patterns in the code– many also need available registers

• Phases interact with each other• Applying optimizations in different orders

generates different code

Phase Ordering Problem

• To find an ordering of optimization phases that produces optimal code with respect to possible phase orderings

• Evaluating each sequence involves compiling, assembling, linking, execution and verifying results

• Best optimization phase ordering depends on– source application– target platform– implementation of optimization phases

• Long standing problem in compiler optimization!!

Phase Ordering Space

• Current compilers incorporate numerous different optimization phases– 15 distinct phases in our compiler backend

• 15! = 1,307,674,368,000

• Phases can enable each other– any phase can be active multiple times

• 1515 = 437,893,890,380,859,375

– cannot restrict sequence length to 15• 1544 = 5.598 * 1051

Addressing Phase Ordering

• Exhaustive Search – universally considered intractable

• We are now able to exhaustively evaluate the optimization phase order space.

Re-stating of Phase Ordering

• Earlier approach– explicitly enumerate all possible optimization

phase orderings

• Our approach– explicitly enumerate all function instances that

can be produced by any combination of phases

Outline

• Experimental framework

• Exhaustive phase order space evaluation

• Faster conventional compilation

• Conclusions

• Summary of my other work

• Future research directions

Outline

• Conclusions

Experimental Framework

• We used the VPO compilation system– established compiler framework, started development

in 1988– comparable performance to gcc –O2

• VPO performs all transformations on a single representation (RTLs), so it is possible to perform most phases in an arbitrary order

• Experiments use all the 15 re-orderable optimization phases in VPO

• Target architecture was the StrongARM SA-100 processor

VPO Optimization Phases

ID Optimization Phase ID Optimization Phase

b branch chaining l loop transformations

c common subexpr. elim. n code abstraction

d remv. unreachable code o eval. order determin.

g loop unrolling q strength reduction

h dead assignment elim. r reverse branches

i block reordering s instruction selection

j minimize loop jumps u remv. useless jumps

k register allocation

Disclaimers

• Did not include optimization phases normally associated with compiler front ends– no memory hierarchy optimizations– no inlining or other interprocedural

optimizations

• Did not vary how phases are applied

• Did not include optimizations that require profile data

Benchmarks• 12 MiBench benchmarks; 244 functions

Category Program Description

autobitcount test processor bit manipulation abilitiesqsort sort strings using quicksort sorting algorithm

networkdijkstra Dijkstra’s shortest path algorithmpatricia construct patricia trie for IP traffic

telecommfft fast fourier transformadpcm compress 16-bit linear PCM samples to 4-bit

consumerjpeg image compression and decompressiontiff2bw convert color .tiff image to b&w image

securitysha secure hash algorithmblowfish symmetric block cipher with variable length key

officestringsearch searches for given words in phrasesispell fast spelling checker

Outline

• Conclusions

Terminology

• Active phase – An optimization phase that modifies the function representation

• Dormant phase – A phase that is unable to find any opportunity to change the function

• Function instance – any semantically, syntactically, and functionally correct representation of the source function (that can be produced by our compiler)

Naïve Optimization Phase Order Space

• All combinations of optimization phase sequences are attempted

a b cd a d a d a d

b c b c b c

Eliminating Consecutively Applied Phases

• A phase just applied in our compiler cannot be immediately active again

b cd a d a d a

c b b c

a b cd

Eliminating Dormant Phases

• Get feedback from the compiler indicating if any transformations were successfully applied in a phase.

b cd a d a d

Identical Function Instances

• Some optimization phases are independent

– example: branch chaining & register allocation

• Different phase sequences can produce the same code

r[2] = 1;r[2] = 1; r[3] = r[4] + r[2];r[3] = r[4] + r[2];

instruction selectioninstruction selection r[3] = r[4] + 1;r[3] = r[4] + 1;

r[2] = 1;r[2] = 1; r[3] = r[4] + r[2];r[3] = r[4] + r[2];

constant propagationconstant propagation r[2] = 1;r[2] = 1; r[3] = r[4] + 1;r[3] = r[4] + 1;

dead assignment eliminationdead assignment elimination r[3] = r[4] + 1;r[3] = r[4] + 1;

Equivalent Function Instancessum = 0;for (i = 0; i < 1000; i++ ) sum += a [ i ];

Source Code

r[10]=0; r[12]=HI[a]; r[12]=r[12]+LO[a]; r[1]=r[12]; r[9]=4000+r[12];L3 r[8]=M[r[1]]; r[10]=r[10]+r[8]; r[1]=r[1]+4; IC=r[1]?r[9]; PC=IC<0,L3;

Register Allocation before Code Motion

Code Motion before Register Allocation

After Mapping Registers

Efficient Detection of Unique Function Instances

• After pruning dormant phases there may be tens or hundreds of thousands of unique instances

• Use a CRC (cyclic redundancy check) checksum on the bytes of the RTLs representing the instructions

• Used a hash table to check if an identical or equivalent function instance already exists in the DAG

Eliminating Identical/Equivalent Function Instances

• Resulting search space is a DAG of function instances

cd a d a

Static Enumeration ResultsFunction Inst. Fn_inst Len CF

Batch Vs.

optimal

start_input_bmp (j) 1,372 120,777 25 70 1.41

correct(i) 1,295 1,348,154 25 663 4.18

main (t) 1,276 2,882,021 29 389 16.25

parse_switches (j) 1,228 180,762 20 53 0.41

start_input_gif (j) 1009 39,352 21 18 2.46

start_input_tga (j) 972 63,458 21 30 1.66

askmode (i) 942 232,453 24 108 7.87

skiptoword (i) 901 439,994 22 103 1.45

start_input_ppm (j) 795 8,521 16 45 2.70

Average (234) 196.2 89,946.7 14.7 36.2 6.46

Exhaustively enumerated the optimization phase order space to

find an optimal phase ordering with respect to code-size

[Published in CGO ’06]

Determining Program Performance

• Almost 175,000 distinct function instances, on average– largest enumerated function has 2,882,021 instances

• Too time consuming to execute each distinct function instance– assemble link execute more expensive than compilation

• Many embedded development environments use simulation– simulation orders of magnitude more expensive than execution

• Use data obtained from a few executions to estimate the performance of all remaining function instances

Determining Program Performance (cont...)

• Function instances having identical control-flow graphs execute each block the same number of times

• Execute application once for each control-flow structure

• Statically estimate the number of cycles required to execute each basic block

• dynamic frequency measure = static cycles * block frequency)

Predicting Relative Performance – I

4 cycles

20 cycles

27 cycles

2 cycles

5 cycles

10 cycles

4 cycles

22 cycles

25 cycles

2 cycles

10 cycles

Total cycles = 744 Total cycles = 789

Dynamic Frequency Results

Function Inst. Fn_inst Len CF Leaf% from optimal

Batch Worst

main (t) 1,276 2,882,021 29 389 15,164 0.0 84.3

parse_switches (j) 1,228 180,762 20 53 2,027 6.7 64.8

askmode (i) 942 232,453 24 108 475 8.4 56.2

skiptoword (i) 901 439,994 22 103 2,834 6.1 49.6

start_input_ppm (j) 795 8,521 16 45 80 1.7 28.4

pfx_list_chk (i) 640 1,269,638 44 136 4,660 4.3 78.6

main (f) 624 2,789,903 33 122 4,214 7.5 46.1

sha_transform (h) 541 548,812 32 98 5,262 9.6 133.4

main (p) 483 14,510 15 10 178 7.7 13.1

Average (79) 234.3 174,574.8 16.1 47.4 813.4 4.8 65.4

Correlation – Dynamic Frequency Counts Vs. Simulator Cycles

• Static performance estimation is inaccurate– ignored cache/branch misprediction penalties

• Most embedded systems have simpler architectures– estimation may be sufficiently accurate– simulator cycles are close to executed cycles

• We show strong correlation between our measure of performance and simulator cycles

Complete Function Correlation

• Example: init_search in stringsearch

Leaf Function Correlation

• Leaf function instances are generated when no additional phases can be successfully applied

• Leaf instances provide a good sampling– represents the only code that can be generated by an

aggressive compiler, like VPO– at least one leaf instance represents an optimal phase

ordering for over 86% of functions– significant percent of leaf instances among optimal

Leaf Function Correlation Statistics

• Pearson’s correlation coefficient

• Accuracy of our estimate of optimal perf.

xy – (xy)/n

sqrt( (x2 – (x)2/n) * (y2 - (y)2/n) )Pcorr =

Lcorr = cycle count for best leaf

cy. cnt for leaf with best dynamic freq count

Leaf Function Correlation Statistics (cont…)

Function PcorrLcorr 0% Lcorr 1%

Ratio Leaves Ratio LeavesAR_btbl...(b) 1.00 1.00 1 1.00 1BW_btbl...(b) 1.00 1.00 2 1.00 2bit_count.(b) 1.00 1.00 2 1.00 2bit_shifter(b) 1.00 1.00 2 1.00 2bitcount(b) 0.89 0.92 1 0.92 1main(b) 1.00 1.00 6 1.00 23ntbl_bitcnt(b) 1.00 0.95 2 0.95 2ntbl_bit…(b) 0.99 1.00 2 1.00 2dequeue(d) 0.99 1.00 6 1.00 6dijkstra(d) 1.00 0.97 4 1.00 269.... …. …. …. …. ….

average 0.96 0.98 4.38 0.996 21

Exhaustively evaluated the optimization phase order space tofind a near-optimal phase ordering

with respect to simulator cycles

[Published in LCTES ’06]

Outline

• Conclusions

Phase Enabling Interaction

• b enables a along the path a-b-a

Phase Enabling Probabilities

Ph St b c d g h i j k l n o q r s u

b 0.72 0.02 0.010.04 0.01 0.02 0.66

c 1.00 0.01 0.680.01

0.02 0.07 0.05 0.15 0.34

d 1.00 1.00 1.00

g 0.22 0.28 0.17 0.05 0.02 0.14 0.34 0.09 0.15

h 0.08 0.16 0.14 0.020.01 0.20

i 0.72 0.04 0.01 0.09

j 0.030.06

k 0.98 0.28 0.01

0.01 0.96

l 0.60 0.73 0.02 0.01 0.01

0.03 0.53

n 0.41 0.36 0.01 0.01 0.01 0.29

o 0.88 0.40

q 0.99 0.02 0.99

r 0.570.06 0.06

s 1.00 0.33 0.41 0.83 0.07 0.050.15 0.07

u 0.01 0.01

Phase Disabling Interaction

• b disables a along the path b-c-d

Disabling Probabilities

Ph b c d g h i j k l n o q r s u

b 1.00 0.28 0.09 0.18 0.20 0.11 0.01

c 0.01 1.00 0.02

1.00 0.08

d 1.00 0.03

g 0.13 1.00 0.06 0.01 0.12 0.22

h 0.01 0.01 1.00

i 0.02 0.22 1.00 0.20 0.01 0.44 0.91

j 0.01 0.08 1.00 0.01 0.16

k 0.010.05 1.00

l 0.02 1.00

0.11 0.04 0.07 1.00

n 0.07 0.01 0.02 0.01 0.01 1.00 1.00 0.01

o 0.01 0.08 0.01 1.00

q 1.00

r 0.06 0.200.36 1.00 0.05

s 0.07 0.03

0.02 1.00

u 0.41 0.02 0.34 0.15 1.00

Faster Conventional Compiler

• Modified VPO to use enabling and disabling phase probabilities to decrease compilation time

# p[i] - current probability of phase i being active # e[i][j] - probability of phase j enabling phase i # d[i][j] - probability of phase j disabling phase i

For each phase i do p[i] = e[i][st]; While (any p[i] > 0) do Select j as the current phase with highest probability of being active Apply phase j If phase j was active then For each phase i, where i != j do

p[i] += ((1-p[i]) * e[i][j]) - (p[i] * d[i][j]) p[j] = 0

Probabilistic Compilation Results

Function Old Compilation Prob. Compilation Prob. / OldAttempted Active Attempted Active Time Size Speed

start_inp...(j) 233 16 55 14 0.469 1.014 N/A parse_swi...(j) 233 14 53 12 0.371 1.016 0.972start_inp...(j) 270 15 55 14 0.353 1.010 N/A start_inp...(j) 233 14 49 13 0.420 1.003 N/A start_inp...(j) 231 11 53 12 0.436 1.004 1.000fft_float(f) 463 28 99 25 0.451 1.012 0.974main(f) 284 20 73 18 0.550 1.007 1.000sha_trans...(h) 284 17 67 16 0.605 0.965 0.953read_scan...(j) 233 13 43 10 0.342 1.018 N/A LZWReadByte(j) 268 12 45 11 0.325 1.014 N/A main(j) 270 12 57 14 0.375 1.007 1.000dijkstra(d) 231 9 43 9 0.409 1.010 1.000.... .... .... .... .... .... .... ....

average 230.3 8.9 47.7 9.6 0.297 1.015 1.005

Outline

• Conclusions

Conclusions

• Phase ordering problem– long standing problem in compiler optimization– exhaustive evaluation always considered infeasible

• Exhaustively evaluated the phase order space– re-interpretation of the problem– novel application of search algorithms– fast pruning techniques– accurate prediction of relative performance

• Analyzed properties of the phase order space to speedup conventional compilation

• published in CGO’06, LCTES’06, submitted to TOPLAS

Challenges

• Exhaustive phase order search is a severe stress test for the compiler– isolate analysis required and invalidated by each phase– produce correct code for all phase orderings– eliminate all memory leaks

• Search algorithm needs to be highly efficient– used CRCs and hashes for function comparisons– stored intermediate function instances to reduce disk

access– maintained logs to restart search after crash

Outline

• Conclusions

• Provides an interactive code improvement paradigm– view low-level program representation– apply existing phases and manual changes in any

order– browse and undo previous changes– automatically obtain performance information– automatically search for effective phase sequences

• Useful as a research as well as teaching tool– employed in three universities

• published in LCTES ’03, TECS ‘06

VISTA – Main Window

Faster Genetic Algorithm Searches

• Improving performance of genetic algorithms– avoid redundant executions of the application

• over 87% of executions were avoided• reduce search time by 62%

– modify search to obtain comparable results in fewer generations

• reduced GA generations by 59%• reduce search time by 35%

• published in PLDI ’04, TACO ’05

Heuristic Search Algorithms

• Analyzing the phase order space to improve heuristic algorithms– detailed performance and cost comparison of different

heuristic algorithms– demonstrated the importance and difficulty of

selecting the correct sequence length– illustrated the importance of leaf function instances– proposed modifications to existing algorithms, and

new search algorithms

• Will be published in CGO ‘07

Dynamic Compilation

• Explored asynchronous dynamic compilation in a virtual machine– demonstrated shortcomings of current popular

compilation strategy– describe importance of minimum compiler utilization– discussed new compilation strategies– explored the changes needed to current compilation

strategies to exploit free cycles

• Submitted to VEE ‘07

Outline

• Conclusions

• Support for parallelism– traditional languages– express parallelism– dynamic scheduling

• Virtual machines– dynamic code generation

and optimization

• Push compilation decisions further down

Compiler Technology Challenges

c o m p i l e r

• multi-core– heterogeneous cores

• No great solution– performance monitoring– software-controlled

reconfiguration

• Can no longer do it alone

High LevelLanguage

MachineArchitecture

Iterative Compilation & Machine Learning

• Improved scope for iterative compilation & machine learning– proliferation of new architectures

• automate tuning compiler heuristics

– tuning important libraries– using performance monitors

• dynamic JIT compilers

• How to use machine learning to optimize and schedule more efficiently ?

Dynamic Compilation

• Virtual machines likely to grow in importance– productivity, portability, interoperability, isolation...

• Challenges– when, what, how to parallelize– using hardware performance monitors– using static analyses to aid dynamic compilation– debugging tools for correctness and performance

debugging

Heterogeneous Multi-core Architectures

• Can provide the best performance, cost, power balance

• Challenges– schedule tasks, allocate resources– dynamic core-specific optimization– automatic data layout to prevent conflicts

Questions ?

Leaf Vs. Non-Leaf Performance

Phase Order Space Evaluation – Summary

generategeneratenext next

optimizationoptimizationsequencesequence

last last phase phase active?active?

identicalidenticalfunctionfunction

instance?instance?

equivalentequivalentfunctionfunction

instance?instance?

calculatecalculatefunctionfunction

performanceperformance

simulatesimulateapplicationapplication

seenseencontrol-flowcontrol-flowstructure?structure?

add nodeadd nodeto DAGto DAG

instance?instance?

Predicting Relative Performance – II

4 cycles

10 cycles

4 cycles

15 cycles

26 cycles

15 cycles

90 cycles

2 cycles

44 cycles

10 cycles

Total cycles = 170 Total cycles = ??

10 cycles

Case when No Leaf is Optimal

Computer & Information Sciences - University of Delaware Colloquium / 55 Exhaustive Phase Order...

Documents

B.S. Kulkarni

15699 Exhaustive Search

Depositors Name SHRIPAD KULKARNI & GANGUBAI R. KULKARNI ... · shripad kulkarni & gangubai r. kulkarni shripad r kulkarni & gangubai r kulkarni shripad r kulkarni & gangutai r kulkarni

Dr. Ajit Kulkarni Dr. Ajit Kulkarni M.D. (Hom.), India M.D. (Hom.), India

Dr Ajit Kulkarni

Niraj Kulkarni Leadership Assessment

Work Samples_Charuta Kulkarni

Sandeep kulkarni architect presentation

International Scientific Colloquium Ilmenau University of ... · The discussions during the IWK will doubtless be both wide and deep, exciting and exhaustive, , providing the material,

Ambareesh kulkarni Professional background

1. U.G.ZALTE, EX-DIRECTOR (OPERATIONS) –MSETCL, A.R. KULKARNI. DATE: 13-15 NOVEMBER 2013 CIGRE TUTORIAL AND COLLOQUIUM ON SMARTGRID MYSORE, KARNATAKA

Exhaustive LSMW Tutorial

JBoss World 2009 | Unmesh Kulkarni JBoss World 2009 | Unmesh Kulkarni Migrating SOA to JBoss Enterprise Middleware Unmesh Kulkarni Director of Software Architecture & Delivery COVAD

Kulkarni NCDEVCON2011 CFBuilder Ext

Adhyatma Yoga - Devarao Kulkarni

Exhaustive Signature Algorithm

Dr Sulbha Kulkarni-II

Kulkarni Glacier August27 Revised

Kulkarni CAReport 101609

Discussion lead by:Samrat Kulkarni