Lecture 14 Loop Optimization and Array Analysis

Carnegie Mellon

Lecture 14Loop Optimization and Array Analysis

I. MotivationII. Data dependence analysis

Chapter 11.1-11.1.4, 11.6

Dror E. Maydan CS243: Loop Optimization and Array Analysis 1

Carnegie Mellon

Software Pipelining

for i = 1 to n A[x] = A[y]

• Can I software pipeline the loop?

Dror E. MaydanCS243: Loop Optimization and Array Analysis 2

LD[1]loop N-1 times LD[i+1] ST[i]

ST[n]

Carnegie Mellon

Software Pipelining

for i = 1 to n A[i] = A[i+n]

for i = 1 to n A[2*i] = A[2*i+1]

• Can I software pipeline the loop?



ST[n]

Carnegie Mellon

Software Pipelining

for i = 1 to n A[i] = A[i-1]

for i = 1 to n A[i] = A[i+1]

• Can I software pipeline the loop?– We moved ldi+1 ahead of storei

• In case 1, load of a[i] ahead of store a[i]• In case 2, load of a[i+2] ahead of store a[i]



ST[n]

Carnegie Mellon

Loop Interchange

for i = 1 to n for j = 1 to n

A[j][i] = A[j-1][i-1] * b[i]

• Should I interchange the two loops?– Stride-1 accesses are better for caches– But one more load in the inner loop– But one less register needed to hold the result of the loop


for j = 1 to n for i = 1 to n A[j][i] = A[j-1][i-1] * b[i]

Carnegie Mellon

Loop Interchange

for i = 1 to n for j = 1 to n

A[j][i] = A[j-1][i-1] * b[i]

Iteration Space

Dependence Vectors


j

i

j

i

j

i

Carnegie Mellon

Data Dependences

• Do two array references, nested in some loop nest and perhaps some if statements, refer to overlapping memory locations– As a first step, we check for overlap but don’t care about the nature of

dependence• Type of dependences

– True dependence:a[i] = a[i-1] …

– Anti-dependence:a[i] = a[i+1]

– Output dependencea[i] =a[i] =

Note that we can’t distinguish true from anti by just checking for overlap


Carnegie Mellon

Memory Disambiguation

• In general undecidable at compile time

read(n)for i =

a[i] = a[n]+ 3

if n > 2 if a > 0 if b > 0 if c > 0 x[an] = x[bn + cn]

Can the compiler solve Fermat’s last theorem?


Carnegie Mellon

Affine Array Accesses and Bounds• Common patterns of data accesses: (i, j, k are loop indexes, m is a loop

invariant constant)A[i], A[j], A[i-1], A[0], A[i+j], A[2*i], A[2*i+1]A[i,j], A[i-1, j+1], A[i+m, j]

• Array indexes are affine expressions of surrounding loop indexes and loop invariant constants– Loop indexes: in, in-1, ... , i1– Integer constants: cx, cx-1, ... , c0

– Loop invariant constants: my, my-1, ... , m1

– Array index: cnin + cn-1in-1+ ... + c1i1+ cxmx + cx-1mx-1+ ... + cn+1m1 + c0 – Affine expression: linear expression + a constant term (c0)

• Loop bounds are affine expressions of surrounding loop indexes and loop invariant constants– for i

• for (j=2*i; j<3*i+7; j+=2)


Carnegie Mellon

Single Loop, Single Array DimensionFOR i := 2 to 5 do A[i-2] = A[i]+1;

• Between read access A[i] and write access A[i-2]there is a dependence if:– there exist two iterations ir and iw within the loop bounds, s.t.

– iterations ir & iw read & write the same array element, respectively

integers iw, ir 2 iw, ir 5 ir = iw - 2

• Between write access A[i-2] and write access A[i-2] there is a dependence if: integers iw, iv 2 iw, iv 5 iw – 2 = iv - 2

– To rule out the case when the same instance of the same reference depends on itself: • add constraint iw iv

• For now, just asking if there is overlap, not the nature of the overlap


Carnegie Mellon

Multiple dimensions, Nested Loops• Only use loop bounds and array indexes that are affine functions of loop

variables and loop invariant constantsfor i = 1 to n for j = 2*i to 100 a[i+2*j+3][4*i+2*j][i*i] = … … = a[1][2*i+1][j]

• Assume a data dependence between the read & write operation if there exists: – a read instance with indexes ir, jr and– a write instance with indexes iw, jw

integers ir, jr, iw, jw 1 iw, ir n 2iw jw 100 2ir jr 100 iw + 2jw + 3 = 1 4iw + 2jw = 2ir + 1• Equate each dimension of array access; ignore non-affine ones

– No solution No data dependence– Solution there may be a dependence


Carnegie Mellon

General Formulation of Data Dependence Analysis

• Equivalent to integer linear programming

integer i A1i b1 A2i = b2

• Integer linear programming is NP-hard – O(nn) where n is the number of variables


For every pair of accesses not necessarily distinct (F1 , f1 ) and (F2, f2) one must be a write operationLet B1i1+b1 0, B2i2+b2 0 be the corresponding loop bound constraints,

integers i1, i2 B1i1 + b1 0, B2i2 + b2 0 F1 i1+ f1 = F2 i2+f2

If the accesses are not distinct, then add the constraint i1 i2

Carnegie Mellon

Data Dependence Analysis Algorithm • Typically solving many tiny, repeated problems

– Integer linear programming packages optimize for large problems

– Use memoization to remember the results of simple tests

• Apply a series of relatively simple tests– Extended GCD, …,

• Backed up by a more expensive algorithm– Fourier-Motzkin

Carnegie Mellon

GCD Test• Ignore Bounds

Integer vector i A1i = b1 • Single equation

anin + an-1in-1+ ... + a1i1 = b

There exists an integer solution iff the gcd(an, an-1, …) divides b

Euclid’s Algorithm (around 300 BC)• Assume a and b are positive integers, and a > b.• Let c be the remainder of a/b.

– If c=0, then gcd(a,b) = b. – Otherwise, gcd(a,b) = gcd(b,c).

• gcd(a1, a2, …, an) = gcd(gcd(a1, a2), a3 …, an)

Carnegie Mellon

GCD Test• Multiple equations: Ai = b

x – 2y + z = 03x + 2y + z =5

x = 2y –z; sub into second equation6y-3z+2y+z=5, 8y – 2z = 5

Replace 5 with 48y-2z = 4Many solutions but simplified system of equations

Carnegie Mellon

GCD Test• Multiple equations: Ai = b

2x – 2y + z = 03x + 2y + z =5

Lcm of 2 and 3 is 66x – 6y +3z = 06x + 4y + 2z = 10

6x = 6y – 3z, sub into second equation6y – 3z + 4y + 2z = 1010y –z = 10

Carnegie Mellon

After GCD Test

integer i A1i b1

• Where i is a reduced set of variables subject to the bounds constraints– Originally say we had an n-dimensional array nested m deep– N equality constraints, 2m variables– Assuming equality constraints are independendent, each

equality constraint eliminates one variable– 2m – N variables


Carnegie Mellon

Fourier-Motzkin Elimination• To eliminate a variable from a set of linear inequalities.• To eliminate a variable x1

– Rewrite all expressions in terms of lower or upper bounds of Cx1– Create a transitive constraint for each pair of lower and upper

bounds. • Example: Let L, U be lower bounds and upper bounds resp

– To eliminate x1:L1(x2, …, xn) Cx1 U1 (x2, …, xn)L2(x2, …, xn) Cx1 U2 (x2, …, xn)

L1(x2, …, xn) U1 (x2, …, xn)L1(x2, …, xn) U2 (x2, …, xn)L2(x2, …, xn) U1 (x2, …, xn)L2(x2, …, xn) U2 (x2, …, xn)

If at any point, you get an inconsistent equation; e.g. 3 < 2, independent

Otherwise there exists a real, but not necessarily integral, solution

Carnegie Mellon

Fourier-Motzkin Elimination• One variable and a series of bounds

– Does the range contain an integer• No, no integral solution• Yes, pick an integer in the middle of the range, and go back to the equation with

two variables• More than one variable and a series of bounds

– Does the range contain an integer• No, might be an integral solution. Branch and bound

– x = 3.5. Add two constraints, x <= 3 and x >= 4. Start over• Yes, pick an integer in the middle of the range, and go back to the equation with

one more variable

Carnegie Mellon

Example

FOR i = 1 to 5FOR j = i+1 to 5

A[i,j] = f(A[i,i], A[i-1,j])


1 i i 5i + 1 j j 5

1 i’ i’ 5i’ + 1 j’ j’ 5

A[i,j] vrs A[i’, i’]i = i’, j=i’

Bounds on read: 1 <= i’; i’<=5; i’+1 <= j’; j’<=5;

Bounds on write: 1 <=i; i <= 5;Sub in i=i’ 1 <= i’; i’<=5; // redundantBounds on write: i+1 <= j; j<=5;Sub in i=i’ and j=i’ i’+1 <= i’; i’<=5;i’+1 <= i’ equivalent to 1 <= 0 // inconsistent so independent

Carnegie Mellon

Example

FOR i = 1 to 5FOR j = i+1 to 5

A[i,j] = f(A[i,i], A[i-1,j])


1 i i 5i + 1 j j 5

1 i’ i’ 5i’ + 1 j’ j’ 5

A[i,j] vrs A[i’-1, j’]i’ = i+1, j’=j

1. 1 i; i 5; // i bounds2. i+1 j; j 5; // j bounds3. 1 i+1; i+1 5; // i’ bounds, equivalent to 0 i; i 44. i+2 j; j 5; // j’ bounds

1 i 4; // combine 1 and 3 i+2 j; j 5; // 4i+2 5, // First step of Fourier-Motzkin on 4, equivalent to i<=3

Pick i=2, j=4, i’ = 3, j’ = 4 A[2,4] = A[2,4]

Carnegie Mellon

Distance Vectors

for i = 1 to n for j = 1 to n A[j][i] = A[j-1][i-1] * b[i]

jw = jr-1, iw = ir-1jr – jw = 1, ir – iw = 1

We say there is a dependence from the write to the read of distance (1, 1)

How do we compute distancesGCD Test: look for normalized equations during gcd test i’ - i = cMore sophisticated: Add constraint i’-i = d to Fourier-Motzkin and see if it is constrained to a single value


j

i

Carnegie Mellon

Direction Vectorsfor i = 1 to n

for j = 1 to iA[i] = A[j] * b[i]

jr=iwjw <= iwjw <= jrjr-jw >= 0jr <= iriw <= irir-iw >= 0

We say there is a dependence from the write to the read of direction (>=, >=)All this means is that the two references may refer to the same location on all iterations when jr >= jw and ir >= iw


Carnegie Mellon

Direction Vectors

We say there is a dependence from the write to the read of direction (>=, >=)All this means is that the two references may refer to the same location on all i iterations and when jr >= jw and ir >= iw

If (ir, jr) –(iw, jw) = some vector d (or a set of vectors), clearly (iw,jw)-(ir, jr) = -d (for all d)So we don’t run dependence analysis twice, once for the write versus the read and once for the read versus the write

Instead we run it once and negate the result to get the opposite edges

Dependence vectors are a relation between two iterations of two references. We choose to draw the dependence from the reference with the earlier iteration to the latter

Why?We have a property that a location is written by the source of an edge before the sinkWe don’t want to change the order in which we access any memory location

As long as after transformation, the edges are still positive, the transformation is legal


Carnegie Mellon

Lexicographically Positive Direction Vectors• Represent dependences as lexicographically positive

– First non-equal component must be “>”. – All “=“ vectors only allowed if source precedes sink in same

iteration

• There was a dependence from the write to the read of direction (>=, >=)– In a given iteration, the read happens before the write

– Dependence from write to read of the positive components of (>=, >=)• (>, >=), (=, >)• Don’t include (=, =) because the read happens before the write

– Dependence from read to write of positive components of (<=, <=)• (=, =)

D.E. MaydanCS243: Loop Optimization and Array Analysis 25

Carnegie Mellon

Direction VectorsComputing Direction Vectors

Start with nothing (*, *, …)Independent: doneDependent

check (>, *, *, …), (=, *, *, …) and (<, *, *, …)Add in a constraint i’-i > 0, etc

If, for example, (<, *, *, …) is dependentcheck (<, >, *, …), (<, =, *, …) and (<, < , *, …)

Exponential blow-upDon’t do it for constant distancesUnused variables are automatically *

for i for j a[j] = a[j+1] .


Documents

Lecture 14 Loop Optimization and Array Analysis