Dependence Testing Optimizing Compilers for Modern Architectures, Chapter 3 Allen and Kennedy Presented by Rachel Tzoref and Rotem Oshman

DO i 1 = L 1, U 1 DO i 2 = L 2, U 2... DO i n = L n, U n S 1 A(f 1 (i 1,...,i n ),...,f m (i 1,...,i n )) =... S 2... = A(g 1 (i 1,...,i n ),...,g m (i 1,...,i n )) ENDDO... ENDDO The General Problem Dependence exists, 9 iteration vectors  such that f j (  )=g j (  ) for all j,1· j · m

Basic Definitions Index: A loop variable Subscript: The pair of expressions that appear in a certain coordinate in a pair of array references A(i,j,m) = A(i,k,j) + C ‹i,i› is the first subscript ‹j,k› is the second subscript ‹m,j› is the third subscript

Distance and Direction Vectors - Reminder DO I=1, N DO J=1, M DO K=1, L A(I+1,J,K-1) = A(I,J,K)+10 ENDDO Distance Vector: (1, 0, -1) Direction Vector: ( )

What can we do? INPUT N; DO i = 1, 100 DO j = 1, 100 A(N+(i+j) 3 ) = A(i (j+N) +(j i )+(j 2 +i 2 )) ENDDO The general case – the problem is undecidable during compile time

What can we do? Most subscripts are polynomials DO i = 1, 100 DO j = 1, 100 DO k = 1, 100 A(i 2 +j 2,4¢k 3 ) = A(k 2,i 4 ) ENDDO Problem reduced to solving a system of Diophantine equations Too complex: consider only linear subscript expressions

What can we do? DO i = 1, 100 DO j = 1, 100 DO k = 1, 100 A(i+j,4¢k) = A(k,i) ENDDO Finding integer solutions to a system of linear Diophantine Equations is NP-Complete Testing may have to be imprecise

Our Goal - Conservative Testing Result = “no dependence” ) no dependence Result = “dependence” + direction vector(s) ) dependence may or may not exist Code is always correct, but may be less than optimal Directions output by testing Directions where dependence exists

What About Nonlinear Subscripts? Can’t analyze something ) assume dependence Example: DO i = 1, 50 DO j = 51, 100 A(i 3 +j 2,i) = A(j 4,j) ENDDO May still prove independence or limit possible direction vectors

A(5,i+1,j) = A(1,i,k) + C ZIV Subscript: no indices SIV Subscript: one index MIV Subscript: more than one index Definition: Subscript Complexity

Definition: Separability A subscript is separable if its indices do not occur in other subscripts A(i+1,j) = A(k,j) + C Both subscripts are separable If two different subscripts contain the same index they are coupled A(i,j,j) = A(i+1,j,k) + C Second and third subscripts are coupled

Coupled Subscript Groups Why are they important? DO i = 1, 100 S 1 A(i+1,i) = B(i) + C S 2 D(i) = A(i,i) ¢ E ENDDO Coupling can cause imprecision in dependence testing

A(j-1,i+1,i+3,k) = A(j+2,j,i,k+1) Building Coupled Groups Test each group separately k i+1 j j-1 i+3 i j+2 k+1

Subscript Tests Input: Separable subscript OR coupled group Loop bounds Output: “No dependence”, OR A set of direction vectors in which dependence may exist

Merging Direction Vectors DO i = 1, 100 DO j = 1, 99 A(i+1,j) = B(i)+C D(j) = A(i,100-j) ENDDO Cartesian product: Direction Vectors:

General Algorithm Overview Input: a pair of array references and loop bounds. 1. Partition subscripts into separable and coupled groups. 2. Classify each subscript as ZIV, SIV or MIV. 3. For each separable subscript: apply single subscript test according to its complexity. If independence established: output “no dependence” and halt. 4. For each coupled group: apply multiple subscript test. If independence established: output “no dependence” and halt. 5. Merge all direction vectors computed in the previous steps into a single set of direction vectors.

Roadmap: Part I Single Subscript Tests: ZIV Strong SIV Weak SIV Weak-Zero SIV Weak-Crossing SIV SIV Tests in Complex Iteration Spaces

Roadmap: Part II MIV Tests: GCD Test Banerjee Inequality Trapezoidal Banerjee Inequality Coupled Subscripts: the Delta Test Empirical Results

Roadmap: Part I Single Subscript Tests: ZIV Strong SIV Weak SIV Weak-Zero SIV Weak-Crossing SIV SIV Tests in Complex Iteration Spaces

ZIV Test DO j = 1, 100 A(e 1 ) = A(e 2 ) + B(j) ENDDO e 1,e 2 are loop invariant expressions e 1 =5, e 2 =4 ) no dependence e 1 =L, e 2 =L+1 ) no dependence e 1 =L, e 2 =K ) dependence If (e 1 -e 2 ) is a non-zero constant then no dependence exists

INPUT L,K DO i = 1, N S 1 B(i+1) = A(L+K) + X(i) S 2 A(L) = B(i) + C ENDDO The values of L and K are unknown If K  0 there is no dependence from S 2 to S 1 K  0 is called the Breaking Condition ZIV Test & Breaking Conditions

IF (K  0) THEN DO i = 1, N B(i+1) = A(L+K) + X(i) ENDDO DO i = 1, N A(L) = B(i) + C ENDDO ELSE DO i = 1, N B(i+1) = A(L+K) + X(i) A(L) = B(i) + C ENDDO ENDIF ZIV Test & Breaking Conditions

IF (K  0) THEN B(2:N+1) = A(L+K) + X(1:N) DO i = 1, N A(L) = B(i) + C ENDDO ELSE DO i = 1, N B(i+1) = A(L+K) + X(i) A(L) = B(i) + C ENDDO ENDIF ZIV Test & Breaking Conditions

Roadmap: Part I Single Subscript Tests: ZIV Strong SIV Weak SIV Weak-Zero SIV Weak-Crossing SIV SIV Tests in Complex Iteration Spaces

Strong SIV Examples:

Strong SIV Test Dependence exists if d is an integer and A(m 1 )

Strong SIV Test DO i = 1, N A(i+2¢N) = A(i+N) + C ENDDO N > N-1 ) No dependence

Strong SIV & Breaking Conditions DO i = 1, L A(i+N) = A(i) + B ENDDO N>L-1 is the breaking condition IF (N¸L) THEN A(N+1:N+L) = A(1:L) + B ELSE DO i = 1, L A(i+N) = A(i) + B ENDDO ENDIF

Roadmap: Part I Single Subscript Tests: ZIV Strong SIV Weak SIV Weak-Zero SIV Weak-Crossing SIV SIV Tests in Complex Iteration Spaces

Weak SIV Examples: Dependence exists for all integer solutions to the following linear Diophantine equation:

Roadmap: Part I Single Subscript Tests: ZIV Strong SIV Weak SIV Weak-Zero SIV Weak-Crossing SIV SIV Tests in Complex Iteration Spaces

Weak-Zero SIV Test Dependence equation (assume a 2 =0): Dependence exists if is an integer and is within the loop bounds

Weak-Zero SIV Test Dependence always caused by a particular iteration

DO i = 1, N A(i,N) = A(1,N) + A(N,N) ENDDO Dependence caused by iterations 1 and N. These iterations can be peeled from the loop: A(1,N) = A(1,N) + A(N,N) DO i = 2, N-1 A(i,N) = A(1,N) + A(N,N) ENDDO A(N,N) = A(1,N) + A(N,N) Weak-Zero SIV & Loop Peeling

DO i = 1, N A(i,N) = A(1,N) + A(N,N) ENDDO Dependence caused by iterations 1 and N. These iterations can be peeled from the loop: A(1,N) = A(1,N) + A(N,N) A(N,N) = A(1,N) + A(N,N) Weak-Zero SIV & Loop Peeling A(2:N-1) = A(1,N) + A(N,N)

Roadmap: Part I Single Subscript Tests: ZIV Strong SIV Weak SIV Weak-Zero SIV Weak-Crossing SIV SIV Tests in Complex Iteration Spaces

Weak-Crossing SIV Line of symmetry A(m 1 ) A(m 2 ) A(m 3 ) When does dependence exist?

Weak-Crossing SIV Test Line of symmetry Dependence exists if the line of symmetry is: within the loop bounds ) L · (c 2 -c 1 )/2a · U A(m 1 )

Weak-Crossing SIV Test Line of symmetry A(m 1 ) Dependence exists if the line of symmetry is: within the loop bounds ) L · (c 2 -c 1 )/2a · U an integer or has a non-integer part equal to ½ ) The line of symmetry is halfway between two integers 

Weak-Crossing SIV & Loop Splitting DO i = 1, N A(i) = A(N-i+1) + C ENDDO Line of symmetry: i=(N+1)/2 DO i = 1, (N+1)/2 A(i) = A(N-i+1) + C ENDDO DO i = (N+1)/2+1, N A(i) = A(N-i+1) + C ENDDO

DO i = 1, N A(i) = A(N-i+1) + C ENDDO Line of symmetry: i=(N+1)/2 Weak-Crossing SIV & Loop Splitting A(1:(N+1)/2) = A(N:(N+1)/2)+C A((N+1)/2+1:N) = A((N+1)/2-1:1)+C

Roadmap: Part I Single Subscript Tests: ZIV Strong SIV Weak SIV Weak-Zero SIV Weak-Crossing SIV SIV Tests in Complex Iteration Spaces

Complex Iteration Spaces Triangular: One of the loop bounds is a function of at least one other loop index DO i = 1, 5 DO j = 1, i

DO i = 1, 5 DO j = i-1, 2¢i-1 Complex Iteration Spaces Trapezoidal: Both loop bounds are functions of at least one other loop index

Complex Iteration Spaces SIV tests can be extended to such loops, with some loss of precision

Strong SIV in Complex Iteration Spaces DO i = 1, N DO j = L 0 +L 1 ¢i, U 0 +U 1 ¢i S 1 A(j+d) = … S 2 … = A(j) + B ENDDO No dependence if ) Unless inequality holds for all values of i in the loop ) assume dependence

Index Set Splitting DO i = 1, 100 DO j = 1, i A(j+20) = A(j) + B ENDDO For there is no dependence

Use i<21 as a breaking condition DO i = 1, 20 DO j = 1, i A(j+20) = A(j) + B ENDDO DO i = 21, 100 DO j = 1, i A(j+20) = A(j) + B ENDDO Index Set Splitting

Use i<21 as a breaking condition DO i = 21, 100 DO j = 1, i A(j+20) = A(j) + B ENDDO Index Set Splitting PARALLEL DO i = 1, 20 A(21:20+i) = A(1:i) + B ENDDO

Weak-Zero SIV in Complex Iteration Spaces DO i = 1, N DO j = L 0 +L 1 ¢i, U 0 +U 1 ¢i S 1 A(c) = … S 2 … = A(j) + B ENDDO No dependence if ) Unless inequality holds for all values of i in the loop ) assume dependence

Coupling in Complex Iteration Spaces DO i = 1, 100 DO j = 1, i A(j+20,i) = A(j,19) + B ENDDO Testing each subscript separately causes imprecision : First subscript: dependence only for i ¸ 21 Second subscript: dependence only for i=19 Subscripts are actually coupled