1. Dependence Testing Optimizing Compilers for Modern Architectures, Chapter 3 Allen and Kennedy Presented by Rachel Tzoref and Rotem Oshman

2. DO i 1 = L 1, U 1 DO i 2 = L 2, U 2... DO i n = L n, U n S 1 A(f 1 (i 1,...,i n ),...,f m (i 1,...,i n )) =... S 2... = A(g 1 (i 1,...,i n ),...,g m (i 1,...,i n )) ENDDO... ENDDO The General Problem Dependence exists, 9 iteration vectors  such that f j (  )=g j (  ) for all j,1· j · m

3. Basic Definitions Index: A loop variable Subscript: The pair of expressions that appear in a certain coordinate in a pair of array references A(i,j,m) = A(i,k,j) + C ‹i,i› is the first subscript ‹j,k› is the second subscript ‹m,j› is the third subscript

4. Distance and Direction Vectors - Reminder DO I=1, N DO J=1, M DO K=1, L A(I+1,J,K-1) = A(I,J,K)+10 ENDDO Distance Vector: (1, 0, -1) Direction Vector: ( )

5. What can we do? INPUT N; DO i = 1, 100 DO j = 1, 100 A(N+(i+j) 3 ) = A(i (j+N) +(j i )+(j 2 +i 2 )) ENDDO The general case – the problem is undecidable during compile time

6. What can we do? Most subscripts are polynomials DO i = 1, 100 DO j = 1, 100 DO k = 1, 100 A(i 2 +j 2,4¢k 3 ) = A(k 2,i 4 ) ENDDO Problem reduced to solving a system of Diophantine equations Too complex: consider only linear subscript expressions

7. What can we do? DO i = 1, 100 DO j = 1, 100 DO k = 1, 100 A(i+j,4¢k) = A(k,i) ENDDO Finding integer solutions to a system of linear Diophantine Equations is NP-Complete Testing may have to be imprecise

8. Our Goal - Conservative Testing Result = “no dependence” ) no dependence Result = “dependence” + direction vector(s) ) dependence may or may not exist Code is always correct, but may be less than optimal Directions output by testing Directions where dependence exists

9. What About Nonlinear Subscripts? Can’t analyze something ) assume dependence Example: DO i = 1, 50 DO j = 51, 100 A(i 3 +j 2,i) = A(j 4,j) ENDDO May still prove independence or limit possible direction vectors

10. A(5,i+1,j) = A(1,i,k) + C ZIV Subscript: no indices SIV Subscript: one index MIV Subscript: more than one index Definition: Subscript Complexity

11. Definition: Separability A subscript is separable if its indices do not occur in other subscripts A(i+1,j) = A(k,j) + C Both subscripts are separable If two different subscripts contain the same index they are coupled A(i,j,j) = A(i+1,j,k) + C Second and third subscripts are coupled

12. Coupled Subscript Groups Why are they important? DO i = 1, 100 S 1 A(i+1,i) = B(i) + C S 2 D(i) = A(i,i) ¢ E ENDDO Coupling can cause imprecision in dependence testing

13. A(j-1,i+1,i+3,k) = A(j+2,j,i,k+1) Building Coupled Groups Test each group separately k i+1 j j-1 i+3 i j+2 k+1

14. Subscript Tests Input: Separable subscript OR coupled group Loop bounds Output: “No dependence”, OR A set of direction vectors in which dependence may exist

15. Merging Direction Vectors DO i = 1, 100 DO j = 1, 99 A(i+1,j) = B(i)+C D(j) = A(i,100-j) ENDDO Cartesian product: Direction Vectors:

16. General Algorithm Overview Input: a pair of array references and loop bounds. 1. Partition subscripts into separable and coupled groups. 2. Classify each subscript as ZIV, SIV or MIV. 3. For each separable subscript: apply single subscript test according to its complexity. If independence established: output “no dependence” and halt. 4. For each coupled group: apply multiple subscript test. If independence established: output “no dependence” and halt. 5. Merge all direction vectors computed in the previous steps into a single set of direction vectors.

17. Roadmap: Part I Single Subscript Tests: ZIV Strong SIV Weak SIV Weak-Zero SIV Weak-Crossing SIV SIV Tests in Complex Iteration Spaces

18. Roadmap: Part II MIV Tests: GCD Test Banerjee Inequality Trapezoidal Banerjee Inequality Coupled Subscripts: the Delta Test Empirical Results

19. Roadmap: Part I Single Subscript Tests: ZIV Strong SIV Weak SIV Weak-Zero SIV Weak-Crossing SIV SIV Tests in Complex Iteration Spaces

20. ZIV Test DO j = 1, 100 A(e 1 ) = A(e 2 ) + B(j) ENDDO e 1,e 2 are loop invariant expressions e 1 =5, e 2 =4 ) no dependence e 1 =L, e 2 =L+1 ) no dependence e 1 =L, e 2 =K ) dependence If (e 1 -e 2 ) is a non-zero constant then no dependence exists

21. INPUT L,K DO i = 1, N S 1 B(i+1) = A(L+K) + X(i) S 2 A(L) = B(i) + C ENDDO The values of L and K are unknown If K  0 there is no dependence from S 2 to S 1 K  0 is called the Breaking Condition ZIV Test & Breaking Conditions

22. IF (K  0) THEN DO i = 1, N B(i+1) = A(L+K) + X(i) ENDDO DO i = 1, N A(L) = B(i) + C ENDDO ELSE DO i = 1, N B(i+1) = A(L+K) + X(i) A(L) = B(i) + C ENDDO ENDIF ZIV Test & Breaking Conditions

23. IF (K  0) THEN B(2:N+1) = A(L+K) + X(1:N) DO i = 1, N A(L) = B(i) + C ENDDO ELSE DO i = 1, N B(i+1) = A(L+K) + X(i) A(L) = B(i) + C ENDDO ENDIF ZIV Test & Breaking Conditions

24. Roadmap: Part I Single Subscript Tests: ZIV Strong SIV Weak SIV Weak-Zero SIV Weak-Crossing SIV SIV Tests in Complex Iteration Spaces

25. Strong SIV Examples:

26. Strong SIV Test Dependence exists if d is an integer and A(m 1 )

27. Strong SIV Test DO i = 1, N A(i+2¢N) = A(i+N) + C ENDDO N > N-1 ) No dependence

28. Strong SIV & Breaking Conditions DO i = 1, L A(i+N) = A(i) + B ENDDO N>L-1 is the breaking condition IF (N¸L) THEN A(N+1:N+L) = A(1:L) + B ELSE DO i = 1, L A(i+N) = A(i) + B ENDDO ENDIF

29. Roadmap: Part I Single Subscript Tests: ZIV Strong SIV Weak SIV Weak-Zero SIV Weak-Crossing SIV SIV Tests in Complex Iteration Spaces

30. Weak SIV Examples: Dependence exists for all integer solutions to the following linear Diophantine equation:

31. Roadmap: Part I Single Subscript Tests: ZIV Strong SIV Weak SIV Weak-Zero SIV Weak-Crossing SIV SIV Tests in Complex Iteration Spaces

32. Weak-Zero SIV Test Dependence equation (assume a 2 =0): Dependence exists if is an integer and is within the loop bounds

33. Weak-Zero SIV Test Dependence always caused by a particular iteration

34. DO i = 1, N A(i,N) = A(1,N) + A(N,N) ENDDO Dependence caused by iterations 1 and N. These iterations can be peeled from the loop: A(1,N) = A(1,N) + A(N,N) DO i = 2, N-1 A(i,N) = A(1,N) + A(N,N) ENDDO A(N,N) = A(1,N) + A(N,N) Weak-Zero SIV & Loop Peeling

35. DO i = 1, N A(i,N) = A(1,N) + A(N,N) ENDDO Dependence caused by iterations 1 and N. These iterations can be peeled from the loop: A(1,N) = A(1,N) + A(N,N) A(N,N) = A(1,N) + A(N,N) Weak-Zero SIV & Loop Peeling A(2:N-1) = A(1,N) + A(N,N)

36. Roadmap: Part I Single Subscript Tests: ZIV Strong SIV Weak SIV Weak-Zero SIV Weak-Crossing SIV SIV Tests in Complex Iteration Spaces

37. Weak-Crossing SIV Line of symmetry A(m 1 ) A(m 2 ) A(m 3 ) When does dependence exist?

38. Weak-Crossing SIV Test Line of symmetry Dependence exists if the line of symmetry is: within the loop bounds ) L · (c 2 -c 1 )/2a · U A(m 1 )

39. Weak-Crossing SIV Test Line of symmetry A(m 1 ) Dependence exists if the line of symmetry is: within the loop bounds ) L · (c 2 -c 1 )/2a · U an integer or has a non-integer part equal to ½ ) The line of symmetry is halfway between two integers 

40. Weak-Crossing SIV & Loop Splitting DO i = 1, N A(i) = A(N-i+1) + C ENDDO Line of symmetry: i=(N+1)/2 DO i = 1, (N+1)/2 A(i) = A(N-i+1) + C ENDDO DO i = (N+1)/2+1, N A(i) = A(N-i+1) + C ENDDO

41. DO i = 1, N A(i) = A(N-i+1) + C ENDDO Line of symmetry: i=(N+1)/2 Weak-Crossing SIV & Loop Splitting A(1:(N+1)/2) = A(N:(N+1)/2)+C A((N+1)/2+1:N) = A((N+1)/2-1:1)+C

42. Roadmap: Part I Single Subscript Tests: ZIV Strong SIV Weak SIV Weak-Zero SIV Weak-Crossing SIV SIV Tests in Complex Iteration Spaces

43. Complex Iteration Spaces Triangular: One of the loop bounds is a function of at least one other loop index 1 1 2 2 5 DO i = 1, 5 DO j = 1, i 345 4 3

44. DO i = 1, 5 DO j = i-1, 2¢i-1 Complex Iteration Spaces Trapezoidal: Both loop bounds are functions of at least one other loop index 1 1 2 2 5 7 34 5 4 3 6 8

45. Complex Iteration Spaces SIV tests can be extended to such loops, with some loss of precision

46. Strong SIV in Complex Iteration Spaces DO i = 1, N DO j = L 0 +L 1 ¢i, U 0 +U 1 ¢i S 1 A(j+d) = … S 2 … = A(j) + B ENDDO No dependence if ) Unless inequality holds for all values of i in the loop ) assume dependence

47. Index Set Splitting DO i = 1, 100 DO j = 1, i A(j+20) = A(j) + B ENDDO For there is no dependence

48. Use i<21 as a breaking condition DO i = 1, 20 DO j = 1, i A(j+20) = A(j) + B ENDDO DO i = 21, 100 DO j = 1, i A(j+20) = A(j) + B ENDDO Index Set Splitting

49. Use i<21 as a breaking condition DO i = 21, 100 DO j = 1, i A(j+20) = A(j) + B ENDDO Index Set Splitting PARALLEL DO i = 1, 20 A(21:20+i) = A(1:i) + B ENDDO

50. Weak-Zero SIV in Complex Iteration Spaces DO i = 1, N DO j = L 0 +L 1 ¢i, U 0 +U 1 ¢i S 1 A(c) = … S 2 … = A(j) + B ENDDO No dependence if ) Unless inequality holds for all values of i in the loop ) assume dependence

51. Coupling in Complex Iteration Spaces DO i = 1, 100 DO j = 1, i A(j+20,i) = A(j,19) + B ENDDO Testing each subscript separately causes imprecision : First subscript: dependence only for i ¸ 21 Second subscript: dependence only for i=19 Subscripts are actually coupled