1. Implementation of Morton Layout for Large Arrays Presented by: Sharad Ratna Bajracharya Advisor: Prof. Larry Dunning 23 rd April 2004 Bowling Green State University

2. Outline Introduction Objectives Implementation Samples Improvement Recommendation Conclusion

3. Introduction Morton Layout is used in two dimensional array. Performance of Morton Layout is comparatively better than row-major or column-major array representation.

4. Introduction Introduction continues... Reports on analysis of the Morton Layout for the performance and efficiency : –An exhaustive evaluation of row-major, column-major and Morton Layouts for large two-dimensional arrays; Jeyarajan Thiyagalingam, Olav Beckman, Paul H. J. Kelly. –Is Morton Layout competitive for large two-dimensional arrays?; Jeyarajan Thiyagalingam and Paul H. J. Kelly. –Improving the Performance of Morton Layout by Array Alignment and Loop Unrolling; Jeyarajan Thiyagalingam, Olav Beckman, Paul H. J. Kelly.

5. Introduction Introduction continues... General Row Major Array Representation –Row major ordering assigns successive elements, moving across the rows and then down the columns, to successive memory locations. 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

6. Introduction Introduction continues... Column Major array representation. 0 4 8 12 1 5 9 13 2 6 10 14 3 7 11 15

7. Introduction Introduction continues... Morton layout is a compromise storage layout between the programming language mandated layouts such as row-major and column-major. 0 1 2 3 0 1 4 5 4 5 6 7 2 3 6 7 8 9 10 11 8 9 12 13 12 13 14 1510 11 14 15 (Row Major)(Morton Storage Layout)

8. Introduction Introduction continues... Morton storage layout works with almost equal overhead whether traversed row- wise or column-wise. Morton layout works fine with square two dimensional array, which size is power of 2 such as 2x2, 4x4, 8x8 etc.

9. Introduction Introduction continues... For non-square matrix, it waste lots of memory spaces. 0 1 2 30 1 4 5 4 5 6 72 3 6 7 8 9 10 118 9 X X 10 11 (Row Major)(Morton Storage Layout)

10. Introduction Introduction continues... How Morton Layout Works? –For any subscript of 2 dimensional array such as array[ 2, 3 ] Binary value of row 2 -> 1 0 Binary value of col 3 -> 1 1 Morton Layout stores at 1 1 0 1 location, i.e. 13th memory location. –Also known as Zip Fastening Array Layout.

11. Introduction Introduction continues... Consider row major large array 12 3 4 5 6 7 …………………….1000 1001100210031004100510061007 ………………...2000 20012002……………………………………………………………………………………... 9001900290039004900590069007 ………………10000........ Result is cache miss, page faults and poor performance.

12. Objectives Improve cache miss and page fault characteristics in Large Array using Morton Array Layouts. Reduce wasted memory in Morton layout. Improvement in extendibility of arrays.

13. Implementation Interleaved bit patterns: 4 -> 0 1 0 0 -> 0 0 1 0 0 0 0 9 -> 1 0 0 1 -> 1 0 0 0 0 0 1 15 -> 1 1 1 1 -> 1 0 1 0 1 0 1 (Interleaved Bits)

14. Implementation Implementation continues Bit interleaved increment and decrement: –Bit interleaved increment: 101 + 1 -> 1 0 0 0 1 + 1 110 -> 1 0 1 0 0 (Changes are in interleaved bits) –For any value “a”, bit interleaved increment is given by: a+1 = ((a | 0xAAAAAAAA) + 1 ) & 0x55555555 0xAAAAAAAA=1010……..10101010 (32 bits) 0x55555555 = 0101…….01010101 (32 bits)

15. Implementation Implementation continues –Bit interleaved increment… a+1 = ((a | 0xAAAAAAAA) + 1) & 0x55555555 0 0 0 1-> Bit interleaved 1 (0 1) OR1 0 1 0 1 0 1 1 +1 1 1 0 0 AND0 1 0 1 0 1 0 0-> Bit interleaved 2 (1 0)

16. Implementation Implementation continues –More examples of bit interleaved increment: 0 0 0 0 0 + 1 = 0 0 0 0 1 0 0 0 0 1 + 1 = 0 0 1 0 1 0 0 1 0 1 + 1 = 1 0 0 0 0 1 0 0 0 0 + 1 = 1 0 0 0 1 1 0 0 0 1 + 1…

17. Implementation Implementation continues –Bit interleaved Decrement: For example, 1 0 0 - 1 -> 1 0 0 0 0 - 1 1 1 -> 0 0 1 0 1 (Changes are in interleaved bits) –For any value “a”, bit interleaved decrement is given by: a-1 = (a - 1 ) & 0x55555555 Where, 0x55555555 = 0101……01010101 (32 bits)

18. Implementation Implementation continues –Bit interleaved decrement… a-1 = (a -1) & 0x55555555 0 1 0 0 0 0-> Bit interleaved 4 (100) - 1 0 0 1 1 1 1 AND0 1 0 1 0 1 0 0 0 1 0 1-> Bit interleaved 3 (11)

19. Implementation Implementation continues –More examples of bit interleaved decrement: …………... 1 0 0 0 0 - 1 = 0 0 1 0 1 0 0 1 0 1 - 1 = 0 0 1 0 0 0 0 1 0 0 - 1 = 0 0 0 0 1 0 0 0 0 1 - 1 = 0 0 0 0 0

20. Implementation Implementation continues Morton Layout Array representation can be implemented in two ways: –First method is by maintaining lookup table of bit interleaved array subscript for address calculation. For example, 0 -> 0 0 0 0 1 -> 0 0 0 1 2 -> 0 1 0 0 3 -> 0 1 0 1

21. Implementation Implementation continues –For example, any array subscript viz. [ 2, 3 ] Value of 2 (1 0 ) from lookuptable -> 0100 Value of 3 ( 1 1) from lookuptable -> 0101 To get the Morton layout address, ROW bitwise shift 1 + COL 0100<<1 + 0101 1000+0101, that is, 1 0 0 0 + 0 1 0 1 1 1 0 1 (zipped address)

22. Implementation Implementation continues –Second Method to implement Morton Array Layout Representation is by only using bit interleaved increment and decrement without lookuptable.

23. Implementation Implementation continues Implemented in C++ as two dimensional array matrix class with Standard Template Library (STL) compatibility so as to make it generic, that is, it is not tied to any particular data structure or object type. Internally data are stored in STL vector sequentially.

24. Implementation Implementation continues Direct accessing the element of array matrix by using array subscript is implemented using lookup table. Random Iterators are defined which make use of bit interleaved increment and decrement without using lookup table. –Iterators are generalization of pointers. They are objects that point to other objects.

25. Implementation Implementation continues Different types of random iterators are implemented to provide the flexibility in using the matrix class, such as, –Row Major iterator –Column Major iterator –Diagonal iterator –Row iterator / Super row iterator –Column iterator / Super column iterator –Reverse Row Major iterator

26. Samples Using Row Major Iterator: Sorted Data: -9 -9 -8 -8 -8 -8 -7 -6 -6 -5 -4 -4 -2 -2 -2 -1 1 1 2 3 5 5 6 7 Original Data: 6 -9 -8 -1 -8 -6 -9 -2 -2 -5 -6 -4 2 3 -4 -8 -2 1 -7 5 5 -8 1 7 //Row Major sorting using STL Sort() mat1=matori; cout<<mat1<<endl; sort(mat1.begin(), mat1.end()); cout<<"Sorted Data:"<<endl; cout<<mat1<<endl; Start End

27. Samples Samples continues... Using Column Major iterator: Original Data: 6 -9 -8 -1 -8 -6 -9 -2 -2 -5 -6 -4 2 3 -4 -8 -2 1 -7 5 5 -8 1 7 Sorted Data: -9 -7 -2 2 -9 -6 -2 3 -8 -6 -2 5 -8 -5 -1 5 -8 -4 1 6 -8 -4 1 7 //Column Major sorting using STL Sort() mat1=matori; cout<<mat1<<endl; sort(mat1.cbegin(), mat1.cend()); cout<<"Sorted Data:"<<endl; cout<<mat1<<endl; Start End

28. Samples Samples continues... Using super row iterator: Original Data: 6 -9 -8 -1 -8 -6 -9 -2 -2 -5 -6 -4 2 3 -4 -8 -2 1 -7 5 5 -8 1 7 Sorted Data: -9 -8 -1 6 -9 -8 -6 -2 -6 -5 -4 -2 -8 -4 2 3 -7 -2 1 5 -8 1 5 7 //Row by row sorting using STL Sort() mat1=matori; cout<<mat1<<endl; for(riter=mat1.r2rbegin();riter!=mat1.r2rend();riter++) { sort((*riter).begin(), (*riter).end()); } cout<<mat1<<endl;

29. Samples Samples continues... Using super column iterator: Original Data: 6 -9 -8 -1 -8 -6 -9 -2 -2 -5 -6 -4 2 3 -4 -8 -2 1 -7 5 5 -8 1 7 Sorted Data: -8 -9 -9 -8 -2 -8 -8 -4 -2 -6 -7 -2 2 -5 -6 -1 5 1 -4 5 6 3 1 7 //Column by column sorting using STL Sort() mat1=matori; cout<<mat1<<endl; for(citer=mat1.c2cbegin();citer!=mat1.c2cend();citer++) { sort((*citer).begin(), (*citer).end()); } cout<<mat1<<endl;

30. Samples Samples continues... Using Resize function: Original Data: 6 -9 -8 -1 -8 -6 -9 -2 -2 -5 -6 -4 2 3 -4 -8 -2 1 -7 5 5 -8 1 7 Sorted Data: 6 -9 -8 -1 0 0 -8 -6 -9 -2 0 0 -2 -5 -6 -4 0 0 2 3 -4 -8 0 0 -2 1 -7 5 0 0 5 -81 7 0 0 0 0 0 //Resizing the matrix mat1=matori; cout<<mat1<<endl; mat1.resize(8, 8, 0); cout<<mat1<<endl;

31. Improvement Morton array representation can be improved if we can utilize the wasted spaces for non-square matrices. This can be achieved to some extent by using partial interleaved bit patterns. –Portion of bits are interleaved and remaining bits are left as it is. This helps in utilizing the wasted space.

32. Improvement Improvement continues For example: Let us consider matrix of size 20 x 4 (actual reqd. space 80). Using Morton layout, it will require 1000001010 + 0000000101 = 1000001111=527+1 =528 spaces With modified version, it will require 1001010 + 0000101 = 1001111 = 79+1 = 80 spaces ->Improved !!!

33. Improvement Improvement continues More details… 1000001010->19 (row) + 0000000101 -> 3 (col) 1000001111->527 (Morton location) 100001010 -> 19 (row) + 000000101 -> 3 (col) 100001111 -> 79 (Improved Morton) Extra interleaving bits removed

34. Improvement Improvement continues In the improved version, only N bits are interleaved where N is total no. of bits in the smallest of total “row-1” and “column-1” in row x column matrix. For example, in 20x4 matrix, the smallest no. is 4 and 4-1=3 which is “11” in binary, that is N=2 as 3 is represented by 2 bits “11”.

35. Improvement Improvement continues Interleaving N bits and leaving remaining bits. For example, for rows=20-1=19=10011 100 10 10 ->2 bits are interleaved N=2 row interleaved bits. For columns=4-1=3=11 000 01 01 -> 2 bits are interleaved N=2 column interleaved bits.

36. Improvement Improvement continues Bit interleaved increment/decrement still works. –For bit interleaved Increment: 001 1010-> Bit interleaved 7 (111) OR000 0101-> Bit Mask 001 1111 + 1 010 0000 AND111 1010-> ~ Bit Mask (complement) 010 0000-> Bit interleaved 8 (1000)

37. Improvement Improvement continues –For bit interleaved Decrement: 010 0000-> Bit interleaved 8 (1000) - 1 001 1111 AND111 1010-> ~ Bit Mask 001 1010-> Bit interleaved 7 (111)

38. Improvement Improvement continues Improved array location is calculated by adding partial bit interleaved row and column. 100 10 10 -> 19 +000 01 01 -> 3 100 11 11 = 79 This method utilizes the wasted space to some extent but it does not work better than original Morton layout for square matrix which are not power of 2.

39. Improvement Improvement continues Improvement for square matrices: –Lets consider matrix NxN and say we want n bits to be interleaved. There is no change in the remaining bits of column bit patterns but for row bit patterns, remaining bits will have special bit patterns which are multiple of  N/2 n . So, separate lookuptables are required for row and column bit patterns. –Row bit and column bit patterns are added to get the modified storage location.

40. Improvement Improvement continues For example, 17x17 matrix with n=2 interleaved bits (actual 289 spaces reqd.): –Space required by normal Morton Layout will be 1000000000+ 0100000000=1100000000 =768+1=769 –With Improved version, we have,  17/2 2  =5 Row LookuptableCol Lookuptable 0000 000000000 0000 0000 001010000 0001 0000 100020000 0100 0000 101030000 0101 0101 000040001 0000 0101 0010…5...0001 0001... Changed by 5 = 101

41. Improvement Improvement continues For 17x17 matrix, –16 from row lookuptable will be, 10100 0000 –16 from col lookuptable will be, 00100 0000 –Total space required will be, 10100 0000 + 00100 0000Improved!!! 11000 0000 -> 384 + 1=385 spaces reqd.

42. Improvement Improvement continues This technique used for the square matrix still leaves some extra space as shown in the example of 17x17 matrix. In some cases, it even works perfectly. However its an improvement over Morton layout for square matrices which are not power of 2.

43. Improvement Improvement continues Generalized improvement for both square and non-square matrices: –Each row and column have respective partially interleaved bit patterns. –Either row or column whichever is greater, will have some non-interleaved and some special bit patterns. –Different lookup tables for rows and columns are required to implement.

44. Improvement Improvement continues –Let’s consider matrix of RxC with n interleaved bits then r=  R/2 n  and c=  C/2 n  –If r>c, row will have i regular non-interleaved bits and some special bit patterns of multiple of j, or vice versa. –If r>c: For Row: For Column: Multiple of j bit patterni regular remaining bitsn interleaved bits Remaining bits

45. Improvement Improvement continues –For r>c, i  abs(r - cx2 i ) is the least where i =1, 2, 3,.….. j =  MAX(r/2 i, c)  –For c>r, i  abs(c - rx2 i ) is the least where i =1, 2, 3,.….. j =  MAX(r, c/2 i ) 

46. Improvement Improvement continues –For example, consider 70x13 matrix with n=2 interleaved bits (actually 910 spaces required). Space required by normal Morton Layout will be, 10000000100010 + 00000001010000 = 10000001110010=8306+1=8307 Here,R=70, C=13, r=  70/2 2  and c=  13/2 2  We have, r>c, When i=1, abs(r - cx2 1 )=10 When i=2, abs(r - cx2 2 )=2 When i=3, abs(r - cx2 3 )=14  i=2 (only used by row in this case)  j=  MAX(r/2 2, c)  =5

47. Improvement Improvement continues –Row LookuptableCol Lookuptable 00000 00 0000000000 00 0000 00000 00 0010100000 00 0001 00000 00 1000200000 00 0100 00000 00 1010300000 00 0101 00000 01 0000400001 00 0000 00000 01 0010…5...00001 00 0001… ……… 00000 11 101015 00101 00 0000...16 Changed by 5 = 101 Only used by Row because row > col

48. Improvement Improvement continues For 70x13 matrix, –69 from row lookuptable will be, 10100 01 0010 –12 from col lookuptable will be, 00011 00 0000 –Total space required will be, 10100 01 0010 + 00011 00 0000 Improved!!! 10111 01 0010 -> 1490 + 1=1491 spaces

49. Recommendations Devise more efficient algorithms to utilize the wasted spaces by Morton Array Layout. If an optimal compromised algorithm is devised which works with both non- square and square matrices, then it could be new research paper or graduate research project.

50. Conclusion Morton Array Layout and its variant to improve the wasted spaces by Morton Layout was implemented in C++. Improvements on Morton Layout such as improvement for non-square and square matrices was introduced. But still optimal algorithm is to be researched.

51. Conclusion Conclusion continues C++ header file of Morton Array Layout matrix class can be downloaded and evaluated from: http://www.sharad.info/cs691 For any defects or feedback regarding this header file, please email me at sharadb@bgnet.bgsu.edu

52. Any Questions ?

53. Thank You !