1. Program Structure

2. r Memory management is designed to be transparent to the user program r As a programmer you typically do not think about how memory is organized.

3. Example 1 Consider the following code segments: int data[128][128]; for (j = 0; j <128; j++) for (i = 0; i < 128;i++) data[i,j] = 0; Does it matter which you use? int data[128][128]; for (i = 0; i <128; i++) for (j = 0; j < 128;j++) data[i,j] = 0;

4. Example 1 data[1][0] data[1][1]data[1][2] … … data[1][127] With 128 word pages (1 word is one integer) Each row is stored in one page (row major order) data[0][0]data[0][1]data[0][2] … … data[0][127] data[127][0]data[127][1]data[127][2] … … data[127][127] … Page 0 Page 1 Page 127

5. Example 1 int data[128][128]; for (j = 0; j <128; j++) for (i = 0; i < 128;i++) data[i,j] = 0; r Access pattern when j is 0: data[0,0], data[1,0], data[2,0] r Are data[0,0] and data[1,0] on the same page? m No r So while j is 0 we have 128 page faults.

6. Example 1 int data[128][128]; for (j = 0; j <128; j++) for (i = 0; i < 128;i++) data[i,j] = 0; r Access pattern when j is 1: data[0,1], data[1,1], data[2,1] r Are these on the same page? No r Now what if the OS allocates fewer than 128 frames m You will page fault on data[0,1] r Number of total page faults is 128*128 = 16,384

7. Example 2 Consider the following code segments: for (i = 0; i < n; i++) for (j = 0; j < n; j++) for (k = 0; k < n; k++) c[i,j] = c[i,j] + a[i,k] * b[k,j]; Which one? for (i = 0; i < n; i++) for (k = 0; k < n; k++) for (j = 0; j < n; j++) c[i,j] = c[i,j] + a[i,k] * b[k,j];

8. Example 2 for (i = 0; i < n; i++) for (j = 0; j < n; j++) for (k = 0; k < n; k++) c[i,j] = c[i,j] + a[i,k] * b[k,j]; r k changes for each iteration r Access pattern for b[k,j] when i is 0, j is 0 is: b[0,0],b[1,0],b[2,0] etc m By incrementing k we are skipping over an entire row of the matrix r Access pattern for c[i,j] when i is 0, j is 0 and for any value of k is c[0,0] r Access pattern for a[i,k] when i is 0 is: a[0,0], a[0,1],…. m All accesses come from the same row

9. Example 2 for (i = 0; i < n; i++) for (k = 0; k < n; k++) for (j = 0; j < n; ++) c[i,j] = c[i,j] + a[i,k] * b[k,j]; r j changes for each iteration r Access pattern for b[k,j] when i is 0, k is 0 is: b[0,0],b[0,1],b[0,2] etc m All accesses are from the same row r Access pattern for c[i,j] when i is 0, k is 0 is: c[0,1], c[0,1], c[0,2] etc m All accesses are from the same row r Access pattern for a[i,k] when i is 0, k is 0 and any j is: a[0,0].

10. Program Structure r Careful selection of data structures and programming structures can increase locality and hence lower the page fault- rate

11. Singly-linked lists vs. 1D-arrays ArraySingly-linked list Fixed size: Resizing is expensiveDynamic size Insertions and Deletions are inefficient: Elements are usually shifted Insertions and Deletions are efficient: No shifting Random access i.e., efficient indexingNo random access  Not suitable for operations requiring accessing elements by index such as sorting No memory waste if the array is full or almost full; otherwise may result in much memory waste. Extra storage needed for references; however uses exactly as much memory as it needs Sequential access is faster because of greater locality of references [Reason: Elements in contiguous memory locations] Sequential access is slow because of low locality of references [Reason: Elements not in contiguous memory locations]

12. Other r Stack has good locality r Hash table scatters references for poor locality

13. Summary r We have briefly reviewed issues related to program structure.