1. Reminder: Array Representation In C++b c d e f g h i j k l
x[]
This representation is called the array-of-arrays representation.
Requires contiguous memory of size 3, 4, 4, and 4 for the 4 1D arrays.
1 memory block of size number of rows and number of rows blocks of size number of columns

2. Reminder: Row-Major Mapping
Example 3 x 4 array:
a b c d
e f g h
i j k l
Convert into 1D array y by collecting elements by rows.
Within a row elements are collected from left to right.
Rows are collected from top to bottom.
We get y[] = {a, b, c, d, e, f, g, h, i, j, k, l}
Used in Pascal and Fortran for example.
row 0
row 1
row 2 …
row i

3. Row-Major Used in Google’s TensorFlow From code comments on GitHub:
"For all types other than TF_STRING, the data buffer stores elements in row major order. “
Source:

4. sparse … many elements are zero dense … few elements are zero
Sparse Matrices
sparse … many elements are zero
dense … few elements are zero

5. Example Of Sparse Matrices
diagonal
tridiagonal
lower triangular (?)
These are structured sparse matrices.
May be mapped into a 1D array so that a mapping function can be used to locate an element.
element (i,j) is at position i(i-1)/2 + j -1 for a lower triangular matrix using row-major representation

6. Unstructured Sparse Matrices
Airline flight matrix.
airports are numbered 1 through n
flight(i,j) = list of nonstop flights from airport i to airport j
n = 1000 (say)
n x n array of list references => 4 million bytes
total number of flights = 20,000 (say)
need at most 20,000 list references => at most 80,000 bytes
Assume that flight(i,j) = null if there is no flight from i to j. Of the 1 million entries in the flight matrix at most 20,000 can be nonnull as each nonnull entry represents at least 1 different flight. So we need to store at most 20,000 nonnull entries.

7. Unstructured Sparse Matrices
Web page matrix.
web pages are numbered 1 through n
web(i,j) = number of links from page i to page j
Web analysis.
authority page … page that has many links to it
hub page … links to many authority pages
Web page analyses are done using a matrix such as web(*,*). See IEEE Computer, August 1999, p 60, for a paper that describes Web analysis based on such a matrix. Operations such as matrix transpose and multiply are used.

8. Web Page Matrix n = 2 billion (and growing by 1 million a day)
n x n array of ints => 4* 1018 integers = 16* 1018 bytes (16 * 109 GB)
each page links to 10 (say) other pages on average
on average there are 10 nonzero entries per row
space needed for nonzero elements is approximately 20 billion x 4 bytes = 80 billion bytes (80 GB)
On average, 10 entries in each row are nonzero.

9. A Sparse Matrix
An example of a sparse matrix: (2 nonzero elements, 10 zero elements)
[[1, 0, 0, 0]
[0, 0, 2, 0]
[0, 0, 0, 0]]

10. Representation Of Unstructured Sparse Matrices
Single linear list in row-major order.
scan the nonzero elements of the sparse matrix in row-major order
each nonzero element is represented by a triple
(row, column, value)
the list of triples may be an array list or a linked list (chain)

11. Single Linear List Example
list =
row
column
value
The single linear list has 6 elements. Each list element is a triple (row, column, value).

12. Array Linear List Representation
row
list = column
value
element
row
column
value
The list and array linear list representation drawings are essentially the same, because of the identity mapping (element[i] = list element i) that is used.

13. Chain Representation Node structure. row col next value
The drawing, above, assumes a custom class for matrices. This custom class does not use the class chain.
If you want to use the class chain, the fields row, col, and value need to be in another
class, say data. The elements of chain will have the data type data.

14. Single Chain row 1 1 2 2 4 4 list = column 3 5 3 4 2 3
value 1 3 5 4 2 7 6
NULL
firstNode

15. One Linear List Per Row row1 = [(3, 3), (5,4)] 0 0 3 0 4
Array linear list per row.

16. Array Of Row Chains Node structure. next value col
The above drawing assumes a custom class for sparse matrices. If you want to use the class chain, the col and value fields will be members of some other class (say) data. The data type of the elements in the chains will be data.

17. Array Of Row Chains 0 0 3 0 4 0 0 5 7 0 0 0 0 0 0 0 2 6 0 0 row[] 3 4
NULL 4 5 5 3
NULL 7 4
NULL
Similar to adjacency lists 2
NULL 6 3

18. Orthogonal List Representation
Both row and column lists.
Node structure.
row
col
next
down
value

19. Row Lists
NULL 1 3 5 4 2 7 6 N

20. Column Lists 1 3 5 4 2 7 6 N

21. Orthogonal Lists 0 0 3 0 4 0 0 5 7 0 0 0 0 0 0 0 2 6 0 0 row[] 1 3 5 4
NULL
row[] 1 3 5 4 2 7 6 N

22. May use circular lists instead of chains.
Variations
May use circular lists instead of chains.
Or, even doubly linked circular lists.

23. Approximate Memory Requirements
500 x 500 matrix with 1994 nonzero elements
2D array x 500 x 4 = 1million bytes
Single Array List 3 x 1994 x 4 = 23,928 bytes
One Chain Per Row x 4 = 25,928

24. Runtime Performance Matrix Transpose
500 x 500 matrix with 1994 nonzero elements
2D array ms
Single Array List ms
One Chain Per Row ms

25. Performance Matrix Addition.
500 x 500 matrices with 1994 and 999 nonzero elements
2D array ms
Single Array List ms
One Chain Per Row Not Measured